Project

General

Profile

Forums » Support »

Regridding

Added by Elvis Asong over 10 years ago

Hi,
I am trying to do regridding with CDO.
However, I do not know how to order my models.
I want to regrid GFDL GCM grids to NCEP grids.

I would like to use bilinear interpolation.
Question: which should come first?
$ cdo... GFDL NCEP

or

$ cdo... NECP GFDL

Thanks for your useful contributions.
Elvis.


Replies (8)

Sum half-hourly data to 6-hourly - Added by Elvis Asong about 8 years ago

OS=LINUX

Hi All,
I have several netCDF files and each file is separated from the other by 30 minutes time step. The start time for accumulation is 00:00:00–00:29:59 UTC. I would like to sum the data by date:

[1] "2015-04-01 00:00:00 CST" "2015-04-01 00:30:00 CST" "2015-04-01 01:00:00 CST" "2015-04-01 01:30:00 CST"
[5] "2015-04-01 02:00:00 CST" "2015-04-01 02:30:00 CST" "2015-04-01 03:00:00 CST" "2015-04-01 03:30:00 CST"
[9] "2015-04-01 04:00:00 CST" "2015-04-01 04:30:00 CST" "2015-04-01 05:00:00 CST" "2015-04-01 05:30:00 CST"
[13] "2015-04-01 06:00:00 CST" "2015-04-01 06:30:00 CST" "2015-04-01 07:00:00 CST" "2015-04-01 07:30:00 CST"
[17] "2015-04-01 08:00:00 CST" "2015-04-01 08:30:00 CST" "2015-04-01 09:00:00 CST" "2015-04-01 09:30:00 CST"
[21] "2015-04-01 10:00:00 CST" "2015-04-01 10:30:00 CST" "2015-04-01 11:00:00 CST" "2015-04-01 11:30:00 CST"
[25] "2015-04-01 12:00:00 CST" "2015-04-01 12:30:00 CST" "2015-04-01 13:00:00 CST" "2015-04-01 13:30:00 CST"
[29] "2015-04-01 14:00:00 CST" "2015-04-01 14:30:00 CST" "2015-04-01 15:00:00 CST" "2015-04-01 15:30:00 CST"
[33] "2015-04-01 16:00:00 CST" "2015-04-01 16:30:00 CST" "2015-04-01 17:00:00 CST" "2015-04-01 17:30:00 CST"
[37] "2015-04-01 18:00:00 CST" "2015-04-01 18:30:00 CST" "2015-04-01 19:00:00 CST" "2015-04-01 19:30:00 CST"
[41] "2015-04-01 20:00:00 CST" "2015-04-01 20:30:00 CST" "2015-04-01 21:00:00 CST" "2015-04-01 21:30:00 CST"
[45] "2015-04-01 22:00:00 CST" "2015-04-01 22:30:00 CST" "2015-04-01 23:00:00 CST" "2015-04-01 23:30:00 CST"
[49] "2015-04-02 00:00:00 CST" "2015-04-02 00:30:00 CST" "2015-04-02 01:00:00 CST" "2015-04-02 01:30:00 CST"
[53] "2015-04-02 02:00:00 CST" "2015-04-02 02:30:00 CST" "2015-04-02 03:00:00 CST" "2015-04-02 03:30:00 CST"
[57] "2015-04-02 04:00:00 CST" "2015-04-02 04:30:00 CST" "2015-04-02 05:00:00 CST" "2015-04-02 05:30:00 CST"
[61] "2015-04-02 06:00:00 CST" "2015-04-02 06:30:00 CST" "2015-04-02 07:00:00 CST" "2015-04-02 07:30:00 CST"
[65] "2015-04-02 08:00:00 CST" "2015-04-02 08:30:00 CST" "2015-04-02 09:00:00 CST" "2015-04-02 09:30:00 CST"
[69] "2015-04-02 10:00:00 CST" "2015-04-02 10:30:00 CST" "2015-04-02 11:00:00 CST" "2015-04-02 11:30:00 CST"
[73] "2015-04-02 12:00:00 CST" "2015-04-02 12:30:00 CST" "2015-04-02 13:00:00 CST" "2015-04-02 13:30:00 CST"
[77] "2015-04-02 14:00:00 CST" "2015-04-02 14:30:00 CST" "2015-04-02 15:00:00 CST" "2015-04-02 15:30:00 CST"
[81] "2015-04-02 16:00:00 CST" "2015-04-02 16:30:00 CST" "2015-04-02 17:00:00 CST" "2015-04-02 17:30:00 CST"
[85] "2015-04-02 18:00:00 CST" "2015-04-02 18:30:00 CST" "2015-04-02 19:00:00 CST" "2015-04-02 19:30:00 CST"
[89] "2015-04-02 20:00:00 CST" "2015-04-02 20:30:00 CST" "2015-04-02 21:00:00 CST" "2015-04-02 21:30:00 CST"
[93] "2015-04-02 22:00:00 CST" "2015-04-02 22:30:00 CST" "2015-04-02 23:00:00 CST" "2015-04-02 23:30:00 CST"
[97] "2015-04-03 00:00:00 CST" "2015-04-03 00:30:00 CST" "2015-04-03 01:00:00 CST" "2015-04-03 01:30:00 CST"
[101] "2015-04-03 02:00:00 CST" "2015-04-03 02:30:00 CST" "2015-04-03 03:00:00 CST" "2015-04-03 03:30:00 CST"
[105] "2015-04-03 04:00:00 CST" "2015-04-03 04:30:00 CST" "2015-04-03 05:00:00 CST" "2015-04-03 05:30:00 CST"
[109] "2015-04-03 06:00:00 CST" "2015-04-03 06:30:00 CST" "2015-04-03 07:00:00 CST" "2015-04-03 07:30:00 CST"
[113] "2015-04-03 08:00:00 CST" "2015-04-03 08:30:00 CST" "2015-04-03 09:00:00 CST" "2015-04-03 09:30:00 CST"
[117] "2015-04-03 10:00:00 CST" "2015-04-03 10:30:00 CST" "2015-04-03 11:00:00 CST" "2015-04-03 11:30:00 CST"
[121] "2015-04-03 12:00:00 CST" "2015-04-03 12:30:00 CST" "2015-04-03 13:00:00 CST" "2015-04-03 13:30:00 CST"
[125] "2015-04-03 14:00:00 CST" "2015-04-03 14:30:00 CST" "2015-04-03 15:00:00 CST" "2015-04-03 15:30:00 CST"
[129] "2015-04-03 16:00:00 CST" "2015-04-03 16:30:00 CST" "2015-04-03 17:00:00 CST" "2015-04-03 17:30:00 CST"
[133] "2015-04-03 18:00:00 CST" "2015-04-03 18:30:00 CST" "2015-04-03 19:00:00 CST" "2015-04-03 19:30:00 CST"
[137] "2015-04-03 20:00:00 CST" "2015-04-03 20:30:00 CST" "2015-04-03 21:00:00 CST" "2015-04-03 21:30:00 CST"
[141] "2015-04-03 22:00:00 CST" "2015-04-03 22:30:00 CST" "2015-04-03 23:00:00 CST" "2015-04-03 23:30:00 CST"

such that the above becomes:

2015-01-01 00:00:00
 2015-01-01 06:00:00
 2015-01-01 12:00:00
 2015-01-01 18:00:00
 2015-01-02 00:00:00
 2015-01-02 06:00:00 and so on ...

For example, if my data is sorted by time, I would have 12 "30 mins" time steps in 6 hours. Thus an index=1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 ... will suffice. Thus, sum all ones and you get "2015-01-01 00:00:00 UTC" , sum all twos and you get "2015-01-01 06:00:00 UTC", sum all threes and you get "2015-01-01 12:00:00 UTC" etc.

QUESTION:

- How can I implement this within CDO? I am very new to CDO commands.
- My files have no record dimension so I will like to add one to all files first.
- Then sum the 30mins timestep data to 6-hourly time step.
- The ofile should have a "time" variable with the 6-hourly time step. I can merge the ofiles using cdo mergetime *.nc4 outfile

The data files for illustration can be found here (and in the attached file list):
http://disc.sci.gsfc.nasa.gov/SSW/#keywords=GPM_3IMERGHH 03

Select 2015-06-01 to 2015-06-30 for Date Range. Spatial Bounding Box is empty. Running the following code on Linux assuming the attached file is in your current working directory the data will be downloaded quickly.

egrep -o '3B.*\.nc4' SSW_download_2016-02-26T05-59-11_65188_nM5bYSUv.inp >temp1.txt;paste SSW_download_2016-02-26T05-59-11_65188_nM5bYSUv.inp temp1.txt > temp2.txt ; awk 'system("wget "$1" -O "$2" ")' temp2.txt

SSW_download_2016-02-26T05-59-11_65188_nM5bYSUv.inp (332 KB) SSW_download_2016-02-26T05-59-11_65188_nM5bYSUv.inp file list you can download for illustration.

RE: Sum half-hourly data to 6-hourly - Added by Elvis Asong about 8 years ago

Elvis Asong wrote:

OS=LINUX

Hi All,
I have several netCDF4 files and each file is separated from the other by 30 minutes time step. The start time for accumulation is 00:00:00–00:29:59 UTC. I would like to sum the data by date:

[1] "2015-04-01 00:00:00 CST" "2015-04-01 00:30:00 CST" "2015-04-01 01:00:00 CST" "2015-04-01 01:30:00 CST"
[5] "2015-04-01 02:00:00 CST" "2015-04-01 02:30:00 CST" "2015-04-01 03:00:00 CST" "2015-04-01 03:30:00 CST"
[9] "2015-04-01 04:00:00 CST" "2015-04-01 04:30:00 CST" "2015-04-01 05:00:00 CST" "2015-04-01 05:30:00 CST"
[13] "2015-04-01 06:00:00 CST" "2015-04-01 06:30:00 CST" "2015-04-01 07:00:00 CST" "2015-04-01 07:30:00 CST"
[17] "2015-04-01 08:00:00 CST" "2015-04-01 08:30:00 CST" "2015-04-01 09:00:00 CST" "2015-04-01 09:30:00 CST"
[21] "2015-04-01 10:00:00 CST" "2015-04-01 10:30:00 CST" "2015-04-01 11:00:00 CST" "2015-04-01 11:30:00 CST"
[25] "2015-04-01 12:00:00 CST" "2015-04-01 12:30:00 CST" "2015-04-01 13:00:00 CST" "2015-04-01 13:30:00 CST"
[29] "2015-04-01 14:00:00 CST" "2015-04-01 14:30:00 CST" "2015-04-01 15:00:00 CST" "2015-04-01 15:30:00 CST"
[33] "2015-04-01 16:00:00 CST" "2015-04-01 16:30:00 CST" "2015-04-01 17:00:00 CST" "2015-04-01 17:30:00 CST"
[37] "2015-04-01 18:00:00 CST" "2015-04-01 18:30:00 CST" "2015-04-01 19:00:00 CST" "2015-04-01 19:30:00 CST"
[41] "2015-04-01 20:00:00 CST" "2015-04-01 20:30:00 CST" "2015-04-01 21:00:00 CST" "2015-04-01 21:30:00 CST"
[45] "2015-04-01 22:00:00 CST" "2015-04-01 22:30:00 CST" "2015-04-01 23:00:00 CST" "2015-04-01 23:30:00 CST"
[49] "2015-04-02 00:00:00 CST" "2015-04-02 00:30:00 CST" "2015-04-02 01:00:00 CST" "2015-04-02 01:30:00 CST"
[53] "2015-04-02 02:00:00 CST" "2015-04-02 02:30:00 CST" "2015-04-02 03:00:00 CST" "2015-04-02 03:30:00 CST"
[57] "2015-04-02 04:00:00 CST" "2015-04-02 04:30:00 CST" "2015-04-02 05:00:00 CST" "2015-04-02 05:30:00 CST"
[61] "2015-04-02 06:00:00 CST" "2015-04-02 06:30:00 CST" "2015-04-02 07:00:00 CST" "2015-04-02 07:30:00 CST"
[65] "2015-04-02 08:00:00 CST" "2015-04-02 08:30:00 CST" "2015-04-02 09:00:00 CST" "2015-04-02 09:30:00 CST"
[69] "2015-04-02 10:00:00 CST" "2015-04-02 10:30:00 CST" "2015-04-02 11:00:00 CST" "2015-04-02 11:30:00 CST"
[73] "2015-04-02 12:00:00 CST" "2015-04-02 12:30:00 CST" "2015-04-02 13:00:00 CST" "2015-04-02 13:30:00 CST"
[77] "2015-04-02 14:00:00 CST" "2015-04-02 14:30:00 CST" "2015-04-02 15:00:00 CST" "2015-04-02 15:30:00 CST"
[81] "2015-04-02 16:00:00 CST" "2015-04-02 16:30:00 CST" "2015-04-02 17:00:00 CST" "2015-04-02 17:30:00 CST"
[85] "2015-04-02 18:00:00 CST" "2015-04-02 18:30:00 CST" "2015-04-02 19:00:00 CST" "2015-04-02 19:30:00 CST"
[89] "2015-04-02 20:00:00 CST" "2015-04-02 20:30:00 CST" "2015-04-02 21:00:00 CST" "2015-04-02 21:30:00 CST"
[93] "2015-04-02 22:00:00 CST" "2015-04-02 22:30:00 CST" "2015-04-02 23:00:00 CST" "2015-04-02 23:30:00 CST"
[97] "2015-04-03 00:00:00 CST" "2015-04-03 00:30:00 CST" "2015-04-03 01:00:00 CST" "2015-04-03 01:30:00 CST"
[101] "2015-04-03 02:00:00 CST" "2015-04-03 02:30:00 CST" "2015-04-03 03:00:00 CST" "2015-04-03 03:30:00 CST"
[105] "2015-04-03 04:00:00 CST" "2015-04-03 04:30:00 CST" "2015-04-03 05:00:00 CST" "2015-04-03 05:30:00 CST"
[109] "2015-04-03 06:00:00 CST" "2015-04-03 06:30:00 CST" "2015-04-03 07:00:00 CST" "2015-04-03 07:30:00 CST"
[113] "2015-04-03 08:00:00 CST" "2015-04-03 08:30:00 CST" "2015-04-03 09:00:00 CST" "2015-04-03 09:30:00 CST"
[117] "2015-04-03 10:00:00 CST" "2015-04-03 10:30:00 CST" "2015-04-03 11:00:00 CST" "2015-04-03 11:30:00 CST"
[121] "2015-04-03 12:00:00 CST" "2015-04-03 12:30:00 CST" "2015-04-03 13:00:00 CST" "2015-04-03 13:30:00 CST"
[125] "2015-04-03 14:00:00 CST" "2015-04-03 14:30:00 CST" "2015-04-03 15:00:00 CST" "2015-04-03 15:30:00 CST"
[129] "2015-04-03 16:00:00 CST" "2015-04-03 16:30:00 CST" "2015-04-03 17:00:00 CST" "2015-04-03 17:30:00 CST"
[133] "2015-04-03 18:00:00 CST" "2015-04-03 18:30:00 CST" "2015-04-03 19:00:00 CST" "2015-04-03 19:30:00 CST"
[137] "2015-04-03 20:00:00 CST" "2015-04-03 20:30:00 CST" "2015-04-03 21:00:00 CST" "2015-04-03 21:30:00 CST"
[141] "2015-04-03 22:00:00 CST" "2015-04-03 22:30:00 CST" "2015-04-03 23:00:00 CST" "2015-04-03 23:30:00 CST"

such that the above becomes:

2015-04-01 00:00:00
2015-04-01 06:00:00
2015-04-01 12:00:00
2015-04-01 18:00:00
2015-04-02 00:00:00
2015-04-02 06:00:00 and so on ...

For example, if my data is sorted by time, I would have 12 "30 mins" time steps in 6 hours. Thus an index=1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 ... will suffice. Thus, sum all ones and you get "2015-01-01 00:00:00 UTC" , sum all twos and you get "2015-01-01 06:00:00 UTC", sum all threes and you get "2015-01-01 12:00:00 UTC" etc.

QUESTION:

- How can I implement this within CDO? I am very new to CDO commands.
- My files have no record dimension so I will like to add one to all files first.
- Then sum the 30mins timestep data to 6-hourly time step.
- The ofile should have a "time" variable with the 6-hourly time step. I can merge the ofiles using cdo mergetime *.nc4 outfile

The data files for illustration can be found here (and in the attached file list):
http://disc.sci.gsfc.nasa.gov/SSW/#keywords=GPM_3IMERGHH 03

Select 2015-06-01 to 2015-06-30 for Date Range. Spatial Bounding Box is empty. Running the following code on Linux assuming the attached file is in your current working directory the data will be downloaded quickly.

egrep -o '3B.*\.nc4' SSW_download_2016-02-26T05-59-11_65188_nM5bYSUv.inp >temp1.txt;paste SSW_download_2016-02-26T05-59-11_65188_nM5bYSUv.inp temp1.txt > temp2.txt ; awk 'system("wget "$1" -O "$2" ")' temp2.txt

SSW_download_2016-02-26T05-59-11_65188_nM5bYSUv.inp (332 KB) SSW_download_2016-02-26T05-59-11_65188_nM5bYSUv.inp

RE: Regridding - Added by Ralf Mueller about 8 years ago

Hi!
I downloaded all the files. There seems to be not time information in the data at all, only in the filename. But the time information is formatted in a way that lexical order is equal to temporal order.

If I got it right, you want to collect 6 consecutive files into one, right? This can be done in two different ways:

  1. cat everything together and split it into groups of 6: At the end, you will not have any reasonable time information within the files, hence using CDOs time-basd operators (timmean,timmax,timsort,...) won't work.
  2. create a correct timeaxis for each of the files and then do the above.

I did this for a subset of the files

cdo -f nc4  -cat '3B-HHR.MS.MRG.3IMERG.20150621*'  20150621.nc4
cdo -f nc4 -settaxis,2015-06-21,0,30min  20150621.nc4 20150621_withTime.nc4
cdo -f nc4 -splitsel,6 20150621_withTime.nc4 20150621_withTime_groupsOf6.nc4

RE: Regridding - Added by Elvis Asong about 8 years ago

Ralf Mueller wrote:

Hi!
I downloaded all the files. There seems to be not time information in the data at all, only in the filename. But the time information is formatted in a way that lexical order is equal to temporal order.

If I got it right, you want to collect 6 consecutive files into one, right? This can be done in two different ways:

  1. cat everything together and split it into groups of 6: At the end, you will not have any reasonable time information within the files, hence using CDOs time-basd operators (timmean,timmax,timsort,...) won't work.
  2. create a correct timeaxis for each of the files and then do the above.

I did this for a subset of the files

[...]

Oh wow! Thanks Ralf.
I think the last line of code should split into groups of 12. From 00:00:00 to 05:30:00 AM has 12 30mins time steps. Thus it adds up to 6hours.
cdo -f nc4 -splitsel,12 20150621_withTime.nc4 20150621_withTime_groupsOf6.nc4

The last line split the files into different files but this is not what I was expecting.

From line 2 of the code:
cdo -f nc4 -settaxis,2015-06-21,0,30min 20150621.nc4 20150621_withTime.nc4 #####Please use more files if you can

I obtain these dates:

cdo sinfop mergedfiles_withTime.nc4

YYYY-MM-DD hh:mm:ss  YYYY-MM-DD hh:mm:ss  YYYY-MM-DD hh:mm:ss  YYYY-MM-DD hh:mm:ss
  2015-04-01 00:00:00  2015-04-01 00:30:00  2015-04-01 01:00:00  2015-04-01 01:30:00
  2015-04-01 02:00:00  2015-04-01 02:30:00  2015-04-01 03:00:00  2015-04-01 03:30:00
  2015-04-01 04:00:00  2015-04-01 04:30:00  2015-04-01 05:00:00  2015-04-01 05:30:00
  2015-04-01 06:00:00  2015-04-01 06:30:00  2015-04-01 07:00:00  2015-04-01 07:30:00
  2015-04-01 08:00:00  2015-04-01 08:30:00  2015-04-01 09:00:00  2015-04-01 09:30:00
  2015-04-01 10:00:00  2015-04-01 10:30:00  2015-04-01 11:00:00  2015-04-01 11:30:00
  2015-04-01 12:00:00  2015-04-01 12:30:00  2015-04-01 13:00:00  2015-04-01 13:30:00
  2015-04-01 14:00:00  2015-04-01 14:30:00  2015-04-01 15:00:00  2015-04-01 15:30:00
  2015-04-01 16:00:00  2015-04-01 16:30:00  2015-04-01 17:00:00  2015-04-01 17:30:00
  2015-04-01 18:00:00  2015-04-01 18:30:00  2015-04-01 19:00:00  2015-04-01 19:30:00
  2015-04-01 20:00:00  2015-04-01 20:30:00  2015-04-01 21:00:00  2015-04-01 21:30:00
  2015-04-01 22:00:00  2015-04-01 22:30:00  2015-04-01 23:00:00  2015-04-01 23:30:00
  2015-04-02 00:00:00  2015-04-02 00:30:00  2015-04-02 01:00:00  2015-04-02 01:30:00
  2015-04-02 02:00:00  2015-04-02 02:30:00  2015-04-02 03:00:00  2015-04-02 03:30:00
  2015-04-02 04:00:00  2015-04-02 04:30:00  2015-04-02 05:00:00  2015-04-02 05:30:00

What I would like line 3 of the code to do is:

sum all data for these dates: 

2015-04-01 00:00:00  2015-04-01 00:30:00  2015-04-01 01:00:00  2015-04-01 01:30:00
  2015-04-01 02:00:00  2015-04-01 02:30:00  2015-04-01 03:00:00  2015-04-01 03:30:00
  2015-04-01 04:00:00  2015-04-01 04:30:00  2015-04-01 05:00:00  2015-04-01 05:30:00

and store it in the date

2015-04-01 00:00:00 #### date of first row is the new date

sum all data for these dates: 

2015-04-01 06:00:00  2015-04-01 06:30:00  2015-04-01 07:00:00  2015-04-01 07:30:00
  2015-04-01 08:00:00  2015-04-01 08:30:00  2015-04-01 09:00:00  2015-04-01 09:30:00
  2015-04-01 10:00:00  2015-04-01 10:30:00  2015-04-01 11:00:00  2015-04-01 11:30:00
and store it in the date

2015-04-01 06:00:00 and so on.

The resulting ofile is ONE BIG FILE containing rainfall accumulations every 6 hours:

2015-04-01 00:00:00
2015-04-01 06:00:00
2015-04-01 12:00:00
2015-04-01 18:00:00
2015-04-02 00:00:00 and so on.

Thanks.

RE: Regridding - Added by Ralf Mueller about 8 years ago

you can create a 6-hourly sum of the large dataset with

CDO_TIMESTAT_DATE=last  cdo -f nc4 -z zip9 -timselsum,12 20150621_withTime.nc4 6hour_sum.nc4

but this will lead to timesteps like

 2015-06-21 05:30:00  2015-06-21 11:30:00  2015-06-21 17:30:00  2015-06-21 23:30:00

Now (or in combination with the above) you can shift the timesteps by 05:30 in the past with

CDO_TIMESTAT_DATE=last cdo -f nc4 -z zip9 -shifttime,-30min -shifttime,-5hour -timselsum,12 20150621_withTime.nc4 6hour_sum_shifted.nc4

this leads to the desired timesteps

 2015-06-21 00:00:00  2015-06-21 06:00:00  2015-06-21 12:00:00  2015-06-21 18:00:00

RE: Regridding - Added by Elvis Asong about 8 years ago

Great! Thanks Ralf. Hopefully the time shifting also shifts the values?

RE: Regridding - Added by Ralf Mueller about 8 years ago

It shifts the time-axes values, but the field values are not changed

RE: Regridding - Added by Elvis Asong about 8 years ago

Oh! OK. I was just curious to know that the field values correspond with the shifted time-axis.

    (1-8/8)