Force .nc files to be on the same grid when merging with "cdo merge"
Added by Joshua Miller 10 days ago
Hello, I have two files "weather.nc" and "precip.nc". They are from different datasets and I want to combine them. However, despite having the same grid type, grid coordinates, and times, CDO does not merge them onto the same grid correctly. Here are the metadata for both files:
cdo sinfo weather.nc
File format : NetCDF4
-1 : Institut Source T Steptype Levels Num Points Num Dtype : Parameter ID
1 : ECMWF unknown v instant 1 1 350000 1 F32 : -1.128
2 : ECMWF unknown v instant 1 1 350000 1 F32 : -2.128
3 : ECMWF unknown v instant 1 1 350000 1 F32 : -3.128
4 : ECMWF unknown v instant 1 1 350000 1 F32 : -4.128
5 : ECMWF unknown v instant 1 1 350000 1 F32 : -5.128
6 : ECMWF unknown v instant 1 1 350000 1 F32 : -6.128
7 : ECMWF unknown v instant 1 1 350000 1 F32 : -7.128
8 : ECMWF unknown v instant 1 1 350000 1 F32 : -8.128
9 : ECMWF unknown v instant 1 1 350000 1 F32 : -9.128
10 : ECMWF unknown v instant 1 1 350000 1 F32 : -10.128
11 : ECMWF unknown v instant 1 1 350000 1 F32 : -11.128
12 : ECMWF unknown v instant 1 1 350000 1 F32 : -12.128
13 : ECMWF unknown v instant 1 1 350000 1 F32 : -13.128
14 : ECMWF unknown v instant 1 1 350000 1 F32 : -14.128
15 : ECMWF unknown v instant 1 1 350000 1 F32 : -15.128
16 : ECMWF unknown v instant 1 1 350000 1 F32 : -16.128
17 : ECMWF unknown v instant 1 1 350000 1 F32 : -17.128
18 : ECMWF unknown v instant 1 1 350000 1 F32 : -18.128
19 : ECMWF unknown v instant 1 1 350000 1 F32 : -19.128
20 : ECMWF unknown v instant 1 1 350000 1 F32 : -20.128
21 : ECMWF unknown v instant 1 1 350000 1 F32 : -21.128
22 : ECMWF unknown v instant 1 1 350000 1 F32 : -22.128
23 : ECMWF unknown v instant 1 1 350000 1 F32 : -23.128
24 : ECMWF unknown v instant 1 1 350000 1 F32 : -24.128
Grid coordinates :
1 : lonlat : points=350000 (700x500)
lon : -24.95 to 44.95 by 0.1 degrees_east
lat : 69.95 to 20.05 by -0.1 degrees_north
Vertical coordinates :
1 : surface : levels=1
Time coordinate :
time : 24 steps
RefTime = 2002-01-01 00:00:00 Units = hours Calendar = proleptic_gregorian
YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss
2002-01-01 00:00:00 2002-01-01 01:00:00 2002-01-01 02:00:00 2002-01-01 03:00:00
2002-01-01 04:00:00 2002-01-01 05:00:00 2002-01-01 06:00:00 2002-01-01 07:00:00
2002-01-01 08:00:00 2002-01-01 09:00:00 2002-01-01 10:00:00 2002-01-01 11:00:00
2002-01-01 12:00:00 2002-01-01 13:00:00 2002-01-01 14:00:00 2002-01-01 15:00:00
2002-01-01 16:00:00 2002-01-01 17:00:00 2002-01-01 18:00:00 2002-01-01 19:00:00
2002-01-01 20:00:00 2002-01-01 21:00:00 2002-01-01 22:00:00 2002-01-01 23:00:00
cdo sinfo: Processed 24 variables over 24 timesteps [0.07s 50MB]
------------------------------------------------------------------------------------------
cdo sinfo precip.nc
File format : NetCDF4
-1 : Institut Source T Steptype Levels Num Points Num Dtype : Parameter ID
1 : unknown Huffman v instant 1 1 350000 1 F32 : -1
Grid coordinates :
1 : lonlat : points=350000 (700x500)
longitude : -24.95 to 44.95 by 0.1 degrees_east
latitude : 69.95 to 20.05 by -0.1 degrees_north
Vertical coordinates :
1 : surface : levels=1
Time coordinate :
time : 24 steps
RefTime = 1980-01-06 00:00:00 Units = seconds Calendar = gregorian
YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss
2002-01-01 00:00:00 2002-01-01 01:00:00 2002-01-01 02:00:00 2002-01-01 03:00:00
2002-01-01 04:00:00 2002-01-01 05:00:00 2002-01-01 06:00:00 2002-01-01 07:00:00
2002-01-01 08:00:00 2002-01-01 09:00:00 2002-01-01 10:00:00 2002-01-01 11:00:00
2002-01-01 12:00:00 2002-01-01 13:00:00 2002-01-01 14:00:00 2002-01-01 15:00:00
2002-01-01 16:00:00 2002-01-01 17:00:00 2002-01-01 18:00:00 2002-01-01 19:00:00
2002-01-01 20:00:00 2002-01-01 21:00:00 2002-01-01 22:00:00 2002-01-01 23:00:00
cdo sinfo: Processed 1 variable over 24 timesteps [0.14s 52MB]
-------------------------------------------------------------------------------------
All seems normal, but when I do "cdo merge weather.nc precip.nc output.nc", "cdo sinfo output.nc" gives:
File format : NetCDF4
-1 : Institut Source T Steptype Levels Num Points Num Dtype : Parameter ID
1 : ECMWF unknown v instant 1 1 350000 1 F32 : -1.128
2 : ECMWF unknown v instant 1 1 350000 1 F32 : -2.128
3 : ECMWF unknown v instant 1 1 350000 1 F32 : -3.128
4 : ECMWF unknown v instant 1 1 350000 1 F32 : -4.128
5 : ECMWF unknown v instant 1 1 350000 1 F32 : -5.128
6 : ECMWF unknown v instant 1 1 350000 1 F32 : -6.128
7 : ECMWF unknown v instant 1 1 350000 1 F32 : -7.128
8 : ECMWF unknown v instant 1 1 350000 1 F32 : -8.128
9 : ECMWF unknown v instant 1 1 350000 1 F32 : -9.128
10 : ECMWF unknown v instant 1 1 350000 1 F32 : -10.128
11 : ECMWF unknown v instant 1 1 350000 1 F32 : -11.128
12 : ECMWF unknown v instant 1 1 350000 1 F32 : -12.128
13 : ECMWF unknown v instant 1 1 350000 1 F32 : -13.128
14 : ECMWF unknown v instant 1 1 350000 1 F32 : -14.128
15 : ECMWF unknown v instant 1 1 350000 1 F32 : -15.128
16 : ECMWF unknown v instant 1 1 350000 1 F32 : -16.128
17 : ECMWF unknown v instant 1 1 350000 1 F32 : -17.128
18 : ECMWF unknown v instant 1 1 350000 1 F32 : -18.128
19 : ECMWF unknown v instant 1 1 350000 1 F32 : -19.128
20 : ECMWF unknown v instant 1 1 350000 1 F32 : -20.128
21 : ECMWF unknown v instant 1 1 350000 1 F32 : -21.128
22 : ECMWF unknown v instant 1 1 350000 1 F32 : -22.128
23 : ECMWF unknown v instant 1 1 350000 1 F32 : -23.128
24 : ECMWF unknown v instant 1 1 350000 1 F32 : -24.128
25 : ECMWF unknown v instant 1 1 350000 2 F32 : -25
Grid coordinates :
1 : lonlat : points=350000 (700x500)
lon : -24.95 to 44.95 by 0.1 degrees_east
lat : 69.95 to 20.05 by -0.1 degrees_north
2 : lonlat : points=350000 (700x500)
longitude : -24.95 to 44.95 by 0.1 degrees_east
latitude : 69.95 to 20.05 by -0.1 degrees_north
Vertical coordinates :
1 : surface : levels=1
Time coordinate :
time : 24 steps
RefTime = 2002-01-01 00:00:00 Units = hours Calendar = proleptic_gregorian
YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss
2002-01-01 00:00:00 2002-01-01 01:00:00 2002-01-01 02:00:00 2002-01-01 03:00:00
2002-01-01 04:00:00 2002-01-01 05:00:00 2002-01-01 06:00:00 2002-01-01 07:00:00
2002-01-01 08:00:00 2002-01-01 09:00:00 2002-01-01 10:00:00 2002-01-01 11:00:00
2002-01-01 12:00:00 2002-01-01 13:00:00 2002-01-01 14:00:00 2002-01-01 15:00:00
2002-01-01 16:00:00 2002-01-01 17:00:00 2002-01-01 18:00:00 2002-01-01 19:00:00
2002-01-01 20:00:00 2002-01-01 21:00:00 2002-01-01 22:00:00 2002-01-01 23:00:00
cdo sinfo: Processed 25 variables over 24 timesteps [0.06s 51MB]
This indicates that CDO didn't put them on the same grid like I wanted. I suspect that the error is due to differing precisions of the lat/lon coordinates in the two files: temp.nc is float32 and precip.nc is float64 (I saw this using Python). And/or they are extremely similar but not totally identical. I want a way using CDO commands to force the two files to have the same grid, so that when they are merged all variables from both files will be on the same grid. The files are too big to be uploaded unfortunately.
Replies (7)
RE: Force .nc files to be on the same grid when merging with "cdo merge" - Added by Estanislao Gavilan 10 days ago
Hi Joshua,
you can try to use the command "selgrid" after merge. It should remove the other grid.
Best regards,
Estanislao
RE: Force .nc files to be on the same grid when merging with "cdo merge" - Added by Joshua Miller 10 days ago
Thanks for the quick reply. There aren't any examples of selgrid being used in the documentation so could you or someone else write the full command I'd need?
RE: Force .nc files to be on the same grid when merging with "cdo merge" - Added by Estanislao Gavilan 10 days ago
Hi Joshua,
it is very simple.
cdo -selgrid=1 merge ..
or
cdo -selgrid=2 merge ..
RE: Force .nc files to be on the same grid when merging with "cdo merge" - Added by Joshua Miller 10 days ago
Hello,
Unfortunately I still don't understand what you mean. Could you please write out all the commands I need as verbosely as possible? Both the command to merge and to 'selgrid'
Cheers
RE: Force .nc files to be on the same grid when merging with "cdo merge" - Added by Estanislao Gavilan 10 days ago
Hi Joshua,
your final file has two grids
Grid coordinates :
1 : lonlat : points=350000 (700x500)
lon : -24.95 to 44.95 by 0.1 degrees_east
lat : 69.95 to 20.05 by -0.1 degrees_north
2 : lonlat : points=350000 (700x500)
longitude : -24.95 to 44.95 by 0.1 degrees_east
latitude : 69.95 to 20.05 by -0.1 degrees_north
You can select one grid using the command selgrid and the number of the grid. Since both grids are the same, you can choose either of them.
cdo -selgrid=1 merge weather.nc precip.nc output.nc
or
cdo -selgrid=2 merge weather.nc precip.nc output.nc
RE: Force .nc files to be on the same grid when merging with "cdo merge" - Added by Karin Meier-Fleischer 10 days ago
The problem is that in the one file the coordinate variables are lon and lat and in the other file they are longitude and latitude, so CDO assumes that there are two different grids. Changing the coordinte variable names of one file like in the other file should solve the problem.
Btw, the correct notation for selgrid is
cdo -selgrid,grids infile outfile
where grids is a 'Comma-separated list of grid names or numbers'.
RE: Force .nc files to be on the same grid when merging with "cdo merge" - Added by Joshua Miller 10 days ago
Thank you Karin! This was the problem. Renaming with ncrename -v longitude,lon -v latitude,lat precip.nc
and then doing the merge worked. No need to do selgrid after either