CDO does not merge files to same grid even if they are the same
Added by Joshua Miller 3 days ago
Hello, I am trying to merge two datasets from different sources together. They originally used different grids but I have modified them so that they have the identical 0.1 x 0.1 degree lat/lon grid. However, after I merge them, when I inspect the file using cdo sinfo, it shows there are now two grids.
cdo sinfo ERA5_data.nc
File format : NetCDF4
-1 : Institut Source T Steptype Levels Num Points Num Dtype : Parameter ID
1 : ECMWF unknown v instant 1 1 350000 1 F32 : -1.128
2 : ECMWF unknown v instant 1 1 350000 1 F32 : -2.128
3 : ECMWF unknown v instant 1 1 350000 1 F32 : -3.128
4 : ECMWF unknown v instant 1 1 350000 1 F32 : -4.128
5 : ECMWF unknown v instant 1 1 350000 1 F32 : -5.128
6 : ECMWF unknown v instant 1 1 350000 1 F32 : -6.128
7 : ECMWF unknown v instant 1 1 350000 1 F32 : -7.128
8 : ECMWF unknown v instant 1 1 350000 1 F32 : -8.128
9 : ECMWF unknown v instant 1 1 350000 1 F32 : -9.128
10 : ECMWF unknown v instant 1 1 350000 1 F32 : -10.128
11 : ECMWF unknown v instant 1 1 350000 1 F32 : -11.128
12 : ECMWF unknown v instant 1 1 350000 1 F32 : -12.128
13 : ECMWF unknown v instant 1 1 350000 1 F32 : -13.128
14 : ECMWF unknown v instant 1 1 350000 1 F32 : -14.128
15 : ECMWF unknown v instant 1 1 350000 1 F32 : -15.128
16 : ECMWF unknown v instant 1 1 350000 1 F32 : -16.128
17 : ECMWF unknown v instant 1 1 350000 1 F32 : -17.128
18 : ECMWF unknown v instant 1 1 350000 1 F32 : -18.128
19 : ECMWF unknown v instant 1 1 350000 1 F32 : -19.128
20 : ECMWF unknown v instant 1 1 350000 1 F32 : -20.128
21 : ECMWF unknown v instant 1 1 350000 1 F32 : -21.128
22 : ECMWF unknown v instant 1 1 350000 1 F32 : -22.128
23 : ECMWF unknown v instant 1 1 350000 1 F32 : -23.128
24 : ECMWF unknown v instant 1 1 350000 1 F32 : -24.128
Grid coordinates :
1 : lonlat : points=350000 (700x500)
lon : -24.95 to 44.95 by 0.1 degrees_east
lat : 69.95 to 20.05 by -0.1 degrees_north
Vertical coordinates :
1 : surface : levels=1
Time coordinate :
time : 24 steps
RefTime = 2002-01-01 00:00:00 Units = hours Calendar = proleptic_gregorian
YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss
2002-01-01 00:00:00 2002-01-01 01:00:00 2002-01-01 02:00:00 2002-01-01 03:00:00
2002-01-01 04:00:00 2002-01-01 05:00:00 2002-01-01 06:00:00 2002-01-01 07:00:00
2002-01-01 08:00:00 2002-01-01 09:00:00 2002-01-01 10:00:00 2002-01-01 11:00:00
2002-01-01 12:00:00 2002-01-01 13:00:00 2002-01-01 14:00:00 2002-01-01 15:00:00
2002-01-01 16:00:00 2002-01-01 17:00:00 2002-01-01 18:00:00 2002-01-01 19:00:00
2002-01-01 20:00:00 2002-01-01 21:00:00 2002-01-01 22:00:00 2002-01-01 23:00:00
cdo sinfo: Processed 24 variables over 24 timesteps [0.79s 50MB]
-------------------------------------------------------------------------------------------cdo sinfo precip.nc
File format : NetCDF4
-1 : Institut Source T Steptype Levels Num Points Num Dtype : Parameter ID
1 : unknown Huffman v instant 1 1 350000 1 F32 : -1
Grid coordinates :
1 : lonlat : points=350000 (700x500)
lon : -24.95 to 44.95 by 0.1 degrees_east
lat : 69.95 to 20.05 by -0.1 degrees_north
Vertical coordinates :
1 : surface : levels=1
Time coordinate :
time : 24 steps
RefTime = 1980-01-06 00:00:00 Units = seconds Calendar = gregorian
YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss
2002-01-01 00:00:00 2002-01-01 01:00:00 2002-01-01 02:00:00 2002-01-01 03:00:00
2002-01-01 04:00:00 2002-01-01 05:00:00 2002-01-01 06:00:00 2002-01-01 07:00:00
2002-01-01 08:00:00 2002-01-01 09:00:00 2002-01-01 10:00:00 2002-01-01 11:00:00
2002-01-01 12:00:00 2002-01-01 13:00:00 2002-01-01 14:00:00 2002-01-01 15:00:00
2002-01-01 16:00:00 2002-01-01 17:00:00 2002-01-01 18:00:00 2002-01-01 19:00:00
2002-01-01 20:00:00 2002-01-01 21:00:00 2002-01-01 22:00:00 2002-01-01 23:00:00
cdo sinfo: Processed 1 variable over 24 timesteps [1.03s 49MB]
----------------------------------------------------------------------------------------
cdo merge ERA5_data.nc precip.nc merged.nc
cdo sinfo merged.nc
File format : NetCDF4
-1 : Institut Source T Steptype Levels Num Points Num Dtype : Parameter ID
1 : ECMWF unknown v instant 1 1 350000 1 F32 : -1.128
2 : ECMWF unknown v instant 1 1 350000 1 F32 : -2.128
3 : ECMWF unknown v instant 1 1 350000 1 F32 : -3.128
4 : ECMWF unknown v instant 1 1 350000 1 F32 : -4.128
5 : ECMWF unknown v instant 1 1 350000 1 F32 : -5.128
6 : ECMWF unknown v instant 1 1 350000 1 F32 : -6.128
7 : ECMWF unknown v instant 1 1 350000 1 F32 : -7.128
8 : ECMWF unknown v instant 1 1 350000 1 F32 : -8.128
9 : ECMWF unknown v instant 1 1 350000 1 F32 : -9.128
10 : ECMWF unknown v instant 1 1 350000 1 F32 : -10.128
11 : ECMWF unknown v instant 1 1 350000 1 F32 : -11.128
12 : ECMWF unknown v instant 1 1 350000 1 F32 : -12.128
13 : ECMWF unknown v instant 1 1 350000 1 F32 : -13.128
14 : ECMWF unknown v instant 1 1 350000 1 F32 : -14.128
15 : ECMWF unknown v instant 1 1 350000 1 F32 : -15.128
16 : ECMWF unknown v instant 1 1 350000 1 F32 : -16.128
17 : ECMWF unknown v instant 1 1 350000 1 F32 : -17.128
18 : ECMWF unknown v instant 1 1 350000 1 F32 : -18.128
19 : ECMWF unknown v instant 1 1 350000 1 F32 : -19.128
20 : ECMWF unknown v instant 1 1 350000 1 F32 : -20.128
21 : ECMWF unknown v instant 1 1 350000 1 F32 : -21.128
22 : ECMWF unknown v instant 1 1 350000 1 F32 : -22.128
23 : ECMWF unknown v instant 1 1 350000 1 F32 : -23.128
24 : ECMWF unknown v instant 1 1 350000 1 F32 : -24.128
25 : ECMWF unknown v instant 1 1 350000 2 F32 : -25
Grid coordinates :
1 : lonlat : points=350000 (700x500)
lon : -24.95 to 44.95 by 0.1 degrees_east
lat : 69.95 to 20.05 by -0.1 degrees_north
2 : lonlat : points=350000 (700x500)
lon_2 : -24.95 to 44.95 by 0.1 degrees_east
lat_2 : 69.95 to 20.05 by -0.1 degrees_north
Vertical coordinates :
1 : surface : levels=1
Time coordinate :
time : 24 steps
RefTime = 2002-01-01 00:00:00 Units = hours Calendar = proleptic_gregorian
YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss
2002-01-01 00:00:00 2002-01-01 01:00:00 2002-01-01 02:00:00 2002-01-01 03:00:00
2002-01-01 04:00:00 2002-01-01 05:00:00 2002-01-01 06:00:00 2002-01-01 07:00:00
2002-01-01 08:00:00 2002-01-01 09:00:00 2002-01-01 10:00:00 2002-01-01 11:00:00
2002-01-01 12:00:00 2002-01-01 13:00:00 2002-01-01 14:00:00 2002-01-01 15:00:00
2002-01-01 16:00:00 2002-01-01 17:00:00 2002-01-01 18:00:00 2002-01-01 19:00:00
2002-01-01 20:00:00 2002-01-01 21:00:00 2002-01-01 22:00:00 2002-01-01 23:00:00
cdo sinfo: Processed 25 variables over 24 timesteps [0.45s 51MB]
Replies (5)
RE: CDO does not merge files to same grid even if they are the same - Added by Karin Meier-Fleischer 3 days ago
Hm, without the files it is hard to say what is going wrong. Which CDO version are you using? Can you upload the output of ncdump for both files or best the files?
RE: CDO does not merge files to same grid even if they are the same - Added by Joshua Miller 3 days ago
Hello, unfortunately the ncdump of the ERA5 file is 2.5G so I can't upload it. If you think it's necessary I can try to extract just one hour from each of the datasets and upload that.
CDO version 2.4.0, Copyright (C) 2002-2024 MPI für Meteorologie
This is free software and comes with ABSOLUTELY NO WARRANTY
Report bugs to <https://mpimet.mpg.de/cdo>
RE: CDO does not merge files to same grid even if they are the same - Added by Karin Meier-Fleischer 3 days ago
The output of
ncdump -h ERA5_data.nc ncdump -h precip.nc
shouldn't be that big.
But you can extract one timestep from each file and upload them.
RE: CDO does not merge files to same grid even if they are the same - Added by Joshua Miller 3 days ago
ncdump -h precip.nc
netcdf precip {
dimensions:
time = UNLIMITED ; // (24 currently)
lon = 700 ;
lat = 500 ;
variables:
double time(time) ;
time:standard_name = "time" ;
time:units = "seconds since 1980-01-06 00:00:00 UTC" ;
time:calendar = "gregorian" ;
time:axis = "T" ;
float lon(lon) ;
lon:standard_name = "longitude" ;
lon:long_name = "longitude" ;
lon:units = "degrees_east" ;
lon:axis = "X" ;
float lat(lat) ;
lat:standard_name = "latitude" ;
lat:long_name = "latitude" ;
lat:units = "degrees_north" ;
lat:axis = "Y" ;
float precipitation(time, lat, lon) ;
precipitation:units = "mm/hr" ;
// global attributes:
:CDI = "Climate Data Interface version 2.4.0 (https://mpimet.mpg.de/cdi)" ;
:Conventions = "CF-1.6" ;
:source = "Huffman, G. J., Stocker, E. F., Bolvin, D. T., Nelkin, E. J., Tan, Jackson. (2024). GPM IMERG Final Precipitation L3 Half Hourly 0.1 degree x 0.1 degree V07. Research Data Archive at the National Center for Atmospheric Research, Computational and Information Systems Laboratory. https://doi.org/10.5065/SRTD-0R12." ;
:title = "IMERG v7 Final hourly European precipitation" ;
:history = "Thu Jan 16 11:12:55 2025: cdo seldate,2002-01-01 /gws/nopw/j04/bris_climdyn/j_miller/IMERG/FilteredHourlyIMERG_2002-01-01_00:00:00-2002-12-31_23:00:00_latlon.nc /gws/nopw/j04/bris_climdyn/j_miller/IMERG/2002/FilteredHourlyIMERG_2002-01-01.nc\nCreated on 2025-01-15" ;
:description = "IMERG v7 Final half-hourly data filtered by Joshua Miller\nHalf-hourly variables are averaged to hourly time resolution" ;
:CDO = "Climate Data Operators version 2.4.0 (https://mpimet.mpg.de/cdo)" ;
-----------------------------------------------------------------------------------------------------------------------
ncdump -h ERA5_data.nc
netcdf ERA5_data {
dimensions:
time = UNLIMITED ; // (24 currently)
lon = 700 ;
lat = 500 ;
variables:
double time(time) ;
time:standard_name = "time" ;
time:units = "hours since 2002-01-01" ;
time:calendar = "proleptic_gregorian" ;
time:axis = "T" ;
double lon(lon) ;
lon:standard_name = "longitude" ;
lon:long_name = "longitude" ;
lon:units = "degrees_east" ;
lon:axis = "X" ;
double lat(lat) ;
lat:standard_name = "latitude" ;
lat:long_name = "latitude" ;
lat:units = "degrees_north" ;
lat:axis = "Y" ;
float temp850(time, lat, lon) ;
temp850:units = "K" ;
temp850:table = 128 ;
temp850:_FillValue = NaNf ;
temp850:missing_value = NaNf ;
float temp700(time, lat, lon) ;
temp700:units = "K" ;
temp700:table = 128 ;
temp700:_FillValue = NaNf ;
temp700:missing_value = NaNf ;
float temp500(time, lat, lon) ;
temp500:units = "K" ;
temp500:table = 128 ;
temp500:_FillValue = NaNf ;
temp500:missing_value = NaNf ;
float temp250(time, lat, lon) ;
temp250:units = "K" ;
temp250:table = 128 ;
temp250:_FillValue = NaNf ;
temp250:missing_value = NaNf ;
float vort850(time, lat, lon) ;
vort850:units = "s-1" ;
vort850:table = 128 ;
vort850:_FillValue = NaNf ;
vort850:missing_value = NaNf ;
float vort700(time, lat, lon) ;
vort700:units = "s-1" ;
vort700:table = 128 ;
vort700:_FillValue = NaNf ;
vort700:missing_value = NaNf ;
float vort500(time, lat, lon) ;
vort500:units = "s-1" ;
vort500:table = 128 ;
vort500:_FillValue = NaNf ;
vort500:missing_value = NaNf ;
float vort250(time, lat, lon) ;
vort250:units = "s-1" ;
vort250:table = 128 ;
vort250:_FillValue = NaNf ;
vort250:missing_value = NaNf ;
float specHum850(time, lat, lon) ;
specHum850:units = "kg kg-1" ;
specHum850:table = 128 ;
specHum850:_FillValue = NaNf ;
specHum850:missing_value = NaNf ;
float specHum700(time, lat, lon) ;
specHum700:units = "kg kg-1" ;
specHum700:table = 128 ;
specHum700:_FillValue = NaNf ;
specHum700:missing_value = NaNf ;
float specHum500(time, lat, lon) ;
specHum500:units = "kg kg-1" ;
specHum500:table = 128 ;
specHum500:_FillValue = NaNf ;
specHum500:missing_value = NaNf ;
float specHum250(time, lat, lon) ;
specHum250:units = "kg kg-1" ;
specHum250:table = 128 ;
specHum250:_FillValue = NaNf ;
specHum250:missing_value = NaNf ;
float relHum850(time, lat, lon) ;
relHum850:units = "%" ;
relHum850:table = 128 ;
relHum850:_FillValue = NaNf ;
relHum850:missing_value = NaNf ;
float relHum700(time, lat, lon) ;
relHum700:units = "%" ;
relHum700:table = 128 ;
relHum700:_FillValue = NaNf ;
relHum700:missing_value = NaNf ;
float relHum500(time, lat, lon) ;
relHum500:units = "%" ;
relHum500:table = 128 ;
relHum500:_FillValue = NaNf ;
relHum500:missing_value = NaNf ;
float relHum250(time, lat, lon) ;
relHum250:units = "%" ;
relHum250:table = 128 ;
relHum250:_FillValue = NaNf ;
relHum250:missing_value = NaNf ;
float u-wind850(time, lat, lon) ;
u-wind850:units = "m s-1" ;
u-wind850:table = 128 ;
u-wind850:_FillValue = NaNf ;
u-wind850:missing_value = NaNf ;
float u-wind700(time, lat, lon) ;
u-wind700:units = "m s-1" ;
u-wind700:table = 128 ;
u-wind700:_FillValue = NaNf ;
u-wind700:missing_value = NaNf ;
float u-wind500(time, lat, lon) ;
u-wind500:units = "m s-1" ;
u-wind500:table = 128 ;
u-wind500:_FillValue = NaNf ;
u-wind500:missing_value = NaNf ;
float u-wind250(time, lat, lon) ;
u-wind250:units = "m s-1" ;
u-wind250:table = 128 ;
u-wind250:_FillValue = NaNf ;
u-wind250:missing_value = NaNf ;
float v-wind850(time, lat, lon) ;
v-wind850:units = "m s-1" ;
v-wind850:table = 128 ;
v-wind850:_FillValue = NaNf ;
v-wind850:missing_value = NaNf ;
float v-wind700(time, lat, lon) ;
v-wind700:units = "m s-1" ;
v-wind700:table = 128 ;
v-wind700:_FillValue = NaNf ;
v-wind700:missing_value = NaNf ;
float v-wind500(time, lat, lon) ;
v-wind500:units = "m s-1" ;
v-wind500:table = 128 ;
v-wind500:_FillValue = NaNf ;
v-wind500:missing_value = NaNf ;
float v-wind250(time, lat, lon) ;
v-wind250:units = "m s-1" ;
v-wind250:table = 128 ;
v-wind250:_FillValue = NaNf ;
v-wind250:missing_value = NaNf ;
// global attributes:
:CDI = "Climate Data Interface version 2.4.0 (https://mpimet.mpg.de/cdi)" ;
:Conventions = "CF-1.6" ;
:institution = "European Centre for Medium-Range Weather Forecasts" ;
:history = "Wed Jan 15 13:53:42 2025: cdo remapnn,IMERG_grid.txt ...: cdo remapnn,target_grid.txt ...: ncatted -a units,lon,o,c,degrees_east ...: cdo -f nc copy era5.grib era5.nc" ;
:NCO = "netCDF Operators version 5.2.7 (Homepage = http://nco.sf.net, Code = http://github.com/nco/nco, Citation = 10.1016/j.envsoft.2008.03.004)" ;
:CDO = "Climate Data Operators version 2.4.0 (https://mpimet.mpg.de/cdo)" ;
RE: CDO does not merge files to same grid even if they are the same - Added by Karin Meier-Fleischer 3 days ago
Can you upload one timestep per file?