How can merge several hundred daily .nc files while formatting its time coordinates?
Added by Frida Perez over 3 years ago
Hi All,
I am working with ~3000 netcdf files that contain daily sea ice concentration. I want to merge all the files into 1 .nc file. Each daily file has 33 variables of which I only want to keep 3. Because I cannot pipe the 'merge time' operator I first ran this command:
cdo select,name=GridLat_SpPolarGrid12km,GridLon_SpPolarGrid12km,SI_12km_SH_ICECON_DAY_SpPolarGrid12km AMSR_U2_L3_SeaIce12km_B04_20*.nc sic_12_21.nc
This gave me a file with all files concatenated into 1 with all three variables.
The SI_12km_SH_ICECON_DAY_SpPolarGrid12km variable has time in the first dimension but it shows up as 'frames' when I look at it on ncview ( as opposed to dates)
This is its ncdumpnetcdf sic_12_21 {
dimensions:
time = UNLIMITED ; // (3289 currently)
XDim_SpPolarGrid12km = 632 ;
YDim_SpPolarGrid12km = 664 ;
variables:
double time(time) ;
time:standard_name = "time" ;
time:units = "day as %Y%m%d.%f" ;
time:calendar = "proleptic_gregorian" ;
time:axis = "T" ;
double GridLon_SpPolarGrid12km(YDim_SpPolarGrid12km, XDim_SpPolarGrid12km) ;
GridLon_SpPolarGrid12km:standard_name = "longitude" ;
GridLon_SpPolarGrid12km:long_name = "longitude" ;
GridLon_SpPolarGrid12km:units = "degrees_east" ;
GridLon_SpPolarGrid12km:_CoordinateAxisType = "Lon" ;
double GridLat_SpPolarGrid12km(YDim_SpPolarGrid12km, XDim_SpPolarGrid12km) ;
GridLat_SpPolarGrid12km:standard_name = "latitude" ;
GridLat_SpPolarGrid12km:long_name = "latitude" ;
GridLat_SpPolarGrid12km:units = "degrees_north" ;
GridLat_SpPolarGrid12km:_CoordinateAxisType = "Lat" ;
int SI_12km_SH_ICECON_DAY_SpPolarGrid12km(time, YDim_SpPolarGrid12km, XDim_SpPolarGrid12km) ;
SI_12km_SH_ICECON_DAY_SpPolarGrid12km:long_name = "Sea ice concentration daily average" ;
SI_12km_SH_ICECON_DAY_SpPolarGrid12km:units = "percent" ;
SI_12km_SH_ICECON_DAY_SpPolarGrid12km:coordinates = "GridLat_SpPolarGrid12km GridLon_SpPolarGrid12km" ;
SI_12km_SH_ICECON_DAY_SpPolarGrid12km:projection = "Polar Stereographic" ;
SI_12km_SH_ICECON_DAY_SpPolarGrid12km:comment = "data value meaning: 0 -- Open Water, 110 -- missing/not calculated, 120 -- Land"
I did the following two commands afterwards: #1 cdo settaxis,2012-07-12,day sic_12_21.nc SIC-time.nc
#2 cdo -a setcalendar,standard SIC-time.nc SICstandard.nc
The dates are now showing as YYYY-MMM-DDD hh:mm:ss, I would like them to be in datetime64 , currently they are floats ((time) float64 2.012e+07 ... 2.021...). The time units are "day as %Y%m%d.%f" , is there a way to get them to "days since 1601-01-01 00:00:00"?
Is there a more efficient way of doing this task on cdo? Any advice/tips are welcome!
Thanks,
Frida
Replies (3)
RE: How can merge several hundred daily .nc files while formatting its time coordinates? - Added by Frida Perez over 3 years ago
I would like to add that these were all .he5 files first and converted to netcdf files via ncls "ncl_convert2nc *.he5"
RE: How can merge several hundred daily .nc files while formatting its time coordinates? - Added by Karin Meier-Fleischer over 3 years ago
Hi Frida,
if you only need 3 variables you can select them during the ncl_convert2nc call
ncl_convert2nc *.he5 -v var1,var2,var3
and then do the mergetime with cdo.
-Karin
RE: How can merge several hundred daily .nc files while formatting its time coordinates? - Added by Frida Perez over 3 years ago
Hi Karin,
This indeed worked! However I did have to use "cat" instead of "mergetime"
Thanks,
Frida