Project

General

Profile

How can merge several hundred daily .nc files while formatting its time coordinates?

Added by Frida Perez almost 3 years ago

Hi All,

I am working with ~3000 netcdf files that contain daily sea ice concentration. I want to merge all the files into 1 .nc file. Each daily file has 33 variables of which I only want to keep 3. Because I cannot pipe the 'merge time' operator I first ran this command:

cdo select,name=GridLat_SpPolarGrid12km,GridLon_SpPolarGrid12km,SI_12km_SH_ICECON_DAY_SpPolarGrid12km AMSR_U2_L3_SeaIce12km_B04_20*.nc sic_12_21.nc

This gave me a file with all files concatenated into 1 with all three variables.

The SI_12km_SH_ICECON_DAY_SpPolarGrid12km variable has time in the first dimension but it shows up as 'frames' when I look at it on ncview ( as opposed to dates)

This is its ncdump
netcdf sic_12_21 {
dimensions:
time = UNLIMITED ; // (3289 currently)
XDim_SpPolarGrid12km = 632 ;
YDim_SpPolarGrid12km = 664 ;
variables:
double time(time) ;
time:standard_name = "time" ;
time:units = "day as %Y%m%d.%f" ;
time:calendar = "proleptic_gregorian" ;
time:axis = "T" ;
double GridLon_SpPolarGrid12km(YDim_SpPolarGrid12km, XDim_SpPolarGrid12km) ;
GridLon_SpPolarGrid12km:standard_name = "longitude" ;
GridLon_SpPolarGrid12km:long_name = "longitude" ;
GridLon_SpPolarGrid12km:units = "degrees_east" ;
GridLon_SpPolarGrid12km:_CoordinateAxisType = "Lon" ;
double GridLat_SpPolarGrid12km(YDim_SpPolarGrid12km, XDim_SpPolarGrid12km) ;
GridLat_SpPolarGrid12km:standard_name = "latitude" ;
GridLat_SpPolarGrid12km:long_name = "latitude" ;
GridLat_SpPolarGrid12km:units = "degrees_north" ;
GridLat_SpPolarGrid12km:_CoordinateAxisType = "Lat" ;
int SI_12km_SH_ICECON_DAY_SpPolarGrid12km(time, YDim_SpPolarGrid12km, XDim_SpPolarGrid12km) ;
SI_12km_SH_ICECON_DAY_SpPolarGrid12km:long_name = "Sea ice concentration daily average" ;
SI_12km_SH_ICECON_DAY_SpPolarGrid12km:units = "percent" ;
SI_12km_SH_ICECON_DAY_SpPolarGrid12km:coordinates = "GridLat_SpPolarGrid12km GridLon_SpPolarGrid12km" ;
SI_12km_SH_ICECON_DAY_SpPolarGrid12km:projection = "Polar Stereographic" ;
SI_12km_SH_ICECON_DAY_SpPolarGrid12km:comment = "data value meaning: 0 -- Open Water, 110 -- missing/not calculated, 120 -- Land"

I did the following two commands afterwards:
#1 cdo settaxis,2012-07-12,day sic_12_21.nc SIC-time.nc
#2 cdo -a setcalendar,standard SIC-time.nc SICstandard.nc

The dates are now showing as YYYY-MMM-DDD hh:mm:ss, I would like them to be in datetime64 , currently they are floats ((time) float64 2.012e+07 ... 2.021...). The time units are "day as %Y%m%d.%f" , is there a way to get them to "days since 1601-01-01 00:00:00"?

Is there a more efficient way of doing this task on cdo? Any advice/tips are welcome!

Thanks,
Frida


Replies (3)

RE: How can merge several hundred daily .nc files while formatting its time coordinates? - Added by Frida Perez almost 3 years ago

I would like to add that these were all .he5 files first and converted to netcdf files via ncls "ncl_convert2nc *.he5"

RE: How can merge several hundred daily .nc files while formatting its time coordinates? - Added by Karin Meier-Fleischer almost 3 years ago

Hi Frida,

if you only need 3 variables you can select them during the ncl_convert2nc call

ncl_convert2nc *.he5 -v var1,var2,var3

and then do the mergetime with cdo.

-Karin

RE: How can merge several hundred daily .nc files while formatting its time coordinates? - Added by Frida Perez almost 3 years ago

Hi Karin,

This indeed worked! However I did have to use "cat" instead of "mergetime"

Thanks,

Frida

    (1-3/3)