Keeping all variables
Added by Brendan DeTracey almost 4 years ago
Hey all,
CDO keeps removing all non-dimensional variables with the following:
$ cdo selgridcell,44009 -reducegrid,/home/brendan/data1/cmip6_data/680/cache/mask_Omon_MIROC6_t-grid.nc -seldate,'2014-07-16T12:00:00' -selname,sigma,eta,depth,zlev,sigma_bnds,zlev_bnds,thetao /home/brendan/data1/cmip6_data/680/MIROC6/raw/past/thetao_Omon_MIROC6_historical_r1i1p1f1_gn_201001-201412.nc thetao_Omon_MIROC6_historical_r1i1p1f1_201407_shallow.nc cdo(1) reducegrid: Process started cdo(2) seldate: Process started cdo(3) selname: Process started cdo(3) selname: Processed 353986938 values from 7 variables over 60 timesteps. cdo(2) seldate: Processed 5990778 values from 7 variables over 56 timesteps. cdo(1) reducegrid: Processed 5806080 values from 7 variables over 1 timestep. cdo selgridcell: Processed 3511998 values from 1 variable over 1 timestep [6.65s 110MB].Looking at the output it appears selgrid cell is the culprit. Any suggestions?
netcdf thetao_Omon_MIROC6_historical_r1i1p1f1_gn_201001-201412 {
dimensions:
time = UNLIMITED ; // (60 currently)
lev = 63 ;
y = 256 ;
x = 360 ;
bnds = 2 ;
vertices = 4 ;
variables:
double time(time) ;
time:bounds = "time_bnds" ;
time:units = "days since 1850-1-1" ;
time:calendar = "gregorian" ;
time:axis = "T" ;
time:long_name = "time" ;
time:standard_name = "time" ;
double time_bnds(time, bnds) ;
double lev(lev) ;
lev:bounds = "lev_bnds" ;
lev:axis = "Z" ;
lev:long_name = "ocean sigma over z coordinate" ;
lev:standard_name = "ocean_sigma_z" ;
lev:formula = "for k <= nsigma: z(n,k,j,i) = eta(n,j,i) + sigma(k)*(min(depth_c,depth(j,i))+eta(n,j,i)) ; for k > nsigma: z(n,k,j,i) = zlev(k)" ;
lev:formula_terms = "sigma: sigma eta: eta depth: depth depth_c: depth_c nsigma: nsigma zlev: zlev" ;
double lev_bnds(lev, bnds) ;
lev_bnds:formula = "for k <= nsigma: z(n,k,j,i) = eta(n,j,i) + sigma(k)*(min(depth_c,depth(j,i))+eta(n,j,i)) ; for k > nsigma: z(n,k,j,i) = zlev(k)" ;
lev_bnds:standard_name = "ocean_sigma_z" ;
lev_bnds:units = "" ;
lev_bnds:formula_terms = "sigma: sigma_bnds eta: eta depth: depth depth_c: depth_c nsigma: nsigma zlev: zlev_bnds" ;
double sigma(lev) ;
sigma:long_name = "vertical coordinate formula term: sigma(k)" ;
float eta(time, y, x) ;
eta:long_name = "Sea Surface Height formula term: thetao" ;
eta:units = "m" ;
eta:original_units = "cm" ;
eta:history = "2019-01-30T08:21:46Z altered by CMOR: Converted units from \'cm\' to \'m\'." ;
float depth(y, x) ;
depth:long_name = "Sea Floor Depth: formula term: thetao" ;
depth:units = "m" ;
depth:units = "m" ;
double depth_c ;
depth_c:long_name = "vertical coordinate formula term: depth_c" ;
int nsigma ;
nsigma:long_name = "vertical coordinate formula term: nsigma" ;
double zlev(lev) ;
zlev:long_name = "vertical coordinate formula term: zlev(k)" ;
double sigma_bnds(lev, bnds) ;
sigma_bnds:long_name = "vertical coordinate formula term: sigma(k+1/2)" ;
double zlev_bnds(lev, bnds) ;
zlev_bnds:long_name = "vertical coordinate formula term: zlev(k+1/2)" ;
double y(y) ;
y:bounds = "y_bnds" ;
y:units = "degrees" ;
y:axis = "Y" ;
y:long_name = "y coordinate of projection" ;
y:standard_name = "projection_y_coordinate" ;
double y_bnds(y, bnds) ;
double x(x) ;
x:bounds = "x_bnds" ;
x:units = "degrees" ;
x:axis = "X" ;
x:long_name = "x coordinate of projection" ;
x:standard_name = "projection_x_coordinate" ;
double x_bnds(x, bnds) ;
float latitude(y, x) ;
latitude:standard_name = "latitude" ;
latitude:long_name = "latitude" ;
latitude:units = "degrees_north" ;
latitude:missing_value = 1.e+20f ;
latitude:_FillValue = 1.e+20f ;
latitude:bounds = "vertices_latitude" ;
float longitude(y, x) ;
longitude:standard_name = "longitude" ;
longitude:long_name = "longitude" ;
longitude:units = "degrees_east" ;
longitude:missing_value = 1.e+20f ;
longitude:_FillValue = 1.e+20f ;
longitude:bounds = "vertices_longitude" ;
float vertices_latitude(y, x, vertices) ;
vertices_latitude:units = "degrees_north" ;
vertices_latitude:missing_value = 1.e+20f ;
vertices_latitude:_FillValue = 1.e+20f ;
float vertices_longitude(y, x, vertices) ;
vertices_longitude:units = "degrees_east" ;
vertices_longitude:missing_value = 1.e+20f ;
vertices_longitude:_FillValue = 1.e+20f ;
float thetao(time, lev, y, x) ;
thetao:standard_name = "sea_water_potential_temperature" ;
thetao:long_name = "Sea Water Potential Temperature" ;
thetao:comment = "Diagnostic should be contributed even for models using conservative temperature as prognostic field." ;
thetao:units = "degC" ;
thetao:original_name = "TO" ;
thetao:cell_methods = "area: mean where sea time: mean" ;
thetao:cell_measures = "area: areacello volume: volcello" ;
thetao:history = "2019-01-30T08:21:46Z altered by CMOR: replaced missing value flag (-999) with standard missing value (1e+20)." ;
thetao:missing_value = 1.e+20f ;
thetao:_FillValue = 1.e+20f ;
thetao:coordinates = "latitude longitude" ;
// global attributes:
...}
netcdf thetao_Omon_MIROC6_historical_r1i1p1f1_201407_shallow {
dimensions:
time = UNLIMITED ; // (1 currently)
bnds = 2 ;
x = 1 ;
vertices = 4 ;
lev = 63 ;
variables:
double time(time) ;
time:standard_name = "time" ;
time:long_name = "time" ;
time:bounds = "time_bnds" ;
time:units = "days since 1850-1-1" ;
time:calendar = "gregorian" ;
time:axis = "T" ;
double time_bnds(time, bnds) ;
double longitude(x) ;
longitude:standard_name = "longitude" ;
longitude:long_name = "longitude" ;
longitude:units = "degrees_east" ;
longitude:bounds = "longitude_bnds" ;
double longitude_bnds(x, vertices) ;
double latitude(x) ;
latitude:standard_name = "latitude" ;
latitude:long_name = "latitude" ;
latitude:units = "degrees_north" ;
latitude:bounds = "latitude_bnds" ;
double latitude_bnds(x, vertices) ;
double lev(lev) ;
lev:standard_name = "ocean_sigma_z" ;
lev:long_name = "ocean sigma over z coordinate" ;
lev:axis = "Z" ;
lev:bounds = "lev_bnds" ;
lev:formula = "for k <= nsigma: z(n,k,j,i) = eta(n,j,i) + sigma(k)*(min(depth_c,depth(j,i))+eta(n,j,i)) ; for k > nsigma: z(n,k,j,i) = zlev(k)" ;
double lev_bnds(lev, bnds) ;
float thetao(time, lev, x) ;
thetao:standard_name = "sea_water_potential_temperature" ;
thetao:long_name = "Sea Water Potential Temperature" ;
thetao:units = "degC" ;
thetao:CDI_grid_type = "unstructured" ;
thetao:coordinates = "latitude longitude" ;
thetao:_FillValue = 1.e+20f ;
thetao:missing_value = 1.e+20f ;
thetao:comment = "Diagnostic should be contributed even for models using conservative temperature as prognostic field." ;
thetao:original_name = "TO" ;
thetao:cell_methods = "area: mean where sea time: mean" ;
thetao:cell_measures = "area: areacello volume: volcello" ;
thetao:history = "2019-01-30T08:21:46Z altered by CMOR: replaced missing value flag (-999) with standard missing value (1e+20)." ;
// global attributes:
...}
Any suggestions?
Replies (2)
RE: Keeping all variables - Added by Uwe Schulzweida almost 4 years ago
Hi Brendan,
This is a general problem with the CDO internal data model. CDO processes only the data variables and the associated coordinate variables of a netCDF file. All coordinate variables and dimensions that are not assigned to a data variable will be lost after processing with CDO!
Variables like depth_c or nsigma are not needed in the file from CDO's point of view. This is mainly because the attribute lev:formula_terms is not evaluated by CDO.
Best,
Uwe
RE: Keeping all variables - Added by Brendan DeTracey almost 4 years ago
Thanks Uwe. Good to know. I used NCO for the calculation. It is always good to have a few different tools from which to choose.