cdo yearmax giving me strange results
Added by Michelle Irizarry over 1 year ago
Hi!
I am running cdo yearmax on a netCDF file that is missing its crs (in.nc which is 71GB in size). This is the command I am using:
cdo yearmax -selyear,1980,2020 in.nc outrx1hr.nc
The only warning I get when running cdo yearmax is:
Warning (cdfScanVarAttr): NetCDF: Variable not found - crs.
Below is the output from ncdump -h on the original file (in.nc) and on the resulting file (outrx1hr.nc).
The map just below shows the results of the computation plotted in R for the first timestep (the first year).
There seems to be some tiling issue with the result. Namely, there's a vertical bar on the left and a horizontal bar on the top that actually belong on the right and bottom, respectively. I know this because I did the yearmax computation using Python as well and it looks right, but not using CDO. Is this tiling issue due to the netCDF file not having the crs defined? Or is it due to the large size of in.nc (71 GB)? How can I add the crs? This is the first time I encounter this issue with CDO and not sure what could be causing it.
Thanks for any help on this.
Michelle
ncdump -h on in.nc:
dimensions: y = 224 ; x = 229 ; time = 368064 ; variables: float lat(y, x) ; lat:_FillValue = NaNf ; lat:description = "LATITUDE, SOUTH IS NEGATIVE" ; lat:long_name = "Latitude, south is negative" ; lat:standard_name = "latitude" ; lat:units = "degree_north" ; float lon(y, x) ; lon:_FillValue = NaNf ; lon:description = "LONGITUDE, WEST IS NEGATIVE" ; lon:long_name = "Longitude, west is negative" ; lon:standard_name = "longitude" ; lon:units = "degree_east" ; int64 time(time) ; time:standard_name = "time" ; time:units = "hours since 1979-10-01" ; time:calendar = "proleptic_gregorian" ; double x(x) ; x:_FillValue = NaN ; x:long_name = "x coordinate of projection" ; x:standard_name = "projection_x_coordinate" ; x:units = "m" ; double y(y) ; y:_FillValue = NaN ; y:long_name = "y coordinate of projection" ; y:standard_name = "projection_y_coordinate" ; y:units = "m" ; float PREC_ACC_NC(time, y, x) ; PREC_ACC_NC:_FillValue = NaNf ; PREC_ACC_NC:description = "ACCUMULATED GRID SCALE PRECIPITATION OVER prec_acc_dt PERIODS OF TIME" ; PREC_ACC_NC:grid_mapping = "crs" ; PREC_ACC_NC:integration_length = "accumulated over prior 60 minutes" ; PREC_ACC_NC:long_name = "Accumulated grid scale precipitation" ; PREC_ACC_NC:units = "mm" ; PREC_ACC_NC:coordinates = "lon lat" ;
ncdump -h on outrx1hr.nc:
dimensions: time = UNLIMITED ; // (2 currently) bnds = 2 ; x = 229 ; y = 224 ; variables: double time(time) ; time:standard_name = "time" ; time:bounds = "time_bnds" ; time:units = "hours since 1979-10-01" ; time:calendar = "proleptic_gregorian" ; time:axis = "T" ; double time_bnds(time, bnds) ; float lon(y, x) ; lon:standard_name = "longitude" ; lon:long_name = "Longitude, west is negative" ; lon:units = "degree_east" ; lon:_CoordinateAxisType = "Lon" ; float lat(y, x) ; lat:standard_name = "latitude" ; lat:long_name = "Latitude, south is negative" ; lat:units = "degree_north" ; lat:_CoordinateAxisType = "Lat" ; double x(x) ; x:standard_name = "projection_x_coordinate" ; x:long_name = "x coordinate of projection" ; x:units = "m" ; x:axis = "X" ; double y(y) ; y:standard_name = "projection_y_coordinate" ; y:long_name = "y coordinate of projection" ; y:units = "m" ; y:axis = "Y" ; float PREC_ACC_NC(time, y, x) ; PREC_ACC_NC:long_name = "Accumulated grid scale precipitation" ; PREC_ACC_NC:units = "mm" ; PREC_ACC_NC:coordinates = "lat lon" ; PREC_ACC_NC:_FillValue = NaNf ; PREC_ACC_NC:missing_value = NaNf ; PREC_ACC_NC:cell_methods = "time: maximum" ; PREC_ACC_NC:description = "ACCUMULATED GRID SCALE PRECIPITATION OVER prec_acc_dt PERIODS OF TIME" ; PREC_ACC_NC:integration_length = "accumulated over prior 60 minutes" ; // global attributes: :CDI = "Climate Data Interface version 1.9.9rc1 (https://mpimet.mpg.de/cdi)" ; :Conventions = "CF-1.6" ; :history = "Tue Oct 17 16:08:19 2023: cdo yearmax -selyear,1980,2020 conus404_hourly_prec_FL.nc outrx1hr.nc" ; :frequency = "year" ; :CDO = "Climate Data Operators version 1.9.9rc1 (https://mpimet.mpg.de/cdo)" ;
Replies (7)
RE: cdo yearmax giving me strange results - Added by Michelle Irizarry over 1 year ago
Warning (cdfScanVarAttr): NetCDF: Variable not found - crs #
- gridID 1
#
gridtype = curvilinear
gridsize = 51296
datatype = float
xsize = 229
ysize = 224
xname = lon
xdimname = x
xlongname = "Longitude, west is negative"
xunits = "degree_east"
yname = lat
ydimname = y
ylongname = "Latitude, south is negative"
yunits = "degree_north"
xvals = -88.99982 -88.96124 -88.92264 -88.88403 -88.84546 -88.80685 -88.76825
RE: cdo yearmax giving me strange results - Added by Michelle Irizarry over 1 year ago
NEVER MIND! I think I found the issue! Thanks!
RE: cdo yearmax giving me strange results - Added by Ralf Mueller over 1 year ago
Hi Michelle!
What was the problem? the ncdump output looks fine to me. But the coordinates might have wrong values - hard to tell.
cheers
ralf
RE: cdo yearmax giving me strange results - Added by Michelle Irizarry over 1 year ago
Hi Ralf, the problem is in the original file. It was just so large it was hard to see it by looking at a few isolated timesteps. I wrote it in Python to netcdf so there must be an issue with that function. Sorry about the posting when it wasn't really a CDO issue. Great news because I use CDO quite often and have never had any issues like this!
RE: cdo yearmax giving me strange results - Added by Mohammed Ali over 1 year ago
Hi Michelle Irizarry, can you providing me your python code please, because i countered same problem here, thank you (san33oon@gmail.com)
RE: cdo yearmax giving me strange results - Added by Mohammed Ali over 1 year ago
here is the file and such a kind of plot that i want
GLB_JJA_PRECT.nc (2.85 MB) GLB_JJA_PRECT.nc | |||
1.png (64.9 KB) 1.png |
RE: cdo yearmax giving me strange results - Added by Michelle Irizarry over 1 year ago
Sorry, in the end, the problem was not Python writing to netCDF but my file transfer between computers had some bytes shifted somehow.