Project

General

Profile

cdo yearmax giving me strange results

Added by Michelle Irizarry 6 months ago

Hi!

I am running cdo yearmax on a netCDF file that is missing its crs (in.nc which is 71GB in size). This is the command I am using:
cdo yearmax -selyear,1980,2020 in.nc outrx1hr.nc
The only warning I get when running cdo yearmax is:
Warning (cdfScanVarAttr): NetCDF: Variable not found - crs.

Below is the output from ncdump -h on the original file (in.nc) and on the resulting file (outrx1hr.nc).

The map just below shows the results of the computation plotted in R for the first timestep (the first year).
There seems to be some tiling issue with the result. Namely, there's a vertical bar on the left and a horizontal bar on the top that actually belong on the right and bottom, respectively. I know this because I did the yearmax computation using Python as well and it looks right, but not using CDO. Is this tiling issue due to the netCDF file not having the crs defined? Or is it due to the large size of in.nc (71 GB)? How can I add the crs? This is the first time I encounter this issue with CDO and not sure what could be causing it.

Thanks for any help on this.
Michelle

ncdump -h on in.nc:

dimensions:
        y = 224 ;
        x = 229 ;
        time = 368064 ;
variables:
        float lat(y, x) ;
                lat:_FillValue = NaNf ;
                lat:description = "LATITUDE, SOUTH IS NEGATIVE" ;
                lat:long_name = "Latitude, south is negative" ;
                lat:standard_name = "latitude" ;
                lat:units = "degree_north" ;
        float lon(y, x) ;
                lon:_FillValue = NaNf ;
                lon:description = "LONGITUDE, WEST IS NEGATIVE" ;
                lon:long_name = "Longitude, west is negative" ;
                lon:standard_name = "longitude" ;
                lon:units = "degree_east" ;
        int64 time(time) ;
                time:standard_name = "time" ;
                time:units = "hours since 1979-10-01" ;
                time:calendar = "proleptic_gregorian" ;
        double x(x) ;
                x:_FillValue = NaN ;
                x:long_name = "x coordinate of projection" ;
                x:standard_name = "projection_x_coordinate" ;
                x:units = "m" ;
        double y(y) ;
                y:_FillValue = NaN ;
                y:long_name = "y coordinate of projection" ;
                y:standard_name = "projection_y_coordinate" ;
                y:units = "m" ;
        float PREC_ACC_NC(time, y, x) ; 
               PREC_ACC_NC:_FillValue = NaNf ;
                PREC_ACC_NC:description = "ACCUMULATED GRID SCALE  PRECIPITATION OVER prec_acc_dt PERIODS OF TIME" ;
                PREC_ACC_NC:grid_mapping = "crs" ;
                PREC_ACC_NC:integration_length = "accumulated over prior 60 minutes" ;
                PREC_ACC_NC:long_name = "Accumulated grid scale precipitation" ;
                PREC_ACC_NC:units = "mm" ;
                PREC_ACC_NC:coordinates = "lon lat" ;

ncdump -h on outrx1hr.nc:
dimensions:
        time = UNLIMITED ; // (2 currently)
        bnds = 2 ;
        x = 229 ;
        y = 224 ;
variables:
        double time(time) ;
                time:standard_name = "time" ;
                time:bounds = "time_bnds" ;
                time:units = "hours since 1979-10-01" ;
                time:calendar = "proleptic_gregorian" ;
                time:axis = "T" ;
        double time_bnds(time, bnds) ;
        float lon(y, x) ;
                lon:standard_name = "longitude" ;
                lon:long_name = "Longitude, west is negative" ;
                lon:units = "degree_east" ;
                lon:_CoordinateAxisType = "Lon" ;
        float lat(y, x) ;
                lat:standard_name = "latitude" ;
                lat:long_name = "Latitude, south is negative" ;
                lat:units = "degree_north" ;
                lat:_CoordinateAxisType = "Lat" ;
        double x(x) ;
                x:standard_name = "projection_x_coordinate" ;
                x:long_name = "x coordinate of projection" ;
                x:units = "m" ;
                x:axis = "X" ;
        double y(y) ;
                y:standard_name = "projection_y_coordinate" ;
                y:long_name = "y coordinate of projection" ;
                y:units = "m" ;
                y:axis = "Y" ;
        float PREC_ACC_NC(time, y, x) ;
                PREC_ACC_NC:long_name = "Accumulated grid scale precipitation" ;
                PREC_ACC_NC:units = "mm" ;
                PREC_ACC_NC:coordinates = "lat lon" ;
                PREC_ACC_NC:_FillValue = NaNf ;
                PREC_ACC_NC:missing_value = NaNf ;
                PREC_ACC_NC:cell_methods = "time: maximum" ;
                PREC_ACC_NC:description = "ACCUMULATED GRID SCALE  PRECIPITATION OVER prec_acc_dt PERIODS OF TIME" ;
                PREC_ACC_NC:integration_length = "accumulated over prior 60 minutes" ;

// global attributes:
                :CDI = "Climate Data Interface version 1.9.9rc1 (https://mpimet.mpg.de/cdi)" ;
                :Conventions = "CF-1.6" ;
                :history = "Tue Oct 17 16:08:19 2023: cdo yearmax -selyear,1980,2020 conus404_hourly_prec_FL.nc outrx1hr.nc" ;
                :frequency = "year" ;
                :CDO = "Climate Data Operators version 1.9.9rc1 (https://mpimet.mpg.de/cdo)" ;


Replies (7)

RE: cdo yearmax giving me strange results - Added by Michelle Irizarry 6 months ago

By the way, griddes says that the type is curvilinear.
Warning (cdfScanVarAttr): NetCDF: Variable not found - crs #
  1. gridID 1 #
    gridtype = curvilinear
    gridsize = 51296
    datatype = float
    xsize = 229
    ysize = 224
    xname = lon
    xdimname = x
    xlongname = "Longitude, west is negative"
    xunits = "degree_east"
    yname = lat
    ydimname = y
    ylongname = "Latitude, south is negative"
    yunits = "degree_north"
    xvals = -88.99982 -88.96124 -88.92264 -88.88403 -88.84546 -88.80685 -88.76825

RE: cdo yearmax giving me strange results - Added by Michelle Irizarry 6 months ago

NEVER MIND! I think I found the issue! Thanks!

RE: cdo yearmax giving me strange results - Added by Ralf Mueller 6 months ago

Hi Michelle!

What was the problem? the ncdump output looks fine to me. But the coordinates might have wrong values - hard to tell.

cheers
ralf

RE: cdo yearmax giving me strange results - Added by Michelle Irizarry 6 months ago

Hi Ralf, the problem is in the original file. It was just so large it was hard to see it by looking at a few isolated timesteps. I wrote it in Python to netcdf so there must be an issue with that function. Sorry about the posting when it wasn't really a CDO issue. Great news because I use CDO quite often and have never had any issues like this!

RE: cdo yearmax giving me strange results - Added by Mohammed Ali 6 months ago

Hi Michelle Irizarry, can you providing me your python code please, because i countered same problem here, thank you ()

RE: cdo yearmax giving me strange results - Added by Mohammed Ali 6 months ago

here is the file and such a kind of plot that i want

RE: cdo yearmax giving me strange results - Added by Michelle Irizarry 6 months ago

Sorry, in the end, the problem was not Python writing to netCDF but my file transfer between computers had some bytes shifted somehow.

    (1-7/7)