Possible bug in 2.4.1 with NaN detection
Added by Paolo Davini 5 months ago
We have a quite peculiar netcdf file (here attached) that seems to be read with difficulty with CDO 2.4.1, while no issues where reported if using CDO 2.4.0
With CDO 2.4.0
paolo@danglard Desktop % cdo info eORCA1_sst_2D.nc
cdi warning (cdfScanVarAttr): NetCDF: Variable not found - lon_bounds
cdi warning (cdfScanVarAttr): NetCDF: Variable not found - lat_bounds
cdi warning (cdfInqContents): Coordinates variable level can't be assigned!
-1 : Date Time Level Gridsize Miss : Minimum Mean Maximum : Parameter ID
1 : 1950-01-01 00:00:00 0 120184 54657 : 269.85 286.92 303.93 : -1
2 : 1950-02-01 00:00:00 0 120184 54657 : 270.18 287.04 305.48 : -1
3 : 1950-03-01 00:00:00 0 120184 54657 : 270.23 286.81 305.41 : -1
4 : 1950-04-01 00:00:00 0 120184 54657 : 270.14 286.63 305.15 : -1
5 : 1950-05-01 00:00:00 0 120184 54657 : 270.13 286.41 305.97 : -1
6 : 1950-06-01 00:00:00 0 120184 54657 : 270.15 286.32 306.21 : -1
With CDO 2.4.1
(env) paolo@danglard Desktop % cdo info eORCA1_sst_2D.nc
cdi warning (cdfScanVarAttr): NetCDF: Variable not found - lon_bounds
cdi warning (cdfInqContents): Coordinates variable level can't be assigned!
-1 : Date Time Level Gridsize Miss : Minimum Mean Maximum : Parameter ID
1 : 1950-01-01 00:00:00 0 120184 54657 : 1.7977e+308 nan-1.7977e+308 : -1
cdo info (Warning): Found 65527 of 54657 missing values!
2 : 1950-02-01 00:00:00 0 120184 54657 : 1.7977e+308 nan-1.7977e+308 : -1
cdo info (Warning): Found 65527 of 54657 missing values!
3 : 1950-03-01 00:00:00 0 120184 54657 : 1.7977e+308 nan-1.7977e+308 : -1
cdo info (Warning): Found 65527 of 54657 missing values!
4 : 1950-04-01 00:00:00 0 120184 54657 : 1.7977e+308 nan-1.7977e+308 : -1
cdo info (Warning): Found 65527 of 54657 missing values!
5 : 1950-05-01 00:00:00 0 120184 54657 : 1.7977e+308 nan-1.7977e+308 : -1
cdo info (Warning): Found 65527 of 54657 missing values!
6 : 1950-06-01 00:00:00 0 120184 54657 : 1.7977e+308 nan-1.7977e+308 : -1
cdo info (Warning): Found 65527 of 54657 missing values!
cdo info: Processed 721104 values from 1 variable over 6 timesteps [0.05s 32MB]
We are aware that the file is not the best "format" possible, but I was wondering if this is a problem of the code or a problem of the file. We realized this since this is part of our CI/CD pipeline and the tests start to fail once we moved to 2.4.1
Any suggestions welcome!!!
Paolo
| eORCA1_sst_2D.nc (7.35 MB) eORCA1_sst_2D.nc |
Replies (11)
RE: Possible bug in 2.4.1 with NaN detection - Added by Uwe Schulzweida 5 months ago
Hi Paolo,
Thanks for this report! This is a bug in CDO release 2.4.1, the NANs are not handled correctly. A new CDO release will be available next week.
Cheers,
Uwe
RE: Possible bug in 2.4.1 with NaN detection - Added by Paolo Davini 5 months ago
Oh, good to hear that you already know it! Thanks for the fast reply!
Best,
Paolo
RE: Possible bug in 2.4.1 with NaN detection - Added by Paolo Davini 3 months ago
Hi Uwe,
I see that 2.4.2 and 2.4.3 has come out solving this bug, I was wondering if there is any timeline in porting the updates to conda (which is still 2.4.1, and our pipeline rely on that).
Thanks a lot
Paolo
RE: Possible bug in 2.4.1 with NaN detection - Added by Ralf Mueller 3 months ago
hi Paolo!
sry, I forgot to update the conda packages - will give it a go today.
cheers
ralf
RE: Possible bug in 2.4.1 with NaN detection - Added by Ralf Mueller 3 months ago
@Paolo is 2.4.3 ok for u? or do you need, 2.4.2, too?
RE: Possible bug in 2.4.1 with NaN detection - Added by Paolo Davini 3 months ago
Thanks!
Not tried yet 2.4.3 but I think we can skip 2.4.2 version, no specific need afaik!
RE: Possible bug in 2.4.1 with NaN detection - Added by Ralf Mueller 3 months ago
hi again!
unfortunately the testing infrastructure in conda-forge has problems to build CDO at the moment. Hence the update procedure for 2.4.3 is on hold, sry
Can you build cdo-2.4.3 locally maybe as a workaround?
RE: Possible bug in 2.4.1 with NaN detection - Added by Paolo Davini 3 months ago
Hi Ralf,
it is not that urgent, we are pinning CDO 2.4.0 in our environment so far.
However, when the build issues are solved we would be very happy to migrate to the most updated version
Thanks!
Paolo
RE: Possible bug in 2.4.1 with NaN detection - Added by Ralf Mueller 3 months ago
alright - I will let u know here when the update was successful
have a nice weekend!
RE: Possible bug in 2.4.1 with NaN detection - Added by Ralf Mueller about 2 months ago
hi Paolo!
conda-forge now has cdo-2.4.4. please give it a try
cheers
ralf
RE: Possible bug in 2.4.1 with NaN detection - Added by Paolo Davini about 1 month ago
Hi Ralf,
thanks, it works!
(though some tests are broken because of some extra safety check on healpix grids you introduced, but it is on our side to adapt!)
Cheers,
Paolo