Are infinties considered missing by CDO?
Added by Matt Thompson over 2 years ago
We recently found that some files we have been producing have stray infinities in them. Obviously a bad thing and we're looking to correct that, but I first thought "No we aren't" because:
$ cdo --version
Climate Data Operators version 2.0.5 (https://mpimet.mpg.de/cdo)
...
$ cdo infon -selname,T2MDEW badfile.nc4
cdo(1) selname: Process started
-1 : Date Time Level Gridsize Miss : Minimum Mean Maximum : Parameter name
1 : 1984-08-18 13:30:00 0 259920 14 : 186.19 275.74 301.42 : T2MDEW
cdo(1) selname: Processed 259920 values from 147 variables over 1 timestep.
cdo infon: Processed 259920 values from 1 variable over 1 timestep [0.06s 27MB].
So sure we have some missing values, but the Maximum is 301, not Infinity.
And yet:
$ ncdump -v T2MDEW badfile.nc4 | rg infinity
191.1258, 188.2285, Infinityf, Infinityf, Infinityf, Infinityf,
Infinityf, Infinityf, 198.5257, 202.6711, 204.7231, 206.4343, 208.0729,
Infinityf, Infinityf, Infinityf, Infinityf, Infinityf, Infinityf,
Infinityf, Infinityf, 199.5454, 203.4404, 206.7863, 209.5758, 210.0919,
There are 14 Infinityf there and CDO says there are 14 missing values. The odd thing is, we've set the missing value to 1e15:
T2MDEW:_FillValue = 1.e+15f ; T2MDEW:missing_value = 1.e+15f ; T2MDEW:fmissing_value = 1.e+15f ; T2MDEW:vmin = -1.e+15f ; T2MDEW:vmax = 1.e+15f ; T2MDEW:valid_range = -1.e+15f, 1.e+15f ;
Is this the expected behavior of CDO in re Infinity? I suppose from the CF Conventions it says:
Since version 2.4 the NUG defines missing data as all values outside of the valid_range, and specifies how the valid_range should be defined from the _FillValue (which has library specified default values) if it hasn’t been explicitly specified.
And I guess Infinity is definitely outside that range!
Thanks,
Matt
Replies (1)
RE: Are infinties considered missing by CDO? - Added by Karin Meier-Fleischer over 2 years ago
Hi Matthew,
I would say ncdump is able to identyfy infinityf in the netCDF file and CDO automatically sets Infinityf values to the missing value. One problem can be that the maximum value of valid_range and the vmax attributes are set to the missing value too.