cdo fldcor result: nan how to avoid?
Added by Erik S about 2 years ago
cdo v2.1.0
cdo -output -fldcor on the attached ncdf results in: nan
Likely an issue with the landmask of the netcdfs. I can simply do cdo -fldcor -setmissvalue,99 n1.nc n2.nc but is this (setting any random value as missingValue) the cdo-recommended way to do?
Since I created the n1.nc and n2.nc as result of previous calculations (using python xarray), I like to learn how to write out them from python so that they are directly compatible with cdo fldcor (without getting "nan" or needing to re-set the missvalue...)
ncSstIpo.nc (51.5 KB) ncSstIpo.nc | infile 2 | ||
ncEisIpo.nc (51.5 KB) ncEisIpo.nc | infile 1 |
Replies (6)
RE: cdo fldcor result: nan how to avoid? - Added by Karin Meier-Fleischer about 2 years ago
Hi Erik,
you set the _FillValue and missing_value attributes of the variables to NaN. The _FillValue and missing_value attributes should be changed to a number outside the variable's data range.
cdo -fldcor -setmissval,-1e20 -setmissval,nan ncEisIpo.nc -setmissval,-1e20 -setmissval,nan ncSstIpo.nc out.nc
Result value is
ipo = -0.785741795791362
RE: cdo fldcor result: nan how to avoid? - Added by Erik S about 2 years ago
Hi Karin,
This means, cdo fldcor cannot/(does not) recognize "NaN" as a valid "ignore this grid cell"-marker to exclude this cell from correlation calculation?
Generalizing your answer to any kind of input netcdf and any kind of variable: This in turn means, I need to individually (per variable/ per physical problem) first check what the variable's data range is, and just then set the _FillValue and missing_value attributes to an arbritary number outside of it? That means, there is no all-time global-fixed-parameter/string as a cdo-predefined "marker" that cdo recognizes to inform cdo on: "this grid cell is to be ignored/skipped"?
RE: cdo fldcor result: nan how to avoid? - Added by Karin Meier-Fleischer about 2 years ago
The _FillValue/missing_value means exactly this, ignore the values for some calculations or set all to missing. See https://code.mpimet.mpg.de/projects/cdo/embedded/cdo.pdf#section.1.9
RE: cdo fldcor result: nan how to avoid? - Added by Erik S about 2 years ago
Sorry, a misunderstanding:
I refer to the specific value (in your case, you put: "-1e20") that can be set to the attributes "_FillValue/missing_value".
My questions is:
Instead of setting it to "-1e20" or to "9999" or to any other arbitrary number outside the variable range (the range might change by variable):
is there a cdo-predefined "keyword" or "string" like "nan", or "NaN" or "NA" or "xyz-whatever", so that I can do:
cdo -fldcor -setmissval,"keyword-for-skip-these-cells" n1.nc -setmissval,"keyword-for-skip-these-cells" n2.nc
If there is such a cdo string-like-keyword?, that I don't need to worry about each variable's data range? because what ever number the variable's data range is, a specific "keyword" is for sure never part of the variable's data range.
RE: cdo fldcor result: nan how to avoid? - Added by Erik S about 2 years ago
(not readable/well formated text in previous my reply):
If there is such a cdo string-like-keyword?, that I don't need to worry about each variable's data range? because what ever number the variable's data range is, a specific "keyword" is for sure never part of the variable's data range.
RE: cdo fldcor result: nan how to avoid? - Added by Karin Meier-Fleischer about 2 years ago
No, there is no keyword for it. CDO following the CF-convention (about missing values see https://cfconventions.org/Data/cf-conventions/cf-conventions-1.7/build/ch02s05.html) needs an explicit missing scalar. You can choose any value like 1e20 in the example above.