Project

General

Profile

cdo fldcor result: nan how to avoid?

Added by Erik S over 1 year ago

cdo v2.1.0

cdo -output -fldcor on the attached ncdf results in: nan

Likely an issue with the landmask of the netcdfs. I can simply do cdo -fldcor -setmissvalue,99 n1.nc n2.nc but is this (setting any random value as missingValue) the cdo-recommended way to do?

Since I created the n1.nc and n2.nc as result of previous calculations (using python xarray), I like to learn how to write out them from python so that they are directly compatible with cdo fldcor (without getting "nan" or needing to re-set the missvalue...)

ncSstIpo.nc (51.5 KB) ncSstIpo.nc infile 2
ncEisIpo.nc (51.5 KB) ncEisIpo.nc infile 1

Replies (6)

RE: cdo fldcor result: nan how to avoid? - Added by Karin Meier-Fleischer over 1 year ago

Hi Erik,

you set the _FillValue and missing_value attributes of the variables to NaN. The _FillValue and missing_value attributes should be changed to a number outside the variable's data range.

cdo -fldcor -setmissval,-1e20 -setmissval,nan ncEisIpo.nc -setmissval,-1e20 -setmissval,nan ncSstIpo.nc out.nc

Result value is

ipo = -0.785741795791362

RE: cdo fldcor result: nan how to avoid? - Added by Erik S over 1 year ago

Hi Karin,

This means, cdo fldcor cannot/(does not) recognize "NaN" as a valid "ignore this grid cell"-marker to exclude this cell from correlation calculation?

Generalizing your answer to any kind of input netcdf and any kind of variable: This in turn means, I need to individually (per variable/ per physical problem) first check what the variable's data range is, and just then set the _FillValue and missing_value attributes to an arbritary number outside of it? That means, there is no all-time global-fixed-parameter/string as a cdo-predefined "marker" that cdo recognizes to inform cdo on: "this grid cell is to be ignored/skipped"?

RE: cdo fldcor result: nan how to avoid? - Added by Karin Meier-Fleischer over 1 year ago

The _FillValue/missing_value means exactly this, ignore the values for some calculations or set all to missing. See https://code.mpimet.mpg.de/projects/cdo/embedded/cdo.pdf#section.1.9

RE: cdo fldcor result: nan how to avoid? - Added by Erik S over 1 year ago

Sorry, a misunderstanding:
I refer to the specific value (in your case, you put: "-1e20") that can be set to the attributes "_FillValue/missing_value".
My questions is:
Instead of setting it to "-1e20" or to "9999" or to any other arbitrary number outside the variable range (the range might change by variable):
is there a cdo-predefined "keyword" or "string" like "nan", or "NaN" or "NA" or "xyz-whatever", so that I can do:

cdo -fldcor -setmissval,"keyword-for-skip-these-cells"  n1.nc   -setmissval,"keyword-for-skip-these-cells"  n2.nc
If there is such a cdo string-like-keyword?, that I don't need to worry about each variable's data range?  because what ever number the variable's data range is, a specific "keyword" is for sure never part of the variable's data range.

RE: cdo fldcor result: nan how to avoid? - Added by Erik S over 1 year ago

(not readable/well formated text in previous my reply):

If there is such a cdo string-like-keyword?, that I don't need to worry about each variable's data range? because what ever number the variable's data range is, a specific "keyword" is for sure never part of the variable's data range.

RE: cdo fldcor result: nan how to avoid? - Added by Karin Meier-Fleischer over 1 year ago

No, there is no keyword for it. CDO following the CF-convention (about missing values see https://cfconventions.org/Data/cf-conventions/cf-conventions-1.7/build/ch02s05.html) needs an explicit missing scalar. You can choose any value like 1e20 in the example above.

    (1-6/6)