Project

General

Profile

Help using 'ifthenelse'

Added by Sarah Grass 11 months ago

I am trying to infill the missing values in one netcdf with another netcdf (which also has some missing values). The spatial extents are the same for all netcdf files. I have
infile1 - A netcdf generated using expr,'var1=isMissval(var1)' from infile3 so it only contains 1 or 0 [0 where not NA and 1 where NA]
infile2 - a netcdf with data and some NA files that I want to fill in the missing values in infile3 with; it is generated from infile3 by using shifttime to shift things backwards 1 day
infile3 - a netcdf with data and some NA files that was used to generate infile1 and infile2

the following code is used: cdo ifthenelse infile1 infile2 infile3 outfile

The issue is the outfile is identical to infile2 and does not appear to be following the logic of when infile 1 = 0 take the values from infile 3. Is this a bug or am I doing something wrong?

I have included a shorter timeseries version of each file.

I am using version 2.05 if that matters.


Replies (7)

RE: Help using 'ifthenelse' - Added by Karin Meier-Fleischer 11 months ago

Hi Sarah,

the output is not the same as infile2_sample.nc.

cdo diff outfile.nc infile2_sample.nc 
               Date     Time   Level Gridsize    Miss    Diff : S Z  Max_Absdiff Max_Reldiff : Parameter ID
     1 : 2002-10-01 00:00:00       0   352080  242150   71822 : F T       98.000      1.0000 : -1         
     2 : 2002-10-02 00:00:00       0   352080  282124   88649 : F T       88.000      1.0000 : -1         
     3 : 2002-10-03 00:00:00       0   352080  291083   51073 : F T       92.000      1.0000 : -1         
     4 : 2002-10-04 00:00:00       0   352080  290929   46496 : F T       90.000      1.0000 : -1         
     5 : 2002-10-05 00:00:00       0   352080  326528   51082 : F T       88.000      1.0000 : -1         
     6 : 2002-10-06 00:00:00       0   352080  203752    8970 : F T       86.000      1.0000 : -1         
  6 of 6 records differ

RE: Help using 'ifthenelse' - Added by Sarah Grass 11 months ago

It still isn't behaving the way I would expect it to. If you interrogate the timeseries in a single cell (in this case lat 40, lon 17):

Infile1 Infile2 Infile 3 Expected Output Actual Output
0 NA 31 31 NA
1 0 NA 0 0
0 NA 0 0 NA

So the first step of checking if infile != 0 and != miss take infile 2 is working, but the second step of if infile1=0 and infile1!= miss take infile 3 is not working and is instead just adding missing values when there are non-missing values in Infile3.

RE: Help using 'ifthenelse' - Added by Karin Meier-Fleischer 11 months ago

Hm, if I use the following simple example everything works as expected using ifthenelse operator:

cdo -f nc -random,global_1,100000 random_data.nc                      # generate random data
cdo -f nc  -remapbil,global_1 -topo topo.nc                           # generate topographic data for mask creation
cdo -gtc,0 -expr,'topo=((topo>=2000)?topo:(topo/0))' topo.nc land.nc  # land >= 2000: 1, ocean: missing
cdo -ifthenelse land.nc  random_data.nc topo.nc outfile_land.nc       # if land data >0 use random_data data otherwise use the topo data

RE: Help using 'ifthenelse' - Added by Sarah Grass 11 months ago

Glad that it is working for you, do you have any suggestions on what I can troubleshoot since it is not working for me?

RE: Help using 'ifthenelse' - Added by Karin Meier-Fleischer 11 months ago

I found a good sub-region in your datasets that can be used to have a closer look at the results.

cdo -seltimestep,1 -sellonlatbox,-115.0,-114.96,49.5,49.45 infile1_sample.nc in1.nc
cdo -seltimestep,1 -sellonlatbox,-115.0,-114.96,49.5,49.45 infile2_sample.nc in2.nc
cdo -seltimestep,1 -sellonlatbox,-115.0,-114.96,49.5,49.45 infile3_sample.nc in3.nc

cdo -ifthenelse in1.nc in2.nc in3.nc outfile.nc
ncdump -v NDSI_Snow_Cover in1.nc  | sed -e '1,/data:/d' -e '$d'

NDSI_Snow_Cover =
  0, 1, 1, 1, 1, 1, 1, 1, 1, 0,
  0, 0, 0, 1, 1, 0, 0, 0, 1, 1,
  0, 0, 0, 0, 0, 0, 1, 1, 1, 0,
  0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
  0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
  0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
  0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
  0, 0, 0, 0, 0, 0, 0, 0, 1, 1,
  0, 0, 0, 0, 0, 0, 0, 1, 1, 1,    
  0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
  0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
  0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ;
ncdump -v NDSI_Snow_Cover in2.nc  | sed -e '1,/data:/d' -e '$d'

 NDSI_Snow_Cover =
  0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
  0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
  0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
  0, 0, 0, 0, 0, 0, 13, 13, 0, 0,
  12, 12, 12, 12, 12, 12, 23, 23, 23, 13,
  28, 28, 17, 31, 31, 29, 29, 29, 18, 0,
  47, 37, 37, 37, 21, 21, 21, 23, 23, 14,
  54, 54, 21, 21, 21, 0, 16, 16, 18, 18,
  45, 21, 21, 48, 21, 21, 33, 37, 37, 37,
  55, 55, 57, 57, 57, 50, 50, 45, 45, 45,
  47, 55, 55, 57, 57, 57, 45, 45, 45, 30,
  47, 47, 58, 58, 58, 48, 48, 48, 28, 28 ;
ncdump -v NDSI_Snow_Cover in3.nc  | sed -e '1,/data:/d' -e '$d'

 NDSI_Snow_Cover =
  0, _, _, _, _, _, _, _, _, 59,
  0, 0, 0, _, _, 0, 0, 0, _, _,
  0, 0, 0, 4, 4, 0, _, _, _, 0,
  0, 0, 0, 13, 13, 0, 12, 14, 22, 34,
  32, 32, 32, 13, 37, 37, 37, 31, 31, 12,
  32, 32, 26, 45, 45, 45, 54, 54, 31, 32,
  68, 49, 49, 72, 72, 72, 54, 54, 40, 39,
  74, 74, 49, 61, 61, 55, 55, 55, _, _,
  81, 60, 60, 60, 61, 61, 62, _, _, _,
  87, 87, 61, 70, 70, 70, 70, 65, 65, 50,
  85, 67, 67, 74, 74, 74, 69, 69, 69, 60,
  79, 79, 68, 68, 74, 73, 73, 73, 66, 66 ;
ncdump -v NDSI_Snow_Cover outfile.nc  | sed -e '1,/data:/d' -e '$d'

 NDSI_Snow_Cover =
  0, 0, 0, 0, 0, 0, 0, 0, 0, 59,
  0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
  0, 0, 0, 4, 4, 0, 0, 0, 0, 0,
  0, 0, 0, 13, 13, 0, 12, 14, 22, 34,
  32, 32, 32, 13, 37, 37, 37, 31, 31, 12,
  32, 32, 26, 45, 45, 45, 54, 54, 31, 32,
  68, 49, 49, 72, 72, 72, 54, 54, 40, 39,
  74, 74, 49, 61, 61, 55, 55, 55, 18, 18,
  81, 60, 60, 60, 61, 61, 62, 37, 37, 37,
  87, 87, 61, 70, 70, 70, 70, 65, 65, 50,
  85, 67, 67, 74, 74, 74, 69, 69, 69, 60,
  79, 79, 68, 68, 74, 73, 73, 73, 66, 66 ;
Closer looks at some lines of the results:

1. line
  0,  1,  1,  1,  1,  1,  1,  1,  1,  0,    in1
  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,    in2
  0,  _,  _,  _,  _,  _,  _,  _,  _, 59,    in3
  0,  0,  0,  0,  0,  0,  0,  0,  0, 59,    out

2. line
  0,  0,  0,  0,  0,  0,  0,  0,  1,  1,    in1
 54, 54, 21, 21, 21,  0, 16, 16, 18, 18,    in2
 74, 74, 49, 61, 61, 55, 55, 55,  _,  _,    in3
 74, 74, 49, 61, 61, 55, 55, 55, 18, 18,    out

9. line
  0,  0,  0,  0,  0,  0,  0,  1,  1,  1,    in1
 45, 21, 21, 48, 21, 21, 33, 37, 37, 37,    in2    
 81, 60, 60, 60, 61, 61, 62,  _,  _,  _,    in3
 81, 60, 60, 60, 61, 61, 62, 37, 37, 37,    out

Looks good to me. Which version (cdo -V) are you using?

RE: Help using 'ifthenelse' - Added by Sarah Grass 11 months ago

I am using CDO ver 2.05 as I am running on cygwin and that is the most recent available in the library, since it is working for you I assume it is likely a version issue then?

RE: Help using 'ifthenelse' - Added by Karin Meier-Fleischer 11 months ago

I used version 2.0.5 too.

    (1-7/7)