Piping arithmetic gives different results than working on files
Added by Elio Campitelli 2 months ago
I was working on removing the ENSO signal on variables and noticed a strange behaviour. Using the attached era_sst.nc file, if I try to remove the ENSO effect within a single piped expression, I get a different result than if I create a file with the ENSO effect and then subtract it.
# Compute a enso-like index cdo -L -fldmean -sellonlatbox,190,240,-5,5 era_sst.nc enso_temp.nc cdo -L -ymonsub enso_temp.nc -ymonmean enso_temp.nc ensno.nc rm enso_temp.nc # Create random slope and intercept terms # in reality these would come from linear regression cdo -L random,era_sst.nc,42 slope.nc cdo -L random,era_sst.nc,45 intercept.nc cdo -L enlarge,era_sst.nc enso.nc enso_field.nc # Expression to compute the effect of ENSO enso_effect="-add -mul slope.nc enso_field.nc intercept.nc" # Compute residuals by piping the whole expression cdo -L -sub era_sst.nc $enso_effect residual_pipe.nc # note the message # cdo sub: Filling up stream2 >(pipe1.7)< by copying the first timestep. # era_sst.nc and the result of enso_effect should have the same number of # timesteps. Why the need to fill up the second stream? # Compute residuals by saving the effect to a file and then subtracting cdo -L $enso_effect enso_effect.nc cdo -L -sub era_sst.nc enso_effect.nc residual_seq.nc # The differnece is not zero! cdo -L info -sub residual_seq.nc residual_pipe.nc | head
Result of the last expression:
-1 : Date Time Level Gridsize Miss : Minimum Mean Maximum : Parameter ID 1 : 1980-01-01 00:00:00 0 10368 3911 : -0.0039062 2.4286e-05 0.0039062 : -1 2 : 1980-02-01 00:00:00 0 10368 3911 : -0.0033875 0.029031 0.061271 : -1 3 : 1980-03-01 00:00:00 0 10368 3911 : -0.24288 -0.12062 0.0037231 : -1 4 : 1980-04-01 00:00:00 0 10368 3911 : -0.86571 -0.43523 0.0022888 : -1 5 : 1980-05-01 00:00:00 0 10368 3911 : -0.78418 -0.39365 0.0022888 : -1 6 : 1980-06-01 00:00:00 0 10368 3911 : -0.57350 -0.28754 0.0022888 : -1 7 : 1980-07-01 00:00:00 0 10368 3911 : -0.0029907 0.12092 0.24309 : -1 8 : 1980-08-01 00:00:00 0 10368 3911 : -0.0029907 0.38596 0.76908 : -1 9 : 1980-09-01 00:00:00 0 10368 3911 : -0.0029907 0.29365 0.58588 : -1
Am I misunderstanding something important about how cdo pipes work? Or is this a bug in cdo itself?
era_sst.nc (20.9 MB) era_sst.nc |
Replies (2)
RE: Piping arithmetic gives different results than working on files - Added by Uwe Schulzweida 2 months ago
The problem occurs in arithmetic operations such as mul/div/sub/add when the first file has only one time step and the second has more than one time step. In this case, the information that there is only one time step is erroneously forwarded to other operators. We will fix this problem in the next CDO version.
A workaround is to swap the two input files in the multiplication:
enso_effect="-add -mul enso_field.nc slope.nc intercept.nc"
RE: Piping arithmetic gives different results than working on files - Added by Elio Campitelli 2 months ago
Thanks for the quick reply and the workaround!