Project

General

Profile

Help to get daily anomalies with CDO

Added by Kendy Silvério about 6 years ago

Hi, dear all!

I have daily precipitation data (from Jan 1, 1974 - Dec 31, 1999), which are attached. And I would like to get from these data daily anomalies only for DJF-season without feb29. I tried to follow some ideas here (https://code.mpimet.mpg.de/boards/1/topics/6612?r=6652#message-6652), but I am getting an error like cdo ydaysub (Abort): Day of year 344 already allocatd (date=19751203)! In this regards, is there any suggestion?

Any suggestion would be very welcome!

Thanks in advance for your quickly feedback!

Cheers

Silverio


Replies (12)

RE: Help to get daily anomalies with CDO - Added by Karin Meier-Fleischer about 6 years ago

Hi Kendy,

could you say what you have tried to do with your data when the error occured. It's not clear what you used from the link.

-Karin

RE: Help to get daily anomalies with CDO - Added by Kendy Silvério about 6 years ago

Hi Karim,

1. I would like to get firstly a time series of anomaly from Jan 01, 1974 to Dec 31, 1999;

2. Then extract only DJF time series without Feb29 from point 1 (above);

I tried following this

cdo -O -select,season=DJF,startdate=1974-12-01,enddate=1999-02-28 -del29feb ppt-obs.nc a1.nc
cdo -O -timselmean,12 a1.nc a2.nc
#cdo -O -ydaymean a2.nc a3.nc
#cdo -O -mergetime -selmon,12 a3.nc -selmon,1,2 a3.nc a4.nc
cdo -selmon,12,1,2 a2.nc a3.nc
cdo -r settaxis,1974-12-01,00:00:00,1day a3.nc a4.nc
cdo -b 32 -ydaysub a4.nc a5.nc
cdo setmissval,-9999.9 a5.nc a6.nc
rm a1.nc a2.nc a3.nc aux4.nc aux5.nc

and I got this:

cdo select: Started child process "del29feb ppt-obs.nc (pipe1.1)".
cdo(2) del29feb: Processed 3245580 values from 1 variable over 9496 timesteps [0.35s]
cdo select: Processed 3026700 values from 1 variable over 9185 timesteps [0.35s 52MB]
cdo timselmean: Processed 3026700 values from 1 variable over 8850 timesteps [0.06s 49MB]
cdo selmonth: Processed 64296 values from 1 variable over 738 timesteps [0.00s 47MB]
cdo settaxis (Warning): Time bounds unsupported by this operator, removed!
cdo settaxis: Processed 64296 values from 1 variable over 188 timesteps [0.00s 47MB]

cdo ydaysub (Abort): Too few streams specified! Operator needs 2 input and 1 output streams.
cdo setmissval: Processed 6156 values from 1 variable over 18 timesteps [0.00s 46MB]

Furthermore, the results have less time steps (18)

cdo infon a6.nc
-1 : Date Time Level Gridsize Miss : Minimum Mean Maximum : Parameter name
1 : 1998-12-03 00:00:00 0 342 229 : 0.20120 2.9911 13.700 : cxc
2 : 1998-12-08 00:00:00 0 342 229 : 0.0064000 3.0129 15.660 : cxc
3 : 1998-12-13 00:00:00 0 342 229 : 0.016000 3.2231 25.800 : cxc
4 : 1998-12-18 00:00:00 0 342 229 : 0.0000 3.2906 11.004 : cxc
5 : 1998-12-23 00:00:00 0 342 229 : 0.019467 3.4318 12.313 : cxc
6 : 1998-12-28 00:00:00 0 342 229 : 0.022800 3.4199 16.920 : cxc
7 : 1999-01-02 00:00:00 0 342 229 : 0.0000 2.8513 9.6116 : cxc
8 : 1999-01-07 00:00:00 0 342 229 : 0.0000 3.4478 10.784 : cxc
9 : 1999-01-12 00:00:00 0 342 229 : 0.10295 3.4408 10.157 : cxc
10 : 1999-01-17 00:00:00 0 342 229 : 0.0000 3.6068 11.572 : cxc
11 : 1999-01-22 00:00:00 0 342 229 : 0.025867 3.8734 10.523 : cxc
12 : 1999-01-27 00:00:00 0 342 229 : 0.0000 3.8852 9.7500 : cxc
13 : 1999-02-01 00:00:00 0 342 229 : 0.013067 3.5964 10.315 : cxc
14 : 1999-02-06 00:00:00 0 342 229 : 0.0000 3.9851 11.960 : cxc
15 : 1999-02-11 00:00:00 0 342 229 : 0.0000 4.2687 11.738 : cxc
16 : 1999-02-16 00:00:00 0 342 229 : 0.070667 4.1123 17.915 : cxc
17 : 1999-02-21 00:00:00 0 342 229 : 0.025371 3.7058 14.900 : cxc
18 : 1999-02-26 00:00:00 0 342 229 : 0.095733 3.5056 13.660 : cxc
cdo infon: Processed 6156 values from 1 variable over 18 timesteps [0.00s 46MB]

So If there's any suggestion to do what I want, I'd love appreciate it.

Many many thanks for your help in advance, Karin.

Silverio

RE: Help to get daily anomalies with CDO - Added by Kendy Silvério about 6 years ago

As final results, I would like to get DJF anomaly time series (90 days) for each consedered year (1974-1999).

Thanks

RE: Help to get daily anomalies with CDO - Added by Karin Meier-Fleischer about 6 years ago

The abort message tells you what is going wrong, there is one input file missing.

cdo ydaysub (Abort): Too few streams specified! Operator needs 2 input and 1 output streams.

Do e.g.

cdo -b 32 -ydaysub a4.nc -ydaymean a4.nc a5.nc

-Karin

RE: Help to get daily anomalies with CDO - Added by Karin Meier-Fleischer about 6 years ago

Try

cdo -select,season=DJF,startdate=1974-12-01,enddate=1999-02-28 -del29feb ppt-obs.nc tmp1.nc
cdo -ydaysub tmp1.nc -ydaymean tmp1.nc anom_file.nc
cdo -O -mergetime -selmon,12 anom_file.nc -selmon,1,2 anom_file.nc anom_file_DJF.nc

-Karin

RE: Help to get daily anomalies with CDO - Added by Kendy Silvério about 6 years ago

Dear Karin,

It works perfect and thanks for the solution.

However, I have another question regarding time:calendar = "365_day" in obtained results (anom_DJF.nc, see output bellow), since in original data (ppt-obs.nc) there is "standard" in section "time:calendar =......" (see output bellow too), which allow me to convert in .ctl without additional effort.

However, now in the final results (anom_DJF.nc) I have time:calendar = "365_day", while I need to have there time:calendar = "standard" with leap calendar by default since I would like to convert it in .ctl and that actually is not beeing possible to do.

May you give me some ideas?

cdo -select,season=DJF,startdate=1974-12-01,enddate=1999-02-28 -del29feb ppt-obs.nc aux.nc
cdo -ydaysub aux.nc -ydaymean aux.nc anomalies.nc
cdo -O -mergetime -selmon,12 anomalies.nc -selmon,1,2 anomalies.nc aux1.nc
cdo setmissval,-9999.9 aux1.nc anom_DJF.nc

Output of obtained results following your solution
ncdump -h anom_DJF.nc
netcdf anom_DJF {
dimensions:
time = UNLIMITED ; // (2250 currently)
lon = 19 ;
lat = 18 ;
variables:
double time(time) ;
time:standard_name = "time" ;
time:units = "days since 1974-12-1 00:00:00" ;
time:calendar = "365_day" ;
time:axis = "T" ;
float lon(lon) ;
lon:standard_name = "longitude" ;
lon:long_name = "longitude" ;
lon:units = "degrees_east" ;
lon:axis = "X" ;
float lat(lat) ;
lat:standard_name = "latitude" ;
lat:long_name = "latitude" ;
lat:units = "degrees_north" ;
lat:axis = "Y" ;
float cxc(time, lat, lon) ;
cxc:long_name = "cx_chuva" ;
cxc:_FillValue = -9999.9f ;
cxc:missing_value = -9999.9f ;

// global attributes:
:CDI = "Climate Data Interface version ?? (http://mpimet.mpg.de/cdi)" ;
:Conventions = "CF-1.6" ;
:history = "Fri Oct 26 13:07:03 2018: cdo setmissval,-9999.9 aux2.nc anom_DJF.nc\n",
"Fri Oct 26 13:07:03 2018: cdo -r settaxis,1974-12-01,00:00:00,1day aux1.nc aux2.nc\n",
"Fri Oct 26 13:07:03 2018: cdo -O -mergetime -selmon,12 anomalies.nc -selmon,1,2 anomalies.nc aux1.nc\n",
"Fri Oct 26 13:07:03 2018: cdo -ydaysub aux.nc -ydaymean aux.nc anomalies.nc\n",
"Fri Oct 26 13:07:03 2018: cdo -select,season=DJF,startdate=1974-12-01,enddate=1999-02-28 -del29feb ppt-obs.nc aux.nc\n",
"Thu Oct 25 11:20:53 2018: cdo -f nc import_binary chuva-obs.ctl ppt-obs.nc" ;
:CDO = "Climate Data Operators version 1.9.3 (http://mpimet.mpg.de/cdo)" ;

Here is output of original data

ncdump -h ppt-obs.nc
netcdf ppt-obs {
dimensions:
time = UNLIMITED ; // (9496 currently)
lon = 19 ;
lat = 18 ;
variables:
double time(time) ;
time:standard_name = "time" ;
time:units = "hours since 1-1-1 00:00:00" ;
time:calendar = "standard" ;
time:axis = "T" ;
float lon(lon) ;
lon:standard_name = "longitude" ;
lon:long_name = "longitude" ;
lon:units = "degrees_east" ;
lon:axis = "X" ;
float lat(lat) ;
lat:standard_name = "latitude" ;
lat:long_name = "latitude" ;
lat:units = "degrees_north" ;
lat:axis = "Y" ;
float cxc(time, lat, lon) ;
cxc:long_name = "cx_chuva" ;
cxc:_FillValue = 777.7f ;
cxc:missing_value = 777.7f ;

// global attributes:
:CDI = "Climate Data Interface version ?? (http://mpimet.mpg.de/cdi)" ;
:Conventions = "CF-1.6" ;
:history = "Thu Oct 25 11:20:53 2018: cdo -f nc import_binary chuva-obs.ctl ppt-obs.nc" ;
:CDO = "Climate Data Operators version 1.9.3 (http://mpimet.mpg.de/cdo)" ;
}

Again many thanks fou your quickly feedback and help.

Silverio

RE: Help to get daily anomalies with CDO - Added by Karin Meier-Fleischer about 6 years ago

Just one question: why do you need to create a .ctl file?

RE: Help to get daily anomalies with CDO - Added by Kendy Silvério about 6 years ago

Hi Karin,

I am working with some scrips that uses .ctl files as input data.

However, before that I use CDO to make some operations in the data then I convert it in ctl for my scripts.

Thanks,

Silverio

RE: Help to get daily anomalies with CDO - Added by Kendy Silvério about 6 years ago

Hi, Karin
I just added in commandline bellow the line "cdo setcalendar,standard aux2.nc anom_DJF.nc" and it jast works

cdo -select,season=DJF,startdate=1974-12-01,enddate=1999-02-28 -del29feb ppt-obs.nc aux.nc
cdo -ydaysub aux.nc -ydaymean aux.nc anomalies.nc
cdo -O -mergetime -selmon,12 anomalies.nc -selmon,1,2 anomalies.nc aux1.nc
cdo setmissval,-9999.9 aux1.nc aux2.nc
cdo setcalendar,standard aux2.nc anom_DJF.nc
./converte_dados_nc.sh anom_DJF.nc 0 0 0 -9999.9 anom_DJF_rb.bin
-----------------------------------
TERMINOU!
NT_TOTAL:2250

MODELO CTL:
dset ^anom_DJF_rb.bin
title
undef -9999.9
xdef 19 linear 15.25 1.5
ydef 18 linear -34.75 1.5
zdef 1 linear 0 1
tdef 2250 linear 00Z01DEC1974 1440mn
vars 1
cxc=>cxc 0 t,y,x cx_chuva
endvars


Thanks a lot Karin for your help.
I really admire you for a great work done by you.
Thanks

Silverio

RE: Help to get daily anomalies with cdo - Added by Kendy Silvério about 6 years ago

Hi Karin,

I am sorry for my question, since it is not about cdo. However, may you know how to shade the area of great wind's (u,v) values with grads? I mean area of great winds after ploting streamlines (set gxout stream)?

Thank you for your qquickly feedback in advance.

Kenedy

RE: Help to get daily anomalies with CDO - Added by Karin Meier-Fleischer about 6 years ago

Hi Kenedy,

sorry, but I can't help with GrADS.

-Karin

    (1-12/12)