CDO

hi Oliver!

i personally do not expect this to be a minor change to the software. I cannot even download the R-package (RClimDex). I think there is currently no time to do this. Maybe with some external funding ;-)

sry
ralf

Hi Ralf,

A colleague of mine has recently already calculated almost all the 27 ETCCDI (http://etccdi.pacificclimate.org/list_27_indices.shtml) indices using CDO. Here they are for the rainfall indices (the temperature ones are in another file): https://github.com/Peter-Gibson/climate/blob/CLIMATE-937/examples/ETCCDI_precip.py

In the most basic cases, a single CDO operator is required. For example rx1day (maximum daily rainfall per year) is cdo yearmax ifile.nc ofile.nc. In other cases, some piping is needed. For example cdd (consecutive dry days) is: cdo yearmax -consects -lec,1 ifile.nc ofile.nc

There is already specialised R software for calculating the ETCCDI indices, like RClimDex as you mentioned, or Climpact2. Although these software work, they are: non-trivial to install; slow compared to CDO; cannot be used in the command line like CDO. And with the CMIP6 suite of model runs coming out soon, computation time with the R software will become even more of a bottleneck. One can leverage the work already done in CDO over the years to calculate all the indices fairly easily (the most challenging might be the bootstrapping for TX/N10/90p indices). However, it would be nice if these indices were made "officially" part of CDO for ease of use. I recommend 27 new operators. Could be used like:

cdo etccdi_rx1day ifile.nc ofile.nc
cdo etccdi_cdd ifile.nc ofile.nc

If the CDO team considers this worthwhile and beneficial to the community, Peter and I are happy to discuss further.

Thanks,
Oliver

Hi Oliver,

I'm starting to integrate ETCCDI indices definitions in CDOs. First, I compare percentile-based indices, which were calculated using CDOs and Rclimdex. Thank you for your offer, if necessary I will come back to it.

Best regards and thanks for your help,
Fabi

Hi,

I finished my analysis and could show that the results of the climdex package for "absolute" and "threshold" indices can be created with CDOs. txx, txn, tnx, tnn, dtr, prcptot are missing in the spectrum of eca_* operators but can be easily produced with statistical operators. The duration and percentile indices on the other hand cannot be calculated with CDOs. I summarized the results on slides:
https://slides.com/wachsylon/cdoetccdi#/1

I recommend 27 new operators

Besides the 6 missing eca_* operators for ETCCDI indices, an argument for 27 new operators is that their names can show that they use the methods recommended by ETCCDI. Also, enabling ETCCDI methods for operators for the percentile indices with an option will not be user friendly.

If these 27 new operators do not replace the eca_* operators (maybe because they are essential for some users) but rather are additional operators, we will have lots of operators of which 9 produce the same results as the old 9. This could lead to confusion. Another aspect is that ETCCDI operators may be not as modular as the eca_* operators. For example, the rclimdex package delets the 29th of feb. in leap years. Will we adopt this method and use it everytime?

If you have information about meta data standards for these clima indices in any projects, I would be glad if you can inform me about that. Discussing meta data will be one of the next steps.
Best regards,
Fabi

Hi,Oliver,I just find the rainfall indices on the website,where can I find the temperature ones ?
Thanks

the documentation of the etccdi operators is part of the document for the eca indices https://code.mpimet.mpg.de/projects/cdo/embedded/cdo_eca.pdf

Ralf Mueller wrote in RE: ETCCDI indices with CDO:

the documentation of the etccdi operators is part of the document for the eca indices https://code.mpimet.mpg.de/projects/cdo/embedded/cdo_eca.pdf

Hi,how can I use etccdi operators? (the question is :Operator >etccdi< not found!)

etccdi is only the name of a group of climate indices. the operators of that group use etccdi as prefix. you can check all operators from the group with

 cdo -h etccdi

NAME
    etccdi_tx90p, etccdi_tx10p, etccdi_tn90p, etccdi_tn10p, etccdi_r95p, 
    etccdi_r99p - 
    ETCCDI conform index for a reference periode calculated with bootstrapping

SYNOPSIS
    <operator>,n,startboot,endboot[,m]  infile1 infile2 infile3 outfile

DESCRIPTION
    This module enables to compute Climate Extremes Indices according to the method recommended
    by the Expert Team on Climate Change Detection and Indices. It differs from the
    corresponding eca_* indices by applying bootstrapping for a reference period
    (see Zhang et al. 2005) given by startboot and endboot and using the R-type 8 method 
    for percentile calculation.
    A requirement for correct percentile calculation is that
    CDO_PCTL_NBINS>=window*(endboot-startboot+1)*(sizeof(double)/sizeof(int))+2
    This demands for high working storage since the entire data of the bootstrapping interval
    need to be hold in storage. Otherwise, a histogram is used to calculate the percentile.
    infile2 (infile3) contains the daily minimum (maximum) of the bootstrapping interval.
    If m=m, the output variable will be saved monthly, otherwise with yearly frequency.

OPERATORS
    etccdi_tx90p  Percentage of Days when Daily Maximum Temperature is Above the 90th Percentile
    etccdi_tx10p  Percentage of Days when Daily Maximum Temperature is Below the 10th Percentile
    etccdi_tn90p  Percentage of Days when Daily Minimum Temperature is Above the 90th Percentile
    etccdi_tn10p  Percentage of Days when Daily Minimum Temperature is Below the 10th Percentile
    etccdi_r95p   Annual Total Precipitation when Daily Precipitation Exceeds the 95th Percentile of Wet Day Precipitation
    etccdi_r99p   Annual Total Precipitation when Daily Precipitation Exceeds the 99th Percentile of Wet Day Precipitation

PARAMETER
    n          INTEGER  Window days, number of timesteps
    startboot  INTEGER  First year of bootstrapping interval
    endboot    INTEGER  Last year of bootstrapping interval
    m          CHARACTER Output frequency

ENVIRONMENT
    CDO_PCTL_NBINS
        Sets the number of histogram bins. The default number is 101.

Hi，I use the oeprator etccdi_r95p, here I got another question, I don't understand.
$ cdo etccdi_r95p,5,1986,2005 CN05.1_Pre_1961_1990_daily_1x1_delete.nc -ydrunmin,5 CN05.1_Pre_1961_1990_daily_1x1_delete.nc -ydrunmax,5 CN05.1_Pre_1961_1990_daily_1x1_delete.nc r95p.nc
cdo(1) ydrunmin: Process started
cdo(2) ydrunmax: Process started
cdo etccdi_r95p: 'CN05.1_Pre_1961_1990_daily_1x1_delete.nc'

cdo etccdi_r95p (Abort): The interval start year '5' is before infile start year '1961'.
terminate called recursively
terminate called without an active exception
Aborted (core dumped)

Thanks for your help

fang Fang wrote in RE: ETCCDI indices with CDO:

Hi，I use the oeprator etccdi_r95p, here I got another question, I don't understand.
$ cdo etccdi_r95p,5,1986,2005 CN05.1_Pre_1961_1990_daily_1x1_delete.nc -ydrunmin,5 CN05.1_Pre_1961_1990_daily_1x1_delete.nc -ydrunmax,5 CN05.1_Pre_1961_1990_daily_1x1_delete.nc r95p.nc
cdo(1) ydrunmin: Process started
cdo(2) ydrunmax: Process started
cdo etccdi_r95p: 'CN05.1_Pre_1961_1990_daily_1x1_delete.nc'

cdo etccdi_r95p (Abort): The interval start year '5' is before infile start year '1961'.
terminate called recursively
terminate called without an active exception
Aborted (core dumped)

Thanks for your help

and when I use etccdi_tn90p, I got that
$ cdo etccdi_tx90p,5,1961,1990,m tmax.nc -ydrunmin,5 tmax.nc -ydrunmax,5 tmax.nc tn90p.nc
cdo(1) ydrunmin: Process started
cdo(2) ydrunmax: Process started
cdo etccdi_tx90p: 'tmax.nc'
cdo etccdi_tx90p: 'tmax.nc'

cdo etccdi_tx90p (Abort): infile1 cannot be a pipe
terminate called recursively
terminate called without an active exception
Aborted (core dumped)

I meet a similar problem using the oeprator etccdi_r95p, how did you solve this error?
$ cdo etccdi_r95p,1979,2008 file.nc -ydrunmin,5 file.nc -ydrunmax,5 file.nc r95p.nc
cdo(1) ydrunmin: Process started
cdo(2) ydrunmax: Process started
cdo etccdi_r95p: 'file.nc'
cdo etccdi_r95p: 'file.nc' cdo etccdi_r95p (Abort): infile1 cannot be a pipe
terminate called without an active exception
terminate called recursively
Aborted (core dumped)

The error message is a bit misleading. Unfortunately, the operator etccdi_r95p cannot be combined with other operators. This CDO command must therefore be split into three parts:

cdo ydrunmin,5 file.nc drunmin.nc
cdo ydrunmax,5 file.nc drunmax.nc
cdo etccdi_r95p,1979,2008 file.nc drunmin.nc drunmax.nc r95p.nc