Project

General

Profile

Forums » Support »

subtracting yearly statistics or running means

Added by Adrian Tompkins over 8 years ago

I have a file with a daily timeseries spanning many years. I want to do two things:

1. Subtract yearly statistics, e.g. for each day subtract the minimum that occurs within the relevant year.

cdo yearmin ifile.nc yearmin.nc
cdo sub ifile.nc yearmin.nc

doesn't work because cdo doesn't know to match timestep 0,1,....364 to the timstep 364 in the second file... I feel like I need a "yearsub" command which doesn't seem to exist.

2. Subtract a running mean... Here I can do it, but cutting the infile to have the same length as the running mean, but is there a way of having the running mean reduce the window near the start and end of the file to produce an output file that is the same length as the input?

At the moment I do
cdo runmean,5 ifile runmean.nc
cdo seltimestep,start+2,end-2 ifile cutifile.nc
cdo sub cutifile.nc runmean.nc ofile.nc

but what I would really like to do is
cdo runmean_leaving_the_tails_in,5 ifile runmean.nc (<-runmean has same length as ifile...)
cdo sub ifile runmean.nc ofile.nc

Any tips much appreciated...

Adrian


Replies (2)

RE: subtracting yearly statistics or running means - Added by Adrian Tompkins over 8 years ago

I'm going to answer my own question... I presume that isn't against protocol! ;)

I realized that, if the window length is N, by pasting the first N steps, time-shifted, to the start of the file, the running stats are calculated with the same number of points but with a rotated window (i.e. step 1 uses a window 1:N, step 2 uses 1:N, etc until step N/2+1 uses a window 2:N+1 as usual). Ditto for the file tail of course.

Code would be:

date1=`cdo -s showdate ${ifile}.nc | awk '{print $1;}'` # start date
date2=`cdo -s showdate ${ifile}.nc | awk '{print $NF;}'` # end date

#
  1. 1. in order to do running statistics we will paste on a repeated window
  2. at start and end #
  1. First calc first nrun dates and select
    win1_1=`echo $date1 | sed "s/-//g"`
    win1_2=`add_dd $win1_1 $runpctlen`
    win2_2=`echo $date2 | sed "s/-//g"`
    win2_1=`add_dd $win2_2 -${runpctlen}`
  1. cut the windows and merge to a longfile
    cdo seldate,${win1_1},${win1_2} ${ifile}.nc win1.nc
    cdo seldate,${win2_1},${win2_2} ${ifile}.nc win2.nc
    cdo mergetime shifttime,${runpctlen}day win1.nc ${ifile}.nc -shifttime,${runpctlen}days win2.nc ${ifile}ext.nc

then use: seldate,$date1,$dates to cut out the series required for further operations..

probably some of the above steps could be combined... and ${runpctlen} is my window length obviously...

RE: subtracting yearly statistics or running means - Added by Adrian Tompkins over 3 years ago

just came back across this answer and of course rather than messing around with dates, much easier to do a cut in this way

cdo seltimestep,1/$n in start_window.nc
cdo seltimestep,-${n}/-1 in end_window.nc

    (1-2/2)