CDO sub Error When Trying to Average Out Seasonality of Data
Added by Mikkel Bugge almost 4 years ago
I've been using CDO quite a lot for working with netcdf timeseries, but I've run into a problem. I saw this stackoverflow issue which is exactly what I need:
https://stackoverflow.com/questions/62168154/how-to-remove-seasonality-from-time-series-data
I used the second command for my monthly data,
cdo sub in.nc -ymonmean in.nc deseasonalized.nc
However when I try to run the second command, I get the following error:
cdo sub (Abort): (pipe1.4) unexpected tsID 0 13 11
from the "tsID" I suspected it could be because I have a timestamp into the year of 2015 which gives me a little over ten years of data, resulting in the ymonmean output file's timestamps to be all 2014 except for January which was in 2015. Fixing this didn't do much, however.
I uploaded the file I am trying to do this with, it is quite compressed, taking up 200mb when extracted.
| sst_monthly.rar (30.9 MB) sst_monthly.rar |
Replies (7)
RE: CDO sub Error When Trying to Average Out Seasonality of Data - Added by Ralf Mueller almost 4 years ago
hi Mikkel!
Please upload the complete script/list of commands you are using. A simple monthly file does not tell me enough to reproduce the error.
If the input file is too large, you can select a single cell from it since the problem/analysis is complete along the time axis:
cdo -selgridcell,1 ....
cheers
ralf
RE: CDO sub Error When Trying to Average Out Seasonality of Data - Added by Mikkel Bugge almost 4 years ago
Hi Ralf. Oh yea sorry, will do:
cdo sub sst_monthly.nc -ymonmean sst_monthly.nc sst_deseason_monthly.nc
The previously included file should contain a timeseries of 121 timesteps.
If not, I included my working data. It should be identical to the one previously uploaded except without the 2015-data.
But - in fact - I found that separating the commands worked for me!
cdo ymonmean sst_monthly.nc sst_ymonmean.nc
cdo sub sst_monthly.nc sst_ymonmean.nc sst_deseason_monthly.nc
I would think these two commands are identical to the piping done above, but I'm not very keen on these operations as of yet.
Also, I use CDO version 1.9.9rc1, if that could be a potential issue.
| sst_monthly2.rar (30.6 MB) sst_monthly2.rar | Same as sst_monthly.rar, but without 2015 data. |
RE: CDO sub Error When Trying to Average Out Seasonality of Data - Added by Mikkel Bugge almost 4 years ago
Mikkel Bugge wrote:
Hi Ralf. Oh yea sorry, will do:
[...]
The previously included file should contain a timeseries of 121 timesteps.
If not, I included my working data. It should be identical to the one previously uploaded except without the 2015-data.But - in fact - I found that separating the commands worked for me!
[...]
I would think these two commands are identical to the piping done above, but I'm not very keen on these operations as of yet.Also, I use CDO version 1.9.9rc1, if that could be a potential issue.
I forgot to add - this is a hdf5 netcdf file. At least in Python that complicates things a little. Perhaps that is why you can't see more than a single month in the data file?
RE: CDO sub Error When Trying to Average Out Seasonality of Data - Added by Ralf Mueller almost 4 years ago
ok, thx.
For clarity I would always use ymonsub instead of sub. This represents exactly what you want to compute and works out of the box (on my machine with 1.9.10)
cdo -L -ymonsub sst_monthly.nc -ymonmean sst_monthly.nc sst_deseason_monthly.nc
In addition it does not rely on a specific default behavior of sub, i.e. how it behaves when the two input streams have in-fact different number of timesteps.
Please give it a try on with your version of CDO. I think it should work,too.
hth
ralf
RE: CDO sub Error When Trying to Average Out Seasonality of Data - Added by Ralf Mueller almost 4 years ago
here is my result file, please check it for correctness
RE: CDO sub Error When Trying to Average Out Seasonality of Data - Added by Mikkel Bugge almost 4 years ago
.. And it certainly did. I'll stick to ymonsub. Thanks, Ralf.
RE: CDO sub Error When Trying to Average Out Seasonality of Data - Added by Mikkel Bugge almost 4 years ago
By the way, your result file is different from mine, showing timesteps only within 2005. Funnily enough I still get what I want using your ymonsub line:
I did exactly:
cdo -L -ymonsub sst_monthly.nc -ymonmean sst_monthly.nc sst_deseason_monthly.nc
But my resulting data seems correct; I get my data centered approximately at 0.0 which for the various timesteps.