CDO

hi!

I have to admit I don;t fully understand. can you upload your input and provide some commands to test with?

cheers
ra;lf

Hi Ralf! of course! i will try to explain step by step my procedure:
I have separate hourly files from temperature (I need the max temperature from 20201112):
20201112_00:00_temp.grb, 20201112_01:00_temp.grb, ... 20201113_00:00_temp.grb, 20201113_01:00_temp.grb, ..., + N hourly files (5 days) from GFS.

step 1: cat the 24 files to merge them in one file from 20201112 01:00 to 20201113 00:00. (upload this input file to don't enlarge too much the upload with all the files. But if you need all of them, please tell me).
command: cat 20201112_01:00_temp.grb 20201112_02:00_temp.grb 20201112_03:00_temp.grb ... 20201113_00:00_temp.grb > cat12_temp.grb
cat12_temp.grb contains temperature from 01:00 from today until 00:00 from tomorrow.

step 2: shift less 1 hour to get the timesteps from 00:00 to 23:00.
if I don't do that, the next step (DAYMAX) gives me two timesteps. One at 20201112 12:00 (with the max temperature from 01:00 to 23:00) and the second at 20201113 00:00 (with the max temperature from only one hour from 2020113 00:00). Should be only one timestepfrom the whole day 20201112.
command: cdo -shifttime,-1hour cat12_temp.grb cat12_shift1h_temp.grb
Result: gives me a file with temperature from 00:00 (not from 20201112 01:00) to 23:00 (not from 20201113 00:00)

step 3: daymax to get the max temperature from the whole day.
command: cdo -daymax cat12_shift1h_temp.grb daymax12_temp.grb
Result: gives me a grib with max temperature from 20201112 but the one timestep is placed at 11:30 (I need to be placed at 20201113 00:00, not at 20201112 11:30).

step 4: shift adding 750 minutes (12 hours and a half from 11:30 to 00:00), to place the timestep at 20201113 00:00
commad: cdo -shifttime,750min daymax12_temp.grb daymax13_00_temp.grb
Result: a file with the max temperature from the whole day where the timestep is placed at 20201113 00:00. Undesrtanding that the max temperature from the day 20201112 should be read as the file tha contains the temperature at 20201113 00:00 because is the whole period from the 20201112.

Sorry, the next two files

Sorry again, and the file with daymax without the step 2 (without shifting -1 hour) that gives me two timesteps, not one with the max temperature.

Thank you for your attention!

Dmp Support wrote:

Hi Ralf! of course! i will try to explain step by step my procedure:
I have separate hourly files from temperature (I need the max temperature from 20201112):
20201112_00:00_temp.grb, 20201112_01:00_temp.grb, ... 20201113_00:00_temp.grb, 20201113_01:00_temp.grb, ..., + N hourly files (5 days) from GFS.

step 1: cat the 24 files to merge them in one file from 20201112 01:00 to 20201113 00:00. (upload this input file to don't enlarge too much the upload with all the files. But if you need all of them, please tell me).
command: cat 20201112_01:00_temp.grb 20201112_02:00_temp.grb 20201112_03:00_temp.grb ... 20201113_00:00_temp.grb > cat12_temp.grb
cat12_temp.grb contains temperature from 01:00 from today until 00:00 from tomorrow.

if you want the maximum temperature of a day, why do you include 0201113_00:00_temp.grb? I does not belong to the day you are interested in. IMO there is no such thing as 24:00.

step 2: shift less 1 hour to get the timesteps from 00:00 to 23:00.
if I don't do that, the next step (DAYMAX) gives me two timesteps. One at 20201112 12:00 (with the max temperature from 01:00 to 23:00) and the second at 20201113 00:00 (with the max temperature from only one hour from 2020113 00:00). Should be only one timestepfrom the whole day 20201112.

you get a second daymax because you included a field from 20201113. For CDO this a different day than 20201112 for obvious reasons.

command: cdo -shifttime,-1hour cat12_temp.grb cat12_shift1h_temp.grb
Result: gives me a file with temperature from 00:00 (not from 20201112 01:00) to 23:00 (not from 20201113 00:00)

step 3: daymax to get the max temperature from the whole day.
command: cdo -daymax cat12_shift1h_temp.grb daymax12_temp.grb
Result: gives me a grib with max temperature from 20201112 but the one timestep is placed at 11:30 (I need to be placed at 20201113 00:00, not at 20201112 11:30).

by default the timestamps are set to be in the middle of the time period you computed the average about. But you cat set the behavior with environment variables

step 4: shift adding 750 minutes (12 hours and a half from 11:30 to 00:00), to place the timestep at 20201113 00:00

I think you can use 12 hours because for a daymax value the time is irrelevant

commad: cdo -shifttime,750min daymax12_temp.grb daymax13_00_temp.grb
Result: a file with the max temperature from the whole day where the timestep is placed at 20201113 00:00. Undesrtanding that the max temperature from the day 20201112 should be read as the file tha contains the temperature at 20201113 00:00 because is the whole period from the 20201112.

IMO setting the timestamp for something related to 20201112 (its daymax value) with the value 20201113 is completely misleading. No one will understand that right away, hence you will always have to tell people something that is not part of the data file. If you forget about it or they forget about it, I guarantee you there will get hardly traceable errors.

Some final thoughts: You put a lot of effort in getting the 00:00 hour timestep into the day before. I think this is the main reason why you have all these issues. So I think from your point of view the data file 20201113_00:00_temp.grb has the wrong timestep, it should rather be 20201112_24:00_temp.grb. if that's the case then why do you finally want to label the daymax of 20201112 with 20201113. you keep the same error that you wanted to fix before with all the shifttime in your own daymax output. That cannot be a good idea - it will create new problem in the future.

you might have good reason to believe the 20201113 00:00 data set belongs to 20201112 - I don't question this, but at the moment I don't understand it ether.

Regarding your processing steps: I think you can do most of the stuff with a single line like this

cdo -shifttime,12hours -daymax -shifttime,-1hour -cat '20201*grb'

hth
ralf

Hi Ralf, thank you for your explanations.
I think I misunderstood something.
Re. including next day at 00 to calculate daymax from current day it's because I thought it was the same as calculating daily rain fall.

I mean, when calculating daily accumulated precipitation is needed to include the next day at 00 because it contains the fallen rain during the previous hour. And not including the first hour from the current day. Is that right?

So I thought should be the same with the other parameters.
So with what you explained, I noticed is not a right thought. Isn't it?

Also thank you for your compressed command to minimise machine efforts and time processing.
Thank you very much!

Dmp Support wrote:

Hi Ralf, thank you for your explanations.
I think I misunderstood something.
Re. including next day at 00 to calculate daymax from current day it's because I thought it was the same as calculating daily rain fall.

thx for clarification

I mean, when calculating daily accumulated precipitation is needed to include the next day at 00 because it contains the fallen rain during the previous hour. And not including the first hour from the current day. Is that right?

This depends on the method you used for calculating an labeling the 00 timestep. It could be that it's exactly like you said: accumulated rain from 23:00 to 00:00 is labeled with 00:00 (IMO this is misleading - again ;-)). But I just cannot say how the input data was created. It could also be the case that the first timestep of the accumulation period was used for labeling the data set.

So I thought should be the same with the other parameters.
So with what you explained, I noticed is not a right thought. Isn't it?

It depends (as usual) - you have to make sure how the input files are created. If the 00:00 timestep actually contains data from 23:00 to 00:00 then your processing is correct. (Although I would label the final data with the day from 23:00 and not with the one from 00:00, but I start to repeat myself ...)
In case the 00:00 timestep covers 00:00 - 01:00 you can leave it out for daymax precip.

Hi Ralf!
You are right! First I should know how the input files really are created. Then I will apply your comments.
Thank you very much!