geopotential height differences between different cdo versions?
Added by Ralf Hand over 2 years ago
Dear all,
I am currently using cdo after to extract several variables from ECHAM raw output and do some basic postprocessing (computing monthly means etc.). I originally was doing the postprocessing of the raw output on our national supercomputing centre, but as we were running out of computing time there and some postpro was still lacking I have now ported my postprocessing script to our local server. The cdo version differs between the supercomputer (cdo 1.9.8) and our local server (cdo 2.0.4).
To verify that everything works correctly after porting my postprocessing script, I redid some of the postprocessing of files that already had been postprocessed at the HPC facility before. I realized that geopotential height shows relatively large differences. At 1000 hPa relative difference between both files is up to almost 100%, with biggest differences in the maritime mid-latitudes and absolute anomalies of about 100m. In the low troposhere (e.g. 850 hPa) I see strong differences between both versions, with lowest difference occuring in regions where we find high topography (probably those grid points where the according pressure level is below the surface?) and an offset of approx. 200m elsewhere. For the upper troposphere the relative difference gets neglectably due to heigher absolute values (usually only about 2% rel. difference at 100 hPa), while the absolute difference is even larger (about 250-300m absolute difference). In the upper level the difference pattern moves more and more towards a spatially smooth offset. I attached some quick-and-dirty-plots of a snapshot of difference patterns for one time step at 1000, 850 and 100 hPa.
In contrast, all other variables that I extracted do exactly agree up to the last digit in both versions. So I was wondering if there has been a change in how geopotential height is computed between version 1.9.8 and 2.0.4.
Cheers,
Ralf
| 1000hPa.gif (55 KB) 1000hPa.gif | geopoth difference at 1000 hPa | ||
| 850hPa.gif (40.8 KB) 850hPa.gif | geopoth difference at 850 hPa | ||
| 100hPa.gif (21.6 KB) 100hPa.gif | geopoth difference at 100 hPa |
Replies (15)
RE: geopotential height differences between different cdo versions? - Added by Ralf Mueller about 2 years ago
hi Ralf!
For further analysis it would be perfect to have the input data. I can rerun commands with all kinds of cdo releases.
So if possible please share the calls and the input data. I have access at CSCS if that help for the data.
cheers
ralf
RE: geopotential height differences between different cdo versions? - Added by Ralf Hand about 2 years ago
Moin Ralf,
thanks for your reply and your help with this. The scripts and the data can be found at
daint:/scratch/snx3000/rhand/experiments/PALAEO-RA/HIPPO_r011i1870p2f12510(/rawdata)
The scripts have evolved over time and unfortunately became a bit messy meanwhile (sorry for that!). For the problem most relevant should be lines 52, 123 and 125 in job2.
I added one of the files that I got as an output when running the postprocessing on my local machine ("HIPPO_r011i1870p2f12510_1870_mon.stdlevs.local.nc")
Cheers,
Ralf
RE: geopotential height differences between different cdo versions? - Added by Ralf Mueller about 2 years ago
I cannot access the original file in the 'output' subdir. can you copy at least on file into /scratch/snx3000/rhand/experiments/PALAEO-RA/HIPPO_r011i1870p2f12510 so that I can check cdo on it?
RE: geopotential height differences between different cdo versions? - Added by Uwe Schulzweida about 2 years ago
The geopotential height was calculated in CDO after and the ECHAM afterburner on half model levels. Unfortunately, a PhD student of Marco Giorgetta has found out that this is not correct. Since CDO version 2.0.3 the geopotential height is calculated on full model levels (see https://code.mpimet.mpg.de/news/518). This essentially results in a shift of half a layer. For more detailed information please contact Marco.
RE: geopotential height differences between different cdo versions? - Added by Ralf Hand about 2 years ago
Dear Ralf and Uwe,
Thanks a lot for your replies! I now repeated the postprocessing locally with an older cdo version (1.9.10) and indeed I can reproduce my results from CSCS with this version :-).
Cheers,
Ralf
RE: geopotential height differences between different cdo versions? - Added by Klaus Pankatz almost 2 years ago
May I reopen this discussion?
Are you sure that a bug has been fixed? My decadal predictions, this year processed with cdo 2.0.6 now show zg500 values of ~5300 m for sellonlatbox,-50,60,20,75 ("Europe").
NCEP and ERA5 reanalyses state values of about 5600 m which is consistent with my MPI-ESM hindcasts processed with cdo 1.9.6.
NCEP Geopotential height:
https://www.researchgate.net/figure/The-annual-mean-500-hPa-geopotential-height-during-the-sub-periods-a-1948-1975-b_fig1_326664707
RE: geopotential height differences between different cdo versions? - Added by Ralf Mueller almost 2 years ago
Moin Klaus!
in order to verify anything we need access to your data, I guess
cheers
ralf
RE: geopotential height differences between different cdo versions? - Added by Klaus Pankatz almost 2 years ago
Here is the set of data I looked at:
decadal start 2021 (cdo 1.9.6)
decadal start 2022 (cdo 2.0.6)
ERA5 reampbil,global_2 to reduce file size
RE: geopotential height differences between different cdo versions? - Added by Ralf Mueller almost 2 years ago
thx Klaus,
but now I need the CDO call to perform the computation.
RE: geopotential height differences between different cdo versions? - Added by Klaus Pankatz almost 2 years ago
Call for model raw to netcdf:
cdo -s -f nc4 -z zip_9 -setmissval,1.e+20 -setreftime,2022-01-01,00:00:00 -invertlat -after /ec
/res4/scratch/dwkp/mpiesm-1.2.01p6/experiments/hckf405s2022_r1i1p1/outdata/echam6/hckf405s2022_r1i1p1_echam6_echam_2028.grb /ec/res4/scratch/dwkp/tmp/cmor/6hr/tmp_zg_6hr_MPI-ESM-LR_hckf405s2022_r1i1p1_2028010100-2028123118.nc"
afterburner nml:
&select
code = 156,
type = 30,
level = 100000,92500,85000,70000,50000,30000,20000,10000,5000,3000,2000,1000,
format = 1,
mean = 0,
interval = 0,
&END
Do you need the model raw output?
RE: geopotential height differences between different cdo versions? - Added by Ralf Mueller almost 2 years ago
hi Klaus!
yes, in order to recompute, I need at least on input file for the cdo call
RE: geopotential height differences between different cdo versions? - Added by Ralf Mueller almost 2 years ago
btw: There was a bugfix for this computation in 2.0.3. (mentioned above). I guess should explain your differences comparing 1.9.6 and 2.0.6
RE: geopotential height differences between different cdo versions? - Added by Klaus Pankatz almost 2 years ago
Here are model raw output and two computations with cdo-1.9.10 and cdo-2.1.1 as available on Atos HPC in Bologna.
| hckf405s2022_r8i1p1_echam6_echam_2027_example.grb (2.15 MB) hckf405s2022_r8i1p1_echam6_echam_2027_example.grb | model raw output | ||
| zg_cdo-1.9.10.nc (868 KB) zg_cdo-1.9.10.nc | cdo 1.9.10 | ||
| zg_cdo-2.1.1.nc (868 KB) zg_cdo-2.1.1.nc | cdo 2.1.1 |
RE: geopotential height differences between different cdo versions? - Added by Christian Stepanek over 1 year ago
Hi all,
I would like to chip in. I just stumbled across this issue when asking myself why a geopoth dataset created with 'cdo after' from ECHAM6 output leads to a quite obvious offset (about 100-300 meters lower for 500 hPa) if I compare values to ERA5 or other model simulations.
When I use cdo version 1.9.10, derived values look plausible if compared to other data sets. When I use cdo version 2.2.0, on the other hand, the values show the kind of offset described above, and are about 300 m lower than if computed with version 1.9.10.
According to what I read above the newer CDO versions should be fixed, while earlier CDO versions produce erroneous values? Yet, values computed with afterburner from earlier CDO version 1.9.10 are more comparable to data from other sources. Sorry if I misunderstood something, but I am quite puzzled now.
All the best and thanks a lot.
Christian
RE: geopotential height differences between different cdo versions? - Added by Uwe Schulzweida over 1 year ago
Hi Christian,
The bug report and the discussion about this problem can be found here: Bug #11346.
Best,
Uwe