Project

General

Profile

Convert grib1 to grib2, to netcdf and calculate geopotential

Added by Joakim Kjellsson almost 6 years ago

Hi

I've got some model output in GRIB format where each file contains a mix of GRIB1 and GRIB2. Some of the data is spectral (u,v,T) and some is reduced Gaussian (q,precip etc).
All files have a mix of vertical coordinates.
Files with spectral data have the prefix ICMSH and files with reduced Gaussian data have the prefix ICMGG.
I'd like to get all data on a regular Gaussian grid, calculate geopotential, put on pressure levels, and convert to netCDF.
What I've done so far is

for vv in SH GG ; do

   grib_set -w editionNumber=1 -s editionNumber=2 ICM${vv}g001_all.grb ICM${vv}g001_all.grb2
   grib_set -w editionNumber=2 -s editionNumber=1 ICM${vv}g001_all.grb ICM${vv}g001_all.grb1

   if [ "$vv" == "SH" ]; then
      cdo -O sp2gpl ICM${vv}g001_all.grb2 ICM${vv}g001_gp_all.grb2
      cdo -O sp2gpl ICM${vv}g001_all.grb1 ICM${vv}g001_gp_all.grb1

   elif [ "$vv" == "GG" ]; then
      cdo -O -setgridtype,regular ICM${vv}g001_all.grb1 ICM${vv}g001_gp_all.grb1
      cdo -O -setgridtype,regular ICM${vv}g001_all.grb2 ICM${vv}g001_gp_all.grb2

   fi   
done

cdo -O merge ICMGGg001_gp_all.grb1 ICMSHg001_gp_all.grb1 ICMg001_gp_all.grb1 
cdo -O merge ICMGGg001_gp_all.grb2 ICMSHg001_gp_all.grb2 ICMg001_gp_all.grb2 

cdo -O geopotheight ICMg001_gp_all.grb1 Z_g001.grb1 
cdo -O geopotheight ICMg001_gp_all.grb2 Z_g001.grb2 

cdo -O merge ICMg001_gp_all.grb1 Z_g001.grb1 ICMg001_all.grb1
cdo -O merge ICMg001_gp_all.grb2 Z_g001.grb2 ICMg001_all.grb2

plevels=100000,92500,85000,75000,60000,50000,40000,30000,20000,10000,5000,2000,1000
cdo -O -f nc ml2pl,${plevels} ICMg001_all.grb1 ICMg001_all_plev_grb1.nc 
cdo -O -f nc ml2pl,${plevels} ICMg001_all.grb2 ICMg001_all_plev_grb2.nc

This gives me two files. The first, ICMg001_all_plev_grb1.nc, has ok values, but all the variable names are gone. The variables are now just named "var129" etc rather than the actual names.
The other file, ICMg001_all_plev_grb2.nc, can't be calculated. The error message is "Error (cdf_put_vara_double) : NetCDF: Numeric conversion not representable". The variable names are ok though.
I think the problem is due to how geopotential height is calculated, which is always negative in Z_g001.grb2.

So it seems like:
1) CDO loses the variable names with GRIB1 data when converting from GRIB2.
2) CDO can't calculate geopotential from GRIB2 data.

Am I doing something wrong?
Does anyone how to get the data in the format I need?

Many thanks
Joakim

PS. I'm enclosing the input data if anyone wants to try and reproduce the results.

ICMGGg001_all.grb (6.52 MB) ICMGGg001_all.grb reduced gaussian data
ICMSHg001_all.grb (13.9 MB) ICMSHg001_all.grb spectral data

Replies (2)

RE: Convert grib1 to grib2, to netcdf and calculate geopotential - Added by Uwe Schulzweida almost 6 years ago

The default installation of CDO uses the CDO internal GRIB1 decoder. That means the GRIB_API is used only for GRIB2. We need this default on our institute because of the decoding performance. Unfortunately the supported for parameter tables is very limited to our purpose.
It might be a good idea to disable the CDO internal GRIB1 decoder. In this case the GRIB_API is also used for GRIB1. That will help a lot to keep the correct variable names.

./configure --disable-cgribex --with-eccodes=... ....

The CDO functions geopotheight and ml2pl were developed for data in GRIB1. GRIB2 should theoriticaly also work but there is a conflict detecting the correct variable for surface_air_pressure with your data. Your data contains lsp and lnps and lnps is available twice. CDO uses lsp in this case but this is wrong.

RE: Convert grib1 to grib2, to netcdf and calculate geopotential - Added by Joakim Kjellsson almost 6 years ago

Hi Uwe!

Many thanks for your reply!
I haven't compiled CDO myself, so I'm not sure exactly what the settings were. But perhaps I can try another install and try to keep GRIB_API decoder and not the internal one. Do you know approximately how bad the performance change is?

I have actually managed to get around the problem anyways (8 hours later on a Sunday afternoon). See enclosed script.
There were two problems with my data:
1) as you said, my file had "lnsp" (log(ps)), "lnsp_2" (log(ps)) and "lsp" (large-scale precip), and CDO thought "lsp" was log(ps), which is totally wrong. Since "lsp" is a surface variable, I now do

cdo selzaxis,hybrid infile.grb hybrid.grb

so that I'm only converting hybrid data to pressure levels.

2) My data contained two LNSP variables, one was log(ps) at the lowest hybrid level, the other one was log(ps) at 1000 hPa. When using grib1, CDO renamed log(ps) at 1000 hPa to LNSP_2, but when using grib2 it got added so that LNSP was available at hybrid levels 1 and 1000 (!) and then CDO did not know what to do.

In the future, I'll make sure my model does not output LNSP at 1000 hPa, as it does not make any sense to have anyway.

Best wishes
Joakim

regrid.sh (6.1 KB) regrid.sh
    (1-2/2)