Problem with ICON-EU EPS conversion

Added by Stephane Iri almost 5 years ago

Hello,

I worked fine with ICON/ICON EPS. Now, i use also ICON-EU EPS from https://opendata.dwd.de/weather/nwp/icon-eu-eps/grib/

To convert ICON-EU EPS grid to regualr lat/lon grid, i use this :

cdo -f grb2  remap,"$TARGET_GRID_DESCRIPTION,$WEIGHTS_FILE" "$INFILE" "$OUTFILE"

where :
TARGET_GRID_DESCRIPTION is the text description file,
WEIGHTS_FILE is the .nc file (see below)

TARGET_GRID_DESCRIPTION looks like this (e.g. target_grid_europe_eu_0125.txt) :

gridtype = lonlat
xsize    = 312
ysize    = 328
xfirst   = -23.5
xinc     = 0.125
yfirst   = 29.5
yinc     = 0.125
# xend = 62.5
# yend = 70.5
# weights_icogl2europe_eu_0125.nc

and i build once the WEIGHTS_FILE file as it :

cdo gennn,${TARGET_GRID_DESCRIPTION} ${ICON_GRID_FILE} ${WEIGHTS_FILE}

where ICON_GRID_FILE is icon_grid_0026_R03B07_G.nc (from https://opendata.dwd.de/weather/lib/cdo/)

So, with ICON and ICON EPS, this method works correctly.
The last command seems works too with ICON-EU EPS :

$ cdo gennn,target_grid_europe_eu_0125.txt icon_grid_0026_R03B07_G.nc weights_icogl2europe_eu_0125.nc
cdo gennn: Nearest neighbor weights from unstructured (2949120) to lonlat (312x328) grid
cdo gennn: Processed 2949120 values from 6 variables over 1 timestep [7.11s 1578MB]

But, with ICON-EU EPS, the final conversion script returns this error :

$ cdo -f grb2  remap,./icon/target_grid_europe_eu_0125.txt,./icon/weights_icogl2europe_eu_0125.nc ./in/icon/20200506/00/ens_eu_ens_z500_0 ./in/icon/20200506/00/ens_eu_ens_z500_0.grib2

cdo remap (Abort): Size of source grid and weights from ./icon/weights_icogl2europe_eu_0125.nc differ!

(files provided from https://opendata.dwd.de/weather/nwp/icon-eu-eps/grib/00/fi/)

From https://isabel.dwd.de/DWD/forschung/nwv/fepub/icon_database_main.pdf (page 13), the mesh grid is defined to R3B08.
I guess that the icon_grid_0026_R03B07_G.nc file is incorrect for ICON-EU EPS. Right ? But no file found.
How can I do my conversions ?
Did I make a mistake ?

Any help is appreciated, thanks
Stephane

Replies (3)

RE: Problem with ICON-EU EPS conversion - Added by Ralf Mueller almost 5 years ago

hi Stephane!

I don't know all these shortcuts, File format : GRIB2 -1 : Institut Source T Steptype Levels Num 1 : DWD unknown v instant 2 : DWD unknown v instant 3 : DWD unknown v instant 4 : DWD unknown v instant 5 : DWD unknown v instant 6 : DWD unknown v instant 7 : DWD unknown v instant 8 : DWD unknown v instant 9 : DWD unknown v instant 10 : DWD unknown v instant 11 : DWD unknown v instant 12 : DWD unknown v instant 13 : DWD unknown v instant 14 : DWD unknown v instant 15 : DWD unknown v instant 16 : DWD unknown v instant 17 : DWD unknown v instant 18 : DWD unknown v instant 19 : DWD unknown v instant 20 : DWD unknown v instant 21 : DWD unknown v instant 22 : DWD unknown v instant 23 : DWD unknown v instant 24 : DWD unknown v instant 25 : DWD unknown v instant 26 : DWD unknown v instant 27 : DWD unknown v instant 28 : DWD unknown v instant 29 : DWD unknown v instant 30 : DWD unknown v instant 31 : DWD unknown v instant 32 : DWD unknown v instant 33 : DWD unknown v instant 34 : DWD unknown v instant 35 : DWD unknown v instant 36 : DWD unknown v instant 37 : DWD unknown v instant 38 : DWD unknown v instant 39 : DWD unknown v instant 40 : DWD unknown v instant Grid coordinates : 1 : unstructured Vertical coordinates : 1 : height Time coordinate : 1 step RefTime = 2020-05-06 12:00:00 YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss 2020-05-07 06:00:00
Here you can check but let's follow the error message from CDO and check the grid of the input file. I downloaded https://opendata.dwd.de/weather/nwp/icon-eu-eps/grib/12/u_10m/icon-eu-eps_europe_icosahedral_single-level_2020050612_017_u_10m.grib2.bz2 and check the grid with

cdo sinfov `lf` Points Num Dtype : Parameter name 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u 1 1 75948 1 P16 : 10u : points=75948 grid : number=28 position=1 uuid : 982dcc6e-fe2e-11e4-9128-0b03674e713a : levels=1 height : 10 m Units = minutes Calendar = proleptic_gregorian YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss the uuid AND the number of the grid: 28. I think you used grib number 26 instead. I don't have your input files, so I cannot reproduce your workflow. So please check the input with respect to that.

hth
ralf

RE: Problem with ICON-EU EPS conversion - Added by Stephane Iri almost 5 years ago

Hi Ralf!

I'm sorry if my variable's description was not clear.

I follow your tips :

$ cdo sinfov icon-eu-eps_europe_icosahedral_pressure-level_2020050700_000_500_fi.grib2 
   File format : GRIB2
    -1 : Institut Source   T Steptype Levels Num    Points Num Dtype : Parameter name
     1 : DWD      unknown  v instant       1   1     75948   1  P16  : z             
   Grid coordinates :
     1 : unstructured             : points=75948
                             grid : number=28  position=1
                             uuid : 982dcc6e-fe2e-11e4-9128-0b03674e713a
   Vertical coordinates :
     1 : pressure                 : levels=1
                             plev : 50000 Pa
   Time coordinate :  unlimited steps
     RefTime =  2020-05-07 00:00:00  Units = minutes  Calendar = proleptic_gregorian
  YYYY-MM-DD hh:mm:ss  YYYY-MM-DD hh:mm:ss  YYYY-MM-DD hh:mm:ss  YYYY-MM-DD hh:mm:ss
  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00
  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00
  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00
  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00
  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00
  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00
  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00
  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00
  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00
  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00
cdo sinfon: Processed 1 variable over 40 timesteps [0.17s 66MB]

uuid (982dcc6e-fe2e-11e4-9128-0b03674e713a) and grid number (28) are checked. Then, i download icon_grid_0028_R02B07_N02.nc.bz2 from https://opendata.dwd.de/weather/lib/cdo/ instead of icon_grid_0026_R03B07_G.nc.bz2

$ ncdump icon_grid_0026_R03B07_G.nc
//...
global attributes:
        :title = "ICON grid description" ;
        :history = "/panfs/e/vol2/gzaengl/icon-dev/build/x86_64-unknown-linux-gnu/bin/grid_command" ;
        :institution = "Max Planck Institute for Meteorology/Deutscher Wetterdienst" ;
        :source = "icon-dev\b" ;
        :uuidOfHGrid = "982dcc6e-fe2e-11e4-9128-0b03674e713a" ;   // <---
        :number_of_grid_used = 28 ;                               // <---
        :ICON_grid_file_uri = "http://icon-downloads.mpimet.mpg.de/grids/public/icon_grid_0028_R02B07_N02.nc" ;
        :centre = 78 ;
        :subcentre = 255 ;
        :outname_style = 2 ;
        :grid_mapping_name = "lat_long_on_sphere" ;
        :crs_id = "urn:ogc:def:cs:EPSG:6.0:6422" ;
        :crs_name = "Spherical 2D Coordinate System" ;
        :ellipsoid_name = "Sphere" ;
        :semi_major_axis = 6371229. ;
        :inverse_flattening = 0. ;
        :grid_level = 7 ;
        :grid_root = 2 ;
        :grid_ID = 2 ;
        :parent_grid_ID = 1 ;
        :max_childdom = 1 ;
//...

I continue my script, the first trys are corrects (no errors are returned). I can now open and visualize converted GRIB in Metview (ECMWF). Thanks !

Two little questions :
1- icon_grid_0028_R02B07_N02.nc are the good grid number and uuid but not same reference grib (R02B07 instead of R03B08 for ICON-EU).
Can it compromise the quality of the output ?

2- The grib contains the 40 members of the ensemble but the conversion gets 40 time steps, not 40 members. I guess that each field is sorted (first time step = first member, 2nd time step is second member, ...) but i'm not sure.

$ cdo sinfov ens_eu_ens_z500_0_0125.grib2 
   File format : GRIB2
    -1 : Institut Source   T Steptype Levels Num    Points Num Dtype : Parameter name
     1 : DWD      unknown  v instant       1   1    102336   1  P16  : z             
   Grid coordinates :
     1 : lonlat                   : points=102336 (312x328)
                              lon : 336.5 to 375.375 by 0.125 degrees_east
                              lat : 29.5 to 70.375 by 0.125 degrees_north
   Vertical coordinates :
     1 : pressure                 : levels=1
                             plev : 50000 Pa
   Time coordinate :  unlimited steps
     RefTime =  2020-05-07 00:00:00  Units = minutes  Calendar = proleptic_gregorian
  YYYY-MM-DD hh:mm:ss  YYYY-MM-DD hh:mm:ss  YYYY-MM-DD hh:mm:ss  YYYY-MM-DD hh:mm:ss
  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00
  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00
  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00
  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00
  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00
  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00
  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00
  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00
  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00
  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00  2020-05-07 00:00:00
cdo sinfon: Processed 1 variable over 40 timesteps [0.18s 66MB]

Is there an option missing from my cdo command ?

cdo gennn,target_grid_europe_eu_0125.txt icon_grid_0026_R03B07_G.nc weights_icogl2europe_eu_0125.nc

or it's about other things and I should contact the DWD ?

Anyways, you're doing a great job !
Many thanks for your help
Stephane

RE: Problem with ICON-EU EPS conversion - Added by Ralf Mueller almost 5 years ago

Stephane Iri wrote:

Hi Ralf!

I'm sorry if my variable's description was not clear.

I follow your tips :

[...]

uuid (982dcc6e-fe2e-11e4-9128-0b03674e713a) and grid number (28) are checked. Then, i download icon_grid_0028_R02B07_N02.nc.bz2 from https://opendata.dwd.de/weather/lib/cdo/ instead of icon_grid_0026_R03B07_G.nc.bz2

[...]

I continue my script, the first trys are corrects (no errors are returned). I can now open and visualize converted GRIB in Metview (ECMWF). Thanks !

Two little questions :
1- icon_grid_0028_R02B07_N02.nc are the good grid number and uuid but not same reference grib (R02B07 instead of R03B08 for ICON-EU).
Can it compromise the quality of the output ?

you should ask DWD because they provide the data, they should know the details

2- The grib contains the 40 members of the ensemble but the conversion gets 40 time steps, not 40 members. I guess that each field is sorted (first time step = first member, 2nd time step is second member, ...) but i'm not sure.

again: DWD should be able to comment on this since it's their data

[...]

Is there an option missing from my cdo command ?

cdo gennn,target_grid_europe_eu_0125.txt icon_grid_0026_R03B07_G.nc myWeights.nc
cdo    gennn: Nearest neighbor weights from unstructured (2949120) to lonlat (312x328) grid
cdo    gennn: Processed 6 variables over 1 timestep [1.67s 958MB].

worked without problems. if you want to use the weights multiple times the combination of gennn and remap is fine. in case you only need them once, you could simple call remapnn instead.

[...]

or it's about other things and I should contact the DWD ?

like I said: your question are very data-specific. so it's hard to comment on them.

Anyways, you're doing a great job !

thx

Many thanks for your help
Stephane

(1-3/3)

Project

General

Profile

CDO