Different result with Python bindings and ensemble data

Hi, I'm trying to use the CDO Python bindings to do select a grid point from a large dataset and re-use directly the result in xarray.

I have a dataset with 50 ensemble members and (time, latitude, longitude) dimensions.

When using cdo directly into the shell I get the right result

cdo -P 16 remapnn,lon=10/lat=53.55 t_850.grib2 test.grib2

which is a new dataset with the dimensions of 50 ensemble members and time.

However, if I try to do the same in the python bindings

cdo.remapnn('lon=10/lat=53.55', input='/home/ekman/ssd/guido/ecmwf-ens/t_850.grib2', options = '-P 16', returnXDataset=True)

I get a dataset where the ensemble member dimension has disappear (see the screenshot). I believe the problem is coming from the fact that in all ensemble members the name of the variable is the same...maybe?

Screen Shot 2022-09-01 at 14.25.56.png (155 KB) Screen Shot 2022-09-01 at 14.25.56.png

Replies (3)

RE: Different result with Python bindings and ensemble data - Added by Karin Meier-Fleischer over 2 years ago

Hi Guido,

I guess that xarray causes the problem but you can try the following (I used a small file containing 5 ensemble members from ECMWF):

tmpfile = cdo.remapnn('lon=10/lat=53.55', input=infile)

ds = xr.open_dataset(tmpfile, engine='cfgrib')

The result dataset ds will be something like:

<xarray.Dataset>
Dimensions:            (number: 5, step: 5, latitude: 1, longitude: 1)
Coordinates:
  * number             (number) int64 1 2 3 4 5
    time               datetime64[ns] ...
  * step               (step) timedelta64[ns] 0 days 00:00:00 ... 2 days 00:0...
    meanSea            float64 ...
  * latitude           (latitude) float64 51.0
  * longitude          (longitude) float64 90.0
    valid_time         (step) datetime64[ns] ...
    heightAboveGround  float64 ...
Data variables:
    msl                (number, step, latitude, longitude) float32 ...
    u10                (number, step, latitude, longitude) float32 ...
    v10                (number, step, latitude, longitude) float32 ...
Attributes:
    GRIB_edition:            2
    GRIB_centre:             ecmf
    GRIB_centreDescription:  European Centre for Medium-Range Weather Forecasts
    GRIB_subCentre:          0
    Conventions:             CF-1.7
    institution:             European Centre for Medium-Range Weather Forecasts
    history:                 2022-09-01T15:32 GRIB to CDM+CF via cfgrib-0.9.1...

RE: Different result with Python bindings and ensemble data - Added by Guido Cioni over 2 years ago

yeah, that's the workaround that I'm using right now...
It would be cleaner to avoid the second step anyway

RE: Different result with Python bindings and ensemble data - Added by Ralf Mueller over 2 years ago

hi Guido!

You also might check what the python bindings to under the hood with setting

cdo.debug = True

and rerunning the call. It will show the CDO calls executed internally so that you can compare it with the one your do manually on the command line.

hth
ralf

(1-3/3)

Project

General

Profile

CDO