ECCODES ERROR : Range of values too large...
Added by Lina Neunhäuserer almost 4 years ago
Hi,
I am remapping 3D ICON-EU output to a lon-lat grid MYGRID with the following specification:
gridtype = lonlat
xsize = 86
ysize = 124
xfirst = -10.75
xinc = 0.5
yfirst = 34.625
yinc = 0.25
I am using the following commands:
! generate remap weights for ICON fields
cdo genbil,${MYGRID} ${PATH}/iefr_${year}${month}${day}00_tmp ./remapweights_rcgGrid_LM
! interpolate ICON variables from ICON grid to my grid
! (horizontally for every level)
! (the example shows the command for the variable "qi" which contains the Cloud Ice Mixing Ratio)
cdo settime,${hour}:00:00 setdate,${year}${month}-${day} -remap,${MYGRID},./remapweights_rcgGrid_LM \
${PATH}/iefr_${year}${month}${day}${hour}_qi ${PATH_GRIB}/rcg2dGrid${year}${month}${day}${hour}_qi
Usually this works fine without any problems. But for a specific date and time (2019060915) and only for the variable qi, I get the following error:
cdo settime: Processed 639840 values from 1 variable [0.71s 114MB]
cdo(2) setdate: Process started
cdo(3) remap: Process started
ECCODES ERROR : Range of values too large. Try a smaller value for decimal precision (less than 13)
ECCODES ERROR : unable to compute packing parameters
ECCODES ERROR : unable to set double array codedValues (Encoding invalid)
stream_gribapi.c at line 3237: grib_set_double_array(gh, "values", data, datasize) failed: Encoding invalid/home/Projekte/myshell.ksh: line 262: 17597: Memory fault
Note that I dont get this error in case I remap the same ICON data to another grid with a finer resolution (0.125x0.0625).
The cdo version I am using here is (cdo -V):
Climate Data Operators version 1.9.6 (http://mpimet.mpg.de/cdo)
System: x86_64-pc-linux-gnu
Features: 31GB 16threads C++14 Fortran DATA PTHREADS OpenMP45 HDF5 NC4/HDF5/threadsafe OPeNDAP SZ UDUNITS2 PROJ.4 MAGICS CURL FFTW3 SSE2
Libraries: HDF5/1.10.4 proj/5.2 curl/7.64.0(h7.63.0)
Filetypes: srv ext ieg grb1 grb2 nc1 nc2 nc4 nc4c nc5
CDI library version : 1.9.6
ecCodes library version : 2.12.0
NetCDF library version : 4.6.2 of Nov 20 2018 06:04:35 $
hdf5 library version : library undefined
exse library version : 1.4.1
FILE library version : 1.8.3
I would be very happy if you could give me a hint what the problem could be...
Best regards, Lina
Replies (4)
RE: ECCODES ERROR : Range of values too large... - Added by Ralf Mueller almost 4 years ago
hi Lina!
you might try another packing, with the -P24 or -P16. I am not sure to be honest. if you provide the paths to all files, I can try myself. do you want grib output?
RE: ECCODES ERROR : Range of values too large... - Added by Lina Neunhäuserer almost 4 years ago
Hi Ralf,
here is the qi file. You can use it for the genbil command as well (instead of _tmp). Yes, I would need GRIB output.
| iefr_2019060915_qi (9.45 MB) iefr_2019060915_qi |
RE: ECCODES ERROR : Range of values too large... - Added by Uwe Schulzweida almost 4 years ago
This seems to be more a problem of ecCodes. I could reproduce the problem with cdo/ecCodes-2.6.0. With cdo/ecCodes-2.17.0 it works.
Cheers,
Uwe
RE: ECCODES ERROR : Range of values too large... - Added by Lina Neunhäuserer almost 4 years ago
Hi Uwe,
thank you for the advice. Unfortunately, we are bound to ecCodes library 2.12.0.
I would be very interested about the problem linked to the qi file I sent to you. It does contain - for whatever reason - very small numbers in the order of 1e-29. This might be a problem. On the other hand, I have qi files for another date and time that contain the same small numbers but work.
Nevertheless, I tried the following:
cdo setmisstoc,100 iefr_2019060915_qi tmp0 ! set original missing vals in qi to 100
cdo nec,100 tmp0 mask0 ! create a mask for the original missing vals
cdo abs iefr_2019060915_qi tmp1 ! create a copy of qi file with absolute values (just to make sure)
cdo gtc,1.0e-25 tmp1 mask1 ! create a mask for the very small values
cdo ifthen mask1 iefr_2019060915_qi tmp2 ! create a file where all very small values are replaced by missing vals
cdo setmisstoc,0 tmp2 tmp3 ! set these missing vals to zero
cdo ifthen mask0 tmp3 iefr_2019060915_qi_new ! restore the original missing vals in their original place
The result of this procedure (qi_new) can be remapped without problems. However, the procedure works even if I set the cutoff value as small as 1.0e-30 which is smaller as the smallest values in the original qi. So the procedure should not change anything. But it obviously does, as the resulting qi_new is larger than the original qi, contains the very small values in the order of 1.0e-29, and can be remapped without problems as well.