CDO

Sorry pressed submit too soon there.

The output field from the above command has values up to ~1.e36 - the missing value indicator which suggests that CDO is not correctly interpreting the misising data indicators in the original data.

I have tried this with 2 builds of CDO:

Climate Data Operators version 1.9.8 (https://mpimet.mpg.de/cdo)
System: x86_64-pc-linux-gnu
F77 version : unknown
Features: 5GB 4threads C++11 Fortran DATA PTHREADS OpenMP HDF5 NC4/HDF5/threadsafe OPeNDAP UDUNITS2 PROJ XML2 CURL FFTW3 SSE3
Libraries: HDF5/1.10.2 proj/6.2.1 xml2/2.9.10 curl/7.61.0(h7.71.1)
Filetypes: srv ext ieg grb1 grb2 nc1 nc2 nc4 nc4c nc5
CDI library version : 1.9.8
cgribex library version : 1.9.4
ecCodes library version : 2.16.0
NetCDF library version : 4.6.1 of Sep 8 2018 17:21:01 $
hdf5 library version : 1.10.2 threadsafe
exse library version : 1.4.1
FILE library version : 1.8.3

and

cdo -V

Climate Data Operators version 1.9.9 (https://mpimet.mpg.de/cdo)
System: x86_64-conda-linux-gnu
CDI library version : 1.9.9
cgribex library version : 1.9.5
ecCodes library version : 2.19.1
NetCDF library version : 4.7.4 of Oct 21 2020 19:55:48 $
hdf5 library version : 1.10.6 threadsafe
exse library version : 1.4.1
FILE library version : 1.9.1

the coordinates attribute of your input cannot be understood by CDO. for a regular grid, you can drop it or set it to 'lon lat'

cdo -setattribute,precipitation_rate@coordinates="lat lon" pr_day_CMIP5_bcc-csm1-1_rcp85_1.nc _pr_day_CMIP5_bcc-csm1-1_rcp85_1.nc

if you look into your input with ncdump -h you can see, that even netcdf thinks that there are missing values

....

 precipitation_rate =
  _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, 
    _, _, _, _, _, _, _, _, _,
  _, 16.9277106877416, 3.41135024791583, 5.61931987758726, 0.512061885092407, 
    6.24593193518441e-13, 3.0639591510923e-13, 3.12128040759594e-15, 
    1.0349868895047e-24, 0, 0, 1.12076808174315e-30, 9.92616782395288e-21, 
    1.24077092807401e-22, 0, 1.27054943034778e-19, 1.20702198012964e-18, 0, 
    0, 0, 0, 0, 5.88767189890025e-13, 0.00337165349719726, 
...

So if you set the right missing value as an additional meta-data of the data variable, you can see them with CDO, too:

cdo -setattribute,precipitation_rate@coordinates="lat lon",precipitation_rate@missing_value=9.969209968386869e+36 pr_day_CMIP5_bcc-csm1-1_rcp85_1.nc _pr_day_CMIP5_bcc-csm1-1_rcp85_1.nc

cdo infov _pr_day_CMIP5_bcc-csm1-1_rcp85_1.nc
    -1 :       Date     Time   Level Gridsize    Miss :     Minimum        Mean     Maximum : Parameter name
     1 : 1980-12-01 12:00:00       0      476      80 :      0.0000      1.1071      32.855 : precipitation_rate

Without that meta-data give, those huge values are taken into account during the inpolation - hence the huge values after it.

In case you mark them properly before the interpolation, the range of values is normal:

cdo remapbil,pr_day_OBS-ERA5_ERA5_obs_1.nc _pr_day_CMIP5_bcc-csm1-1_rcp85_1.nc pr_day_CMIP5_bcc-csm1-1_rcp85_bil.nc
Warning (cdfInqContents): Coordinates variable season_year can't be assigned!
cdo    remapbil: Bilinear weights from lonlat (34x14) to lonlat (369x141) grid, with source mask (396)
cdo    remapbil: Processed 476 values from 1 variable over 1 timestep [0.04s 57MB].

$ cdo infov pr_day_CMIP5_bcc-csm1-1_rcp85_bil.nc
    -1 :       Date     Time   Level Gridsize    Miss :     Minimum        Mean     Maximum : Parameter name
     1 : 1980-12-01 12:00:00       0    52029    8109 :      0.0000     0.91855      31.983 : precipitation_rate
cdo    infon: Processed 52029 values from 1 variable over 1 timestep [0.01s 45MB].

hth
ra;lf

thank you very much for that really fast and helpful response. I am now able to move on with my own work.

I did try and set the missing value as I thought it was connected to that and I tried to use setmissval - I thought that didn't work but it does.

Is there a reason that cdo doesn't recognise the default netCDF fill value? The documentation states:

"It is not necessary to define your own _FillValue attribute for a variable if the default fill value for the type of the variable is adequate."

https://www.unidata.ucar.edu/software/netcdf/docs/attribute_conventions.html

Nicholas Savage wrote:

thank you very much for that really fast and helpful response. I am now able to move on with my own work.

I did try and set the missing value as I thought it was connected to that and I tried to use setmissval - I thought that didn't work but it does.

Is there a reason that cdo doesn't recognise the default netCDF fill value? The documentation states:

"It is not necessary to define your own _FillValue attribute for a variable if the default fill value for the type of the variable is adequate."

https://www.unidata.ucar.edu/software/netcdf/docs/attribute_conventions.html

This is the docu of the netcdf library. CDO is not a general netcdf editing tool like the NCOs. Instead it requires netcdf files to follow the CF-convention. In there the missing_value/_FillValue has to be used to indicate the exact value. And if you read a bit further in this documentation, there is "However, use of the default fill value for data type byte is not recommended".

CDO does not rely on any defaults because they are part of the library. So in case you read a file, which was written with a netcdf lirbary that had a different numerical fill value than the library you read it with, you have a problem. In order to have self-describing files, a real numerical value is needed.

cheers

thank you for such a clear explanation of the CDO philosophy