HDF error messages dependent on use of cdo version and apply keyword

Hi, I am getting some error messages relating to HDF errors depending on my use of cdo version and the apply keyword.

For this version of CDO,

cdo --version
Climate Data Operators version 2.1.1 (https://mpimet.mpg.de/cdo)
System: x86_64-conda-linux-gnu
CXX Compiler: /home/conda/feedstock_root/build_artifacts/cdo_1677416397422/_build_env/bin/x86_64-conda-linux-gnu-c++ -fPIC -DPIC -g -O2 -fopenmp -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/project/17001770/climate_branch/cmp/chuaxr/envs/v3_ds_surge_snakemake/4f04623e0afd9f9ef82e97d0859c249f_/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/cdo_1677416397422/work=/usr/local/src/conda/cdo-2.1.1 -fdebug-prefix-map=/home/project/17001770/climate_branch/cmp/chuaxr/envs/v3_ds_surge_snakemake/4f04623e0afd9f9ef82e97d0859c249f_=/usr/local/src/conda-prefix -fopenmp -pthread
CXX version : unknown
C Compiler: /home/conda/feedstock_root/build_artifacts/cdo_1677416397422/_build_env/bin/x86_64-conda-linux-gnu-cc -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/project/17001770/climate_branch/cmp/chuaxr/envs/v3_ds_surge_snakemake/4f04623e0afd9f9ef82e97d0859c249f_/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/cdo_1677416397422/work=/usr/local/src/conda/cdo-2.1.1 -fdebug-prefix-map=/home/project/17001770/climate_branch/cmp/chuaxr/envs/v3_ds_surge_snakemake/4f04623e0afd9f9ef82e97d0859c249f_=/usr/local/src/conda-prefix -fopenmp -pthread -pthread
C version : unknown
F77 Compiler: /home/conda/feedstock_root/build_artifacts/cdo_1677416397422/_build_env/bin/x86_64-conda-linux-gnu-gfortran -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/project/17001770/climate_branch/cmp/chuaxr/envs/v3_ds_surge_snakemake/4f04623e0afd9f9ef82e97d0859c249f_/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/cdo_1677416397422/work=/usr/local/src/conda/cdo-2.1.1 -fdebug-prefix-map=/home/project/17001770/climate_branch/cmp/chuaxr/envs/v3_ds_surge_snakemake/4f04623e0afd9f9ef82e97d0859c249f_=/usr/local/src/conda-prefix
F77 version : GNU Fortran (conda-forge gcc 11.3.0-19) 11.3.0
Features: 503GB 256threads c++17 OpenMP45 Fortran pthreads HDF5 NC4/HDF5/threadsafe OPeNDAP udunits2 proj xml2 magics curl fftw3 sse3
Libraries: yac/2.6.1 HDF5/1.12.2 proj/9.1.1 xml2/2.10.3 curl/7.88.1 magics/4.13.0
CDI data types: SizeType=size_t
CDI file types: srv ext ieg grb1 grb2 nc1 nc2 nc4 nc4c nc5 nczarr
CDI library version : 2.1.1
cgribex library version : 2.0.2
ecCodes library version : 2.28.0
NetCDF library version : 4.9.1 of Feb 24 2023 22:22:03 $
HDF5 library version : 1.12.2 threadsafe
exse library version : 1.4.2
FILE library version : 1.9.1

if I try to use 'apply', I get these messages:

cdo -add -apply,-seltimestep,1 [ psl_era5_201401.nc psl_era5_201401.nc ] out.nc
cdo(1) seltimestep: Process started
cdo(2) seltimestep: Process started
Warning (cdfInqContents): Coordinates variable realization can't be assigned!
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
#000: H5A.c line 528 in H5Aopen_by_name(): can't open attribute
major: Attribute
minor: Can't open object
#001: H5VLcallback.c line 1091 in H5VL_attr_open(): attribute open failed
major: Virtual Object Layer
minor: Can't open object
#002: H5VLcallback.c line 1058 in H5VL__attr_open(): attribute open failed
major: Virtual Object Layer
minor: Can't open object
#003: H5VLnative_attr.c line 130 in H5VL__native_attr_open(): can't open attribute
major: Attribute
minor: Can't open object
#004: H5Aint.c line 545 in H5A__open_by_name(): unable to load attribute info from object header
major: Attribute
minor: Unable to initialize object
#005: H5Oattribute.c line 476 in H5O__attr_open_by_name(): can't open attribute
major: Attribute
minor: Can't open object
#006: H5Adense.c line 394 in H5A__dense_open(): can't locate attribute in name index
major: Attribute
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
#000: H5A.c line 528 in H5Aopen_by_name(): can't open attribute
major: Attribute
minor: Can't open object
#001: H5VLcallback.c line 1091 in H5VL_attr_open(): attribute open failed
major: Virtual Object Layer
minor: Can't open object
#002: H5VLcallback.c line 1058 in H5VL__attr_open(): attribute open failed
major: Virtual Object Layer
minor: Can't open object
#003: H5VLnative_attr.c line 130 in H5VL__native_attr_open(): can't open attribute
major: Attribute
minor: Can't open object
#004: H5Aint.c line 545 in H5A__open_by_name(): unable to load attribute info from object header
major: Attribute
minor: Unable to initialize object
#005: H5Oattribute.c line 476 in H5O__attr_open_by_name(): can't open attribute
major: Attribute
minor: Can't open object
#006: H5Adense.c line 394 in H5A__dense_open(): can't locate attribute in name index
major: Attribute
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
#000: H5A.c line 528 in H5Aopen_by_name(): can't open attribute
major: Attribute
minor: Can't open object
#001: H5VLcallback.c line 1091 in H5VL_attr_open(): attribute open failed
major: Virtual Object Layer
minor: Can't open object
#002: H5VLcallback.c line 1058 in H5VL__attr_open(): attribute open failed
major: Virtual Object Layer
minor: Can't open object
#003: H5VLnative_attr.c line 130 in H5VL__native_attr_open(): can't open attribute
major: Attribute
minor: Can't open object
#004: H5Aint.c line 545 in H5A__open_by_name(): unable to load attribute info from object header
major: Attribute
minor: Unable to initialize object
#005: H5Oattribute.c line 476 in H5O__attr_open_by_name(): can't open attribute
major: Attribute
minor: Can't open object
#006: H5Adense.c line 394 in H5A__dense_open(): can't locate attribute in name index
major: Attribute
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
#000: H5A.c line 528 in H5Aopen_by_name(): can't open attribute
major: Attribute
minor: Can't open object
#001: H5VLcallback.c line 1091 in H5VL_attr_open(): attribute open failed
major: Virtual Object Layer
minor: Can't open object
#002: H5VLcallback.c line 1058 in H5VL__attr_open(): attribute open failed
major: Virtual Object Layer
minor: Can't open object
#003: H5VLnative_attr.c line 130 in H5VL__native_attr_open(): can't open attribute
major: Attribute
minor: Can't open object
#004: H5Aint.c line 545 in H5A__open_by_name(): unable to load attribute info from object header
major: Attribute
minor: Unable to initialize object
#005: H5Oattribute.c line 494 in H5O__attr_open_by_name(): can't locate attribute: '_QuantizeBitGroomNumberOfSignificantDigits'
major: Attribute
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
#000: H5A.c line 528 in H5Aopen_by_name(): can't open attribute
major: Attribute
minor: Can't open object
#001: H5VLcallback.c line 1091 in H5VL_attr_open(): attribute open failed
major: Virtual Object Layer
minor: Can't open object
#002: H5VLcallback.c line 1058 in H5VL__attr_open(): attribute open failed
major: Virtual Object Layer
minor: Can't open object
#003: H5VLnative_attr.c line 130 in H5VL__native_attr_open(): can't open attribute
major: Attribute
minor: Can't open object
#004: H5Aint.c line 545 in H5A__open_by_name(): unable to load attribute info from object header
major: Attribute
minor: Unable to initialize object
#005: H5Oattribute.c line 494 in H5O__attr_open_by_name(): can't locate attribute: '_QuantizeGranularBitRoundNumberOfSignificantDigits'
major: Attribute
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
#000: H5A.c line 528 in H5Aopen_by_name(): can't open attribute
major: Attribute
minor: Can't open object
#001: H5VLcallback.c line 1091 in H5VL_attr_open(): attribute open failed
major: Virtual Object Layer
minor: Can't open object
#002: H5VLcallback.c line 1058 in H5VL__attr_open(): attribute open failed
major: Virtual Object Layer
minor: Can't open object
#003: H5VLnative_attr.c line 130 in H5VL__native_attr_open(): can't open attribute
major: Attribute
minor: Can't open object
#004: H5Aint.c line 545 in H5A__open_by_name(): unable to load attribute info from object header
major: Attribute
minor: Unable to initialize object
#005: H5Oattribute.c line 494 in H5O__attr_open_by_name(): can't locate attribute: '_QuantizeBitRoundNumberOfSignificantBits'
major: Attribute
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
#000: H5A.c line 528 in H5Aopen_by_name(): can't open attribute
major: Attribute
minor: Can't open object
#001: H5VLcallback.c line 1091 in H5VL_attr_open(): attribute open failed
major: Virtual Object Layer
minor: Can't open object
#002: H5VLcallback.c line 1058 in H5VL__attr_open(): attribute open failed
major: Virtual Object Layer
minor: Can't open object
#003: H5VLnative_attr.c line 130 in H5VL__native_attr_open(): can't open attribute
major: Attribute
minor: Can't open object
#004: H5Aint.c line 545 in H5A__open_by_name(): unable to load attribute info from object header
major: Attribute
minor: Unable to initialize object
#005: H5Oattribute.c line 476 in H5O__attr_open_by_name(): can't open attribute
major: Attribute
minor: Can't open object
#006: H5Adense.c line 394 in H5A__dense_open(): can't locate attribute in name index
major: Attribute
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
#000: H5A.c line 528 in H5Aopen_by_name(): can't open attribute
major: Attribute
minor: Can't open object
#001: H5VLcallback.c line 1091 in H5VL_attr_open(): attribute open failed
major: Virtual Object Layer
minor: Can't open object
#002: H5VLcallback.c line 1058 in H5VL__attr_open(): attribute open failed
major: Virtual Object Layer
minor: Can't open object
#003: H5VLnative_attr.c line 130 in H5VL__native_attr_open(): can't open attribute
major: Attribute
minor: Can't open object
#004: H5Aint.c line 545 in H5A__open_by_name(): unable to load attribute info from object header
major: Attribute
minor: Unable to initialize object
#005: H5Oattribute.c line 476 in H5O__attr_open_by_name(): can't open attribute
major: Attribute
minor: Can't open object
#006: H5Adense.c line 394 in H5A__dense_open(): can't locate attribute in name index
major: Attribute
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
#000: H5A.c line 528 in H5Aopen_by_name(): can't open attribute
major: Attribute
minor: Can't open object
#001: H5VLcallback.c line 1091 in H5VL_attr_open(): attribute open failed
major: Virtual Object Layer
minor: Can't open object
#002: H5VLcallback.c line 1058 in H5VL__attr_open(): attribute open failed
major: Virtual Object Layer
minor: Can't open object
#003: H5VLnative_attr.c line 130 in H5VL__native_attr_open(): can't open attribute
major: Attribute
minor: Can't open object
#004: H5Aint.c line 545 in H5A__open_by_name(): unable to load attribute info from object header
major: Attribute
minor: Unable to initialize object
#005: H5Oattribute.c line 476 in H5O__attr_open_by_name(): can't open attribute
major: Attribute
minor: Can't open object
#006: H5Adense.c line 394 in H5A__dense_open(): can't locate attribute in name index
major: Attribute
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
#000: H5A.c line 528 in H5Aopen_by_name(): can't open attribute
major: Attribute
minor: Can't open object
#001: H5VLcallback.c line 1091 in H5VL_attr_open(): attribute open failed
major: Virtual Object Layer
minor: Can't open object
#002: H5VLcallback.c line 1058 in H5VL__attr_open(): attribute open failed
major: Virtual Object Layer
minor: Can't open object
#003: H5VLnative_attr.c line 130 in H5VL__native_attr_open(): can't open attribute
major: Attribute
minor: Can't open object
#004: H5Aint.c line 545 in H5A__open_by_name(): unable to load attribute info from object header
major: Attribute
minor: Unable to initialize object
#005: H5Oattribute.c line 476 in H5O__attr_open_by_name(): can't open attribute
major: Attribute
minor: Can't open object
#006: H5Adense.c line 394 in H5A__dense_open(): can't locate attribute in name index
major: Attribute
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
#000: H5A.c line 528 in H5Aopen_by_name(): can't open attribute
major: Attribute
minor: Can't open object
#001: H5VLcallback.c line 1091 in H5VL_attr_open(): attribute open failed
major: Virtual Object Layer
minor: Can't open object
#002: H5VLcallback.c line 1058 in H5VL__attr_open(): attribute open failed
major: Virtual Object Layer
minor: Can't open object
#003: H5VLnative_attr.c line 130 in H5VL__native_attr_open(): can't open attribute
major: Attribute
minor: Can't open object
#004: H5Aint.c line 545 in H5A__open_by_name(): unable to load attribute info from object header
major: Attribute
minor: Unable to initialize object
#005: H5Oattribute.c line 476 in H5O__attr_open_by_name(): can't open attribute
major: Attribute
minor: Can't open object
#006: H5Adense.c line 394 in H5A__dense_open(): can't locate attribute in name index
major: Attribute
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
#000: H5A.c line 528 in H5Aopen_by_name(): can't open attribute
major: Attribute
minor: Can't open object
#001: H5VLcallback.c line 1091 in H5VL_attr_open(): attribute open failed
major: Virtual Object Layer
minor: Can't open object
#002: H5VLcallback.c line 1058 in H5VL__attr_open(): attribute open failed
major: Virtual Object Layer
minor: Can't open object
#003: H5VLnative_attr.c line 130 in H5VL__native_attr_open(): can't open attribute
major: Attribute
minor: Can't open object
#004: H5Aint.c line 545 in H5A__open_by_name(): unable to load attribute info from object header
major: Attribute
minor: Unable to initialize object
#005: H5Oattribute.c line 476 in H5O__attr_open_by_name(): can't open attribute
major: Attribute
minor: Can't open object
#006: H5Adense.c line 394 in H5A__dense_open(): can't locate attribute in name index
major: Attribute
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
#000: H5A.c line 528 in H5Aopen_by_name(): can't open attribute
major: Attribute
minor: Can't open object
#001: H5VLcallback.c line 1091 in H5VL_attr_open(): attribute open failed
major: Virtual Object Layer
minor: Can't open object
#002: H5VLcallback.c line 1058 in H5VL__attr_open(): attribute open failed
major: Virtual Object Layer
minor: Can't open object
#003: H5VLnative_attr.c line 130 in H5VL__native_attr_open(): can't open attribute
major: Attribute
minor: Can't open object
#004: H5Aint.c line 545 in H5A__open_by_name(): unable to load attribute info from object header
major: Attribute
minor: Unable to initialize object
#005: H5Oattribute.c line 494 in H5O__attr_open_by_name(): can't locate attribute: '_QuantizeBitGroomNumberOfSignificantDigits'
major: Attribute
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
#000: H5A.c line 528 in H5Aopen_by_name(): can't open attribute
major: Attribute
minor: Can't open object
#001: H5VLcallback.c line 1091 in H5VL_attr_open(): attribute open failed
major: Virtual Object Layer
minor: Can't open object
#002: H5VLcallback.c line 1058 in H5VL__attr_open(): attribute open failed
major: Virtual Object Layer
minor: Can't open object
#003: H5VLnative_attr.c line 130 in H5VL__native_attr_open(): can't open attribute
major: Attribute
minor: Can't open object
#004: H5Aint.c line 545 in H5A__open_by_name(): unable to load attribute info from object header
major: Attribute
minor: Unable to initialize object
#005: H5Oattribute.c line 494 in H5O__attr_open_by_name(): can't locate attribute: '_QuantizeGranularBitRoundNumberOfSignificantDigits'
major: Attribute
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
#000: H5A.c line 528 in H5Aopen_by_name(): can't open attribute
major: Attribute
minor: Can't open object
#001: H5VLcallback.c line 1091 in H5VL_attr_open(): attribute open failed
major: Virtual Object Layer
minor: Can't open object
#002: H5VLcallback.c line 1058 in H5VL__attr_open(): attribute open failed
major: Virtual Object Layer
minor: Can't open object
#003: H5VLnative_attr.c line 130 in H5VL__native_attr_open(): can't open attribute
major: Attribute
minor: Can't open object
#004: H5Aint.c line 545 in H5A__open_by_name(): unable to load attribute info from object header
major: Attribute
minor: Unable to initialize object
#005: H5Oattribute.c line 494 in H5O__attr_open_by_name(): can't locate attribute: '_QuantizeBitRoundNumberOfSignificantBits'
major: Attribute
minor: Object not found
Warning (cdfInqContents): Coordinates variable realization can't be assigned!
cdo(2) seltimestep: Processed 1038240 values from 1 variable over 2 timesteps
cdo(1) seltimestep: Processed 1038240 values from 1 variable over 2 timesteps
cdo add: Processed 2076480 values from 2 variables over 2 timesteps [0.08s 71MB]

But the same calculation without 'apply' seems ok

cdo -seltimestep,1 -add psl_era5_201401.nc psl_era5_201401.nc out.nc
cdo(1) add: Process started
Warning (cdfInqContents): Coordinates variable realization can't be assigned!
Warning (cdfInqContents): Coordinates variable realization can't be assigned!
cdo(1) add: Processed 64370880 values from 2 variables over 62 timesteps
cdo seltimestep: Processed 1038240 values from 1 variable over 2 timesteps [0.39s 52MB]

For this version,

Climate Data Operators version 2.1.1 (https://mpimet.mpg.de/cdo)
System: x86_64-conda-linux-gnu
CXX Compiler: /home/conda/feedstock_root/build_artifacts/cdo_1671208954590/_build_env/bin/x86_64-conda-linux-gnu-c++ -fPIC -DPIC -g -O2 -fopenmp -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/project/17001770/climate_branch/cmp/chuaxr/miniconda3/envs/research/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/cdo_1671208954590/work=/usr/local/src/conda/cdo-2.1.1 -fdebug-prefix-map=/home/project/17001770/climate_branch/cmp/chuaxr/miniconda3/envs/research=/usr/local/src/conda-prefix -fopenmp -pthread
CXX version : unknown
C Compiler: /home/conda/feedstock_root/build_artifacts/cdo_1671208954590/_build_env/bin/x86_64-conda-linux-gnu-cc -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/project/17001770/climate_branch/cmp/chuaxr/miniconda3/envs/research/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/cdo_1671208954590/work=/usr/local/src/conda/cdo-2.1.1 -fdebug-prefix-map=/home/project/17001770/climate_branch/cmp/chuaxr/miniconda3/envs/research=/usr/local/src/conda-prefix -fopenmp -pthread -pthread
C version : unknown
F77 Compiler: /home/conda/feedstock_root/build_artifacts/cdo_1671208954590/_build_env/bin/x86_64-conda-linux-gnu-gfortran -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/project/17001770/climate_branch/cmp/chuaxr/miniconda3/envs/research/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/cdo_1671208954590/work=/usr/local/src/conda/cdo-2.1.1 -fdebug-prefix-map=/home/project/17001770/climate_branch/cmp/chuaxr/miniconda3/envs/research=/usr/local/src/conda-prefix
F77 version : GNU Fortran (conda-forge gcc 11.3.0-19) 11.3.0
Features: 503GB 256threads c++17 OpenMP45 Fortran pthreads HDF5 NC4/HDF5/threadsafe OPeNDAP udunits2 proj xml2 magics curl fftw3 sse3
Libraries: yac/2.6.1 HDF5/1.12.2 proj/9.1.1 xml2/2.10.3 curl/7.87.0(h7.86.0) magics/4.12.1
CDI data types: SizeType=size_t
CDI file types: srv ext ieg grb1 grb2 nc1 nc2 nc4 nc4c nc5 nczarr
CDI library version : 2.1.1
cgribex library version : 2.0.2
ecCodes library version : 2.27.1
NetCDF library version : 4.8.1 of Oct 31 2022 22:16:44 $
HDF5 library version : 1.12.2 threadsafe
exse library version : 1.4.2
FILE library version : 1.9.1

using 'apply' doesn't give an error:

cdo -add -apply,-seltimestep,1 [ psl_era5_201401.nc psl_era5_201401.nc ] out.nc
cdo(1) seltimestep: Process started
cdo(2) seltimestep: Process started
Warning (cdfInqContents): Coordinates variable realization can't be assigned!
Warning (cdfInqContents): Coordinates variable realization can't be assigned!
cdo(2) seltimestep: Processed 1038240 values from 1 variable over 2 timesteps
cdo(1) seltimestep: Processed 1038240 values from 1 variable over 2 timesteps
cdo add: Processed 2076480 values from 2 variables over 2 timesteps [0.09s 68MB]

Any thoughts as to what is going on? Is it safe to ignore the errors?

psl_era5_201401.nc (123 MB) psl_era5_201401.nc

Replies (4)

RE: HDF error messages dependent on use of cdo version and apply keyword - Added by Ralf Mueller about 1 year ago

hi Chua!

The apply keyword is meant to be used with operators, which have an arbitrary number of input files (e.g. select, merge, cat). add only takes two inputs, but this is an interesting use-case.
using apply invokes a multi-threaded access to the inputs, but the underlying HD5-installation might be build in a non-threadsafe way. Yours seem to be thread-safe, but I would add the -L to serialize the IO in the call:

% cdo -s -add -apply,-seltimestep,1 [ psl_era5_201401.nc psl_era5_201401.nc ] out.nc      
Warning (cdfInqContents): Coordinates variable realization can't be assigned!
Warning (cdfInqContents): Coordinates variable realization can't be assigned!
cdo    add (Warning): Using a non-thread-safe NetCDF4/HDF5 library in a multi-threaded environment may lead to erroneous results!
cdo    add (Warning): Use a thread-safe NetCDF4/HDF5 library or the CDO option -L to avoid such errors.

% cdo -s -L -add -apply,-seltimestep,1 [ psl_era5_201401.nc psl_era5_201401.nc ] out.nc 
Warning (cdfInqContents): Coordinates variable realization can't be assigned!
Warning (cdfInqContents): Coordinates variable realization can't be assigned!

using

Climate Data Operators version 2.1.1 (https://mpimet.mpg.de/cdo)
System: x86_64-pc-linux-gnu
CXX Compiler: g++ -g -O3 -Wall -fopenmp  -pthread
CXX version : g++ (GCC) 12.2.1 20230201
C Compiler: gcc -g -O3 -Wall -fopenmp  -pthread -pthread
C version : gcc (GCC) 12.2.1 20230201
F77 Compiler: gfortran -g -O2
F77 version : GNU Fortran (GCC) 12.2.1 20230201
Features: 15GB 8threads c++17 OpenMP45 Fortran pthreads HDF5 NC4/HDF5 OPeNDAP sz udunits2 proj xml2 magics curl fftw3 sse2
Libraries: yac/2.6.1 HDF5/1.12.2 proj/9.1.1 xml2/2.10.3 curl/7.87.0 magics/4.12.0
CDI data types: SizeType=size_t
CDI file types: srv ext ieg grb1 grb2 nc1 nc2 nc4 nc4c nc5 nczarr 
     CDI library version : 2.1.1
 cgribex library version : 2.0.2
 ecCodes library version : 2.27.1
  NetCDF library version : 4.9.0 of Jun 29 2022 19:28:14 $
    HDF5 library version : 1.12.2
    exse library version : 1.4.2
    FILE library version : 1.9.1

RE: HDF error messages dependent on use of cdo version and apply keyword - Added by XR Chua about 1 year ago

Thanks Ralf!

Re 1: add is not my actual use case, I wanted an example I could use with just one file

Re 2: I can still get the error message despite the use of -L (see below). Is it due to the NetCDF library version used in the CDO installation?

System: x86_64-conda-linux-gnu
CXX Compiler: /home/conda/feedstock_root/build_artifacts/cdo_1677416397422/_build_env/bin/x86_64-conda-linux-gnu-c++ -fPIC -DPIC -g -O2 -fopenmp -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/project/17001770/climate_branch/cmp/chuaxr/envs/persiann_vs_imerg/35f6c6eca1147905aa9783c9e43f35c5_/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/cdo_1677416397422/work=/usr/local/src/conda/cdo-2.1.1 -fdebug-prefix-map=/home/project/17001770/climate_branch/cmp/chuaxr/envs/persiann_vs_imerg/35f6c6eca1147905aa9783c9e43f35c5_=/usr/local/src/conda-prefix -fopenmp -pthread
CXX version : unknown
C Compiler: /home/conda/feedstock_root/build_artifacts/cdo_1677416397422/_build_env/bin/x86_64-conda-linux-gnu-cc -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/project/17001770/climate_branch/cmp/chuaxr/envs/persiann_vs_imerg/35f6c6eca1147905aa9783c9e43f35c5_/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/cdo_1677416397422/work=/usr/local/src/conda/cdo-2.1.1 -fdebug-prefix-map=/home/project/17001770/climate_branch/cmp/chuaxr/envs/persiann_vs_imerg/35f6c6eca1147905aa9783c9e43f35c5_=/usr/local/src/conda-prefix -fopenmp -pthread -pthread
C version : unknown
F77 Compiler: /home/conda/feedstock_root/build_artifacts/cdo_1677416397422/_build_env/bin/x86_64-conda-linux-gnu-gfortran -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/project/17001770/climate_branch/cmp/chuaxr/envs/persiann_vs_imerg/35f6c6eca1147905aa9783c9e43f35c5_/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/cdo_1677416397422/work=/usr/local/src/conda/cdo-2.1.1 -fdebug-prefix-map=/home/project/17001770/climate_branch/cmp/chuaxr/envs/persiann_vs_imerg/35f6c6eca1147905aa9783c9e43f35c5_=/usr/local/src/conda-prefix
F77 version : GNU Fortran (conda-forge gcc 11.3.0-19) 11.3.0
Features: 503GB 256threads c++17 OpenMP45 Fortran pthreads HDF5 NC4/HDF5/threadsafe OPeNDAP udunits2 proj xml2 magics curl fftw3 sse3
Libraries: yac/2.6.1 HDF5/1.12.2 proj/9.1.1 xml2/2.10.3 curl/7.88.1 magics/4.13.0
CDI data types: SizeType=size_t
CDI file types: srv ext ieg grb1 grb2 nc1 nc2 nc4 nc4c nc5 nczarr
CDI library version : 2.1.1
cgribex library version : 2.0.2
ecCodes library version : 2.28.0
NetCDF library version : 4.9.1 of Feb 24 2023 22:22:03 $
HDF5 library version : 1.12.2 threadsafe
exse library version : 1.4.2
FILE library version : 1.9.1

cdo -L -add -apply,-seltimestep,1 [ psl_era5_201401.nc psl_era5_201401.nc ] out.nc
cdo(1) seltimestep: Process started
cdo(2) seltimestep: Process started
Warning (cdfInqContents): Coordinates variable realization can't be assigned!
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
  #000: H5A.c line 528 in H5Aopen_by_name(): can't open attribute
    major: Attribute
    minor: Can't open object
  #001: H5VLcallback.c line 1091 in H5VL_attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #002: H5VLcallback.c line 1058 in H5VL__attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #003: H5VLnative_attr.c line 130 in H5VL__native_attr_open(): can't open attribute
    major: Attribute
    minor: Can't open object
  #004: H5Aint.c line 545 in H5A__open_by_name(): unable to load attribute info from object header
    major: Attribute
    minor: Unable to initialize object
  #005: H5Oattribute.c line 476 in H5O__attr_open_by_name(): can't open attribute
    major: Attribute
    minor: Can't open object
  #006: H5Adense.c line 394 in H5A__dense_open(): can't locate attribute in name index
    major: Attribute
    minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
  #000: H5A.c line 528 in H5Aopen_by_name(): can't open attribute
    major: Attribute
    minor: Can't open object
  #001: H5VLcallback.c line 1091 in H5VL_attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #002: H5VLcallback.c line 1058 in H5VL__attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #003: H5VLnative_attr.c line 130 in H5VL__native_attr_open(): can't open attribute
    major: Attribute
    minor: Can't open object
  #004: H5Aint.c line 545 in H5A__open_by_name(): unable to load attribute info from object header
    major: Attribute
    minor: Unable to initialize object
  #005: H5Oattribute.c line 476 in H5O__attr_open_by_name(): can't open attribute
    major: Attribute
    minor: Can't open object
  #006: H5Adense.c line 394 in H5A__dense_open(): can't locate attribute in name index
    major: Attribute
    minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
  #000: H5A.c line 528 in H5Aopen_by_name(): can't open attribute
    major: Attribute
    minor: Can't open object
  #001: H5VLcallback.c line 1091 in H5VL_attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #002: H5VLcallback.c line 1058 in H5VL__attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #003: H5VLnative_attr.c line 130 in H5VL__native_attr_open(): can't open attribute
    major: Attribute
    minor: Can't open object
  #004: H5Aint.c line 545 in H5A__open_by_name(): unable to load attribute info from object header
    major: Attribute
    minor: Unable to initialize object
  #005: H5Oattribute.c line 476 in H5O__attr_open_by_name(): can't open attribute
    major: Attribute
    minor: Can't open object
  #006: H5Adense.c line 394 in H5A__dense_open(): can't locate attribute in name index
    major: Attribute
    minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
  #000: H5A.c line 528 in H5Aopen_by_name(): can't open attribute
    major: Attribute
    minor: Can't open object
  #001: H5VLcallback.c line 1091 in H5VL_attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #002: H5VLcallback.c line 1058 in H5VL__attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #003: H5VLnative_attr.c line 130 in H5VL__native_attr_open(): can't open attribute
    major: Attribute
    minor: Can't open object
  #004: H5Aint.c line 545 in H5A__open_by_name(): unable to load attribute info from object header
    major: Attribute
    minor: Unable to initialize object
  #005: H5Oattribute.c line 494 in H5O__attr_open_by_name(): can't locate attribute: '_QuantizeBitGroomNumberOfSignificantDigits'
    major: Attribute
    minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
  #000: H5A.c line 528 in H5Aopen_by_name(): can't open attribute
    major: Attribute
    minor: Can't open object
  #001: H5VLcallback.c line 1091 in H5VL_attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #002: H5VLcallback.c line 1058 in H5VL__attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #003: H5VLnative_attr.c line 130 in H5VL__native_attr_open(): can't open attribute
    major: Attribute
    minor: Can't open object
  #004: H5Aint.c line 545 in H5A__open_by_name(): unable to load attribute info from object header
    major: Attribute
    minor: Unable to initialize object
  #005: H5Oattribute.c line 494 in H5O__attr_open_by_name(): can't locate attribute: '_QuantizeGranularBitRoundNumberOfSignificantDigits'
    major: Attribute
    minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
  #000: H5A.c line 528 in H5Aopen_by_name(): can't open attribute
    major: Attribute
    minor: Can't open object
  #001: H5VLcallback.c line 1091 in H5VL_attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #002: H5VLcallback.c line 1058 in H5VL__attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #003: H5VLnative_attr.c line 130 in H5VL__native_attr_open(): can't open attribute
    major: Attribute
    minor: Can't open object
  #004: H5Aint.c line 545 in H5A__open_by_name(): unable to load attribute info from object header
    major: Attribute
    minor: Unable to initialize object
  #005: H5Oattribute.c line 494 in H5O__attr_open_by_name(): can't locate attribute: '_QuantizeBitRoundNumberOfSignificantBits'
    major: Attribute
    minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
  #000: H5A.c line 528 in H5Aopen_by_name(): can't open attribute
    major: Attribute
    minor: Can't open object
  #001: H5VLcallback.c line 1091 in H5VL_attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #002: H5VLcallback.c line 1058 in H5VL__attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #003: H5VLnative_attr.c line 130 in H5VL__native_attr_open(): can't open attribute
    major: Attribute
    minor: Can't open object
  #004: H5Aint.c line 545 in H5A__open_by_name(): unable to load attribute info from object header
    major: Attribute
    minor: Unable to initialize object
  #005: H5Oattribute.c line 476 in H5O__attr_open_by_name(): can't open attribute
    major: Attribute
    minor: Can't open object
  #006: H5Adense.c line 394 in H5A__dense_open(): can't locate attribute in name index
    major: Attribute
    minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
  #000: H5A.c line 528 in H5Aopen_by_name(): can't open attribute
    major: Attribute
    minor: Can't open object
  #001: H5VLcallback.c line 1091 in H5VL_attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #002: H5VLcallback.c line 1058 in H5VL__attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #003: H5VLnative_attr.c line 130 in H5VL__native_attr_open(): can't open attribute
    major: Attribute
    minor: Can't open object
  #004: H5Aint.c line 545 in H5A__open_by_name(): unable to load attribute info from object header
    major: Attribute
    minor: Unable to initialize object
  #005: H5Oattribute.c line 476 in H5O__attr_open_by_name(): can't open attribute
    major: Attribute
    minor: Can't open object
  #006: H5Adense.c line 394 in H5A__dense_open(): can't locate attribute in name index
    major: Attribute
    minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
  #000: H5A.c line 528 in H5Aopen_by_name(): can't open attribute
    major: Attribute
    minor: Can't open object
  #001: H5VLcallback.c line 1091 in H5VL_attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #002: H5VLcallback.c line 1058 in H5VL__attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #003: H5VLnative_attr.c line 130 in H5VL__native_attr_open(): can't open attribute
    major: Attribute
    minor: Can't open object
  #004: H5Aint.c line 545 in H5A__open_by_name(): unable to load attribute info from object header
    major: Attribute
    minor: Unable to initialize object
  #005: H5Oattribute.c line 476 in H5O__attr_open_by_name(): can't open attribute
    major: Attribute
    minor: Can't open object
  #006: H5Adense.c line 394 in H5A__dense_open(): can't locate attribute in name index
    major: Attribute
    minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
  #000: H5A.c line 528 in H5Aopen_by_name(): can't open attribute
    major: Attribute
    minor: Can't open object
  #001: H5VLcallback.c line 1091 in H5VL_attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #002: H5VLcallback.c line 1058 in H5VL__attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #003: H5VLnative_attr.c line 130 in H5VL__native_attr_open(): can't open attribute
    major: Attribute
    minor: Can't open object
  #004: H5Aint.c line 545 in H5A__open_by_name(): unable to load attribute info from object header
    major: Attribute
    minor: Unable to initialize object
  #005: H5Oattribute.c line 476 in H5O__attr_open_by_name(): can't open attribute
    major: Attribute
    minor: Can't open object
  #006: H5Adense.c line 394 in H5A__dense_open(): can't locate attribute in name index
    major: Attribute
    minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
  #000: H5A.c line 528 in H5Aopen_by_name(): can't open attribute
    major: Attribute
    minor: Can't open object
  #001: H5VLcallback.c line 1091 in H5VL_attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #002: H5VLcallback.c line 1058 in H5VL__attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #003: H5VLnative_attr.c line 130 in H5VL__native_attr_open(): can't open attribute
    major: Attribute
    minor: Can't open object
  #004: H5Aint.c line 545 in H5A__open_by_name(): unable to load attribute info from object header
    major: Attribute
    minor: Unable to initialize object
  #005: H5Oattribute.c line 476 in H5O__attr_open_by_name(): can't open attribute
    major: Attribute
    minor: Can't open object
  #006: H5Adense.c line 394 in H5A__dense_open(): can't locate attribute in name index
    major: Attribute
    minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
  #000: H5A.c line 528 in H5Aopen_by_name(): can't open attribute
    major: Attribute
    minor: Can't open object
  #001: H5VLcallback.c line 1091 in H5VL_attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #002: H5VLcallback.c line 1058 in H5VL__attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #003: H5VLnative_attr.c line 130 in H5VL__native_attr_open(): can't open attribute
    major: Attribute
    minor: Can't open object
  #004: H5Aint.c line 545 in H5A__open_by_name(): unable to load attribute info from object header
    major: Attribute
    minor: Unable to initialize object
  #005: H5Oattribute.c line 476 in H5O__attr_open_by_name(): can't open attribute
    major: Attribute
    minor: Can't open object
  #006: H5Adense.c line 394 in H5A__dense_open(): can't locate attribute in name index
    major: Attribute
    minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
  #000: H5A.c line 528 in H5Aopen_by_name(): can't open attribute
    major: Attribute
    minor: Can't open object
  #001: H5VLcallback.c line 1091 in H5VL_attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #002: H5VLcallback.c line 1058 in H5VL__attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #003: H5VLnative_attr.c line 130 in H5VL__native_attr_open(): can't open attribute
    major: Attribute
    minor: Can't open object
  #004: H5Aint.c line 545 in H5A__open_by_name(): unable to load attribute info from object header
    major: Attribute
    minor: Unable to initialize object
  #005: H5Oattribute.c line 494 in H5O__attr_open_by_name(): can't locate attribute: '_QuantizeBitGroomNumberOfSignificantDigits'
    major: Attribute
    minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
  #000: H5A.c line 528 in H5Aopen_by_name(): can't open attribute
    major: Attribute
    minor: Can't open object
  #001: H5VLcallback.c line 1091 in H5VL_attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #002: H5VLcallback.c line 1058 in H5VL__attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #003: H5VLnative_attr.c line 130 in H5VL__native_attr_open(): can't open attribute
    major: Attribute
    minor: Can't open object
  #004: H5Aint.c line 545 in H5A__open_by_name(): unable to load attribute info from object header
    major: Attribute
    minor: Unable to initialize object
  #005: H5Oattribute.c line 494 in H5O__attr_open_by_name(): can't locate attribute: '_QuantizeGranularBitRoundNumberOfSignificantDigits'
    major: Attribute
    minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 1:
  #000: H5A.c line 528 in H5Aopen_by_name(): can't open attribute
    major: Attribute
    minor: Can't open object
  #001: H5VLcallback.c line 1091 in H5VL_attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #002: H5VLcallback.c line 1058 in H5VL__attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #003: H5VLnative_attr.c line 130 in H5VL__native_attr_open(): can't open attribute
    major: Attribute
    minor: Can't open object
  #004: H5Aint.c line 545 in H5A__open_by_name(): unable to load attribute info from object header
    major: Attribute
    minor: Unable to initialize object
  #005: H5Oattribute.c line 494 in H5O__attr_open_by_name(): can't locate attribute: '_QuantizeBitRoundNumberOfSignificantBits'
    major: Attribute
    minor: Object not found
Warning (cdfInqContents): Coordinates variable realization can't be assigned!
cdo(2) seltimestep: Processed 1038240 values from 1 variable over 2 timesteps
cdo(1) seltimestep: Processed 1038240 values from 1 variable over 2 timesteps
cdo    add: Processed 2076480 values from 2 variables over 2 timesteps [0.09s 71MB]

RE: HDF error messages dependent on use of cdo version and apply keyword - Added by Uwe Schulzweida about 1 year ago

Yes this is a bug in NetCDF 4.9.0 + 4.9.1. We have already sent a bug report to NetCDF. The error messages can be ignored. Unfortunately, the error messages cannot be turned off.

RE: HDF error messages dependent on use of cdo version and apply keyword - Added by XR Chua about 1 year ago

Thanks Uwe and Ralf for your help!

(1-4/4)

Project

General

Profile

CDO