Error (cdf_put_vara_float) : NetCDF: HDF error

Added by Beata Szabo-Takacs over 6 years ago

Dear CDO Users,

I would like to merge seven Euro-Cordex netCDF files with cdo mergetime operator, but I get the following error message:

cdf_put_vara_float : ncid = 458752 varid = 9 val0 = 0.000000
    Error (cdf_put_vara_float) : NetCDF: HDF error
    Segmentation fault (core dumped)

I wanted to attach one of these netCDF files but their size is too large. I tride to create a smaller test.nc file with

cdo sellonlatbox,6,29,46,63 pr_EUR-11_MPI-M-MPI-ESM-LR_historical_r1i1p1_CLMcom-CCLM4-8-17_v1_day_19710101-19751231.nc test.nc

but I got the error:

Error (cdf_enddef) : NetCDF: HDF error 
   Segmentation fault (core dumped)

I tried also ncks -d time,1,5 pr_EUR-11_MPI-M-MPI-ESM-LR_historical_r1i1p1_CLMcom-CCLM4-8-17_v1_day_19710101-19751231.nc test.nc but I got:

nco_err_exit(): ERROR Short NCO-generated message (usually name of function that triggered error): nco_enddef()
  nco_err_exit(): ERROR Error code is -101. Translation into English with nc_strerror(-101) is "NetCDF: HDF error" 
  nco_err_exit(): ERROR NCO will now exit with system call exit(EXIT_FAILURE)
  Segmentation fault (core dumped)

I do not know how I should fix this error. Could someone suggest a solution? Thank you for your help in advance!

Replies (4)

RE: Error (cdf_put_vara_float) : NetCDF: HDF error - Added by Karin Meier-Fleischer over 6 years ago

Hi Beata,

which version of CDO are you using?

I've downloaded some files from ESGF and with CDO version 1.8.2 I don't have any problem with mergetime.

infile1=pr_EUR-11_MPI-M-MPI-ESM-LR_historical_r1i1p1_CLMcom-CCLM4-8-17_v1_day_19710101-19751231.nc
infile2=pr_EUR-11_MPI-M-MPI-ESM-LR_historical_r1i1p1_CLMcom-CCLM4-8-17_v1_day_19760101-19801231.nc
infile3=pr_EUR-11_MPI-M-MPI-ESM-LR_historical_r1i1p1_CLMcom-CCLM4-8-17_v1_day_19810101-19851231.nc

cdo -f nc -mergetime $infile1 $infile2 $infile3 pr_merged.nc

pr_merged.nc is 3.8GB

cdo sinfo pr_merged.nc

   File format : NetCDF2
    -1 : Institut Source   Steptype Levels Num    Points Num Dtype : Parameter ID
     1 : unknown  CLMcom-CCLM4-8-17 instant       1   1    174688   1  F32  : -1            
   Grid coordinates :
     1 : curvilinear              : points=174688 (424x412)
                              lon : -44.59386 to 64.96438 degrees_east
                              lat : 21.98783 to 72.585 degrees_east
                        available : cellbounds
                          mapping : rotated_latitude_longitude
                             rlon : -28.375 to 18.155 by 0.11 degrees
                             rlat : -23.375 to 21.835 by 0.1100006 degrees
   Vertical coordinates :
     1 : surface                  : levels=1
   Time coordinate :  5479 steps
     RefTime =  1949-12-01 00:00:00  Units = days  Calendar = proleptic_gregorian  Bounds = true
  YYYY-MM-DD hh:mm:ss  YYYY-MM-DD hh:mm:ss  YYYY-MM-DD hh:mm:ss  YYYY-MM-DD hh:mm:ss
  1971-01-01 12:00:00  1971-01-02 12:00:00  1971-01-03 12:00:00  1971-01-04 12:00:00
  1971-01-05 12:00:00  1971-01-06 12:00:00  1971-01-07 12:00:00  1971-01-08 12:00:00
  1971-01-09 12:00:00  1971-01-10 12:00:00  1971-01-11 12:00:00  1971-01-12 12:00:00
  1971-01-13 12:00:00  1971-01-14 12:00:00  1971-01-15 12:00:00  1971-01-16 12:00:00
  1971-01-17 12:00:00  1971-01-18 12:00:00  1971-01-19 12:00:00  1971-01-20 12:00:00
  1971-01-21 12:00:00  1971-01-22 12:00:00  1971-01-23 12:00:00  1971-01-24 12:00:00
  1971-01-25 12:00:00  1971-01-26 12:00:00  1971-01-27 12:00:00  1971-01-28 12:00:00
  1971-01-29 12:00:00  1971-01-30 12:00:00  1971-01-31 12:00:00  1971-02-01 12:00:00
  1971-02-02 12:00:00  1971-02-03 12:00:00  1971-02-04 12:00:00  1971-02-05 12:00:00
  1971-02-06 12:00:00  1971-02-07 12:00:00  1971-02-08 12:00:00  1971-02-09 12:00:00
  1971-02-10 12:00:00  1971-02-11 12:00:00  1971-02-12 12:00:00  1971-02-13 12:00:00
  1971-02-14 12:00:00  1971-02-15 12:00:00  1971-02-16 12:00:00  1971-02-17 12:00:00
  1971-02-18 12:00:00  1971-02-19 12:00:00  1971-02-20 12:00:00  1971-02-21 12:00:00
  1971-02-22 12:00:00  1971-02-23 12:00:00  1971-02-24 12:00:00  1971-02-25 12:00:00
  1971-02-26 12:00:00  1971-02-27 12:00:00  1971-02-28 12:00:00  1971-03-01 12:00:00
   ................................................................................
   ................................................................................
   .............................................
  1985-11-03 12:00:00  1985-11-04 12:00:00  1985-11-05 12:00:00  1985-11-06 12:00:00
  1985-11-07 12:00:00  1985-11-08 12:00:00  1985-11-09 12:00:00  1985-11-10 12:00:00
  1985-11-11 12:00:00  1985-11-12 12:00:00  1985-11-13 12:00:00  1985-11-14 12:00:00
  1985-11-15 12:00:00  1985-11-16 12:00:00  1985-11-17 12:00:00  1985-11-18 12:00:00
  1985-11-19 12:00:00  1985-11-20 12:00:00  1985-11-21 12:00:00  1985-11-22 12:00:00
  1985-11-23 12:00:00  1985-11-24 12:00:00  1985-11-25 12:00:00  1985-11-26 12:00:00
  1985-11-27 12:00:00  1985-11-28 12:00:00  1985-11-29 12:00:00  1985-11-30 12:00:00
  1985-12-01 12:00:00  1985-12-02 12:00:00  1985-12-03 12:00:00  1985-12-04 12:00:00
  1985-12-05 12:00:00  1985-12-06 12:00:00  1985-12-07 12:00:00  1985-12-08 12:00:00
  1985-12-09 12:00:00  1985-12-10 12:00:00  1985-12-11 12:00:00  1985-12-12 12:00:00
  1985-12-13 12:00:00  1985-12-14 12:00:00  1985-12-15 12:00:00  1985-12-16 12:00:00
  1985-12-17 12:00:00  1985-12-18 12:00:00  1985-12-19 12:00:00  1985-12-20 12:00:00
  1985-12-21 12:00:00  1985-12-22 12:00:00  1985-12-23 12:00:00  1985-12-24 12:00:00
  1985-12-25 12:00:00  1985-12-26 12:00:00  1985-12-27 12:00:00  1985-12-28 12:00:00
  1985-12-29 12:00:00  1985-12-30 12:00:00  1985-12-31 12:00:00
cdo sinfo: Processed 1 variable over 5479 timesteps ( 0.13s )

-Karin

RE: Error (cdf_put_vara_float) : NetCDF: HDF error - Added by Beata Szabo-Takacs over 6 years ago

HI Karin,
My cdo version is:

cdo -V Climate Data Operators version 1.8.2 (http://mpimet.mpg.de/cdo) Compiled: by salek.m on fenix (x86_64-unknown-linux-gnu) Jun 23 2017 10:59:30 Compiler: gcc -std=gnu99 -g -O2 -fopenmp version: gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-4) Features: DATA PTHREADS OpenMP3 NC4/HDF5 OPeNDAP SSE2 Libraries: Filetypes: srv ext ieg grb1 nc1 nc2 nc4 nc4c CDI library version : 1.8.2 of Jun 23 2017 10:58:23 CGRIBEX library version : 1.7.6 of Dec 20 2016 19:55:24 NetCDF library version : 4.3.3.1 of Dec 10 2015 16:44:18 $ HDF5 library version : 1.8.12 SERVICE library version : 1.4.0 of Jun 23 2017 10:58:04 EXTRA library version : 1.4.0 of Jun 23 2017 10:57:57 IEG library version : 1.4.0 of Jun 23 2017 10:58:02 FILE library version : 1.8.3 of Jun 23 2017 10:57:58

RE: Error (cdf_put_vara_float) : NetCDF: HDF error - Added by Karin Meier-Fleischer over 6 years ago

I use a higher version of NetCDF and HDF5 but in my opinion the error message sounds more like a data problem.

  NetCDF library version : 4.4.1.1 of May 11 2017 17:19:25 $
    HDF5 library version : 1.10.1

Can you download some of the data files from the ESGF again and do the mergetime with these files?

-Karin

RE: Error (cdf_put_vara_float) : NetCDF: HDF error - Added by Beata Szabo-Takacs over 6 years ago

Dear Karin,
Thank you for your help. Meantime I solved the issue. The server where I use CDO was overladen. I deleted some large netCDF files from the server and after that I could merge again the MPI-ESM-LR-CLM4.8.17 netCDF files without error message.

Beata

(1-4/4)

Project

General

Profile

CDO