cdo info failing - Error (cdfGetSliceSlapDescription) : Internal error, variable tas has an unsupported array structure!
Added by Pierre-Antoine Bretonnière over 5 years ago
Hello,
I'm having issues with the command cdo info for the file described below (it was too big to be uploaded). The other cdo commands seem to work fine.
I get the following error:
pbretonn@bscearth319:/esarchive/exp/CMIP5/decadal/miroc5/cmip5-decadal_i1p1/monthly_mean/tas$ cdo info tas_20110101.nc
-1 : Date Time Level Gridsize Miss : Minimum Mean Maximum : Parameter ID
Error (cdfGetSliceSlapDescription) : Internal error, variable tas has an unsupported array structure!
I'm using CDO/1.9.0 compiled with these options:
Climate Data Operators version 1.9.0 (http://mpimet.mpg.de/cdo)
Compiled: by eadmin on moore (x86_64-unknown-linux-gnu) Aug 8 2017 18:16:24
CXX Compiler: mpicxx -std=gnu++11 -O3 -march=native -fPIC -fopenmp
CXX version : unknown
C Compiler: mpicc -std=gnu99 -O3 -march=native -fPIC -fopenmp
C version : unknown
Features: DATA PTHREADS OpenMP4 HDF5 NC4/HDF5/threadsafe OPeNDAP SSE4_2
Libraries: HDF5/1.8.14
Filetypes: srv ext ieg grb1 grb2 nc1 nc2 nc4 nc4c
CDI library version : 1.9.0 of Aug 8 2017 18:14:20
CGRIBEX library version : 1.8.1 of Jul 12 2017 18:11:37
GRIB_API library version : 1.13.0
NetCDF library version : 4.3.3.1 of Feb 10 2016 12:09:04 $
HDF5 library version : 1.8.14 threadsafe
SERVICE library version : 1.4.0 of Aug 8 2017 18:13:57
EXTRA library version : 1.4.0 of Aug 8 2017 18:13:51
IEG library version : 1.4.0 of Aug 8 2017 18:13:54
FILE library version : 1.8.3 of Aug 8 2017 18:13:51
I've tried with other versions of cdo that we have installed locally and it seems that we can make the cdo info works for versions 1.5.3, 1.6.3 and 1.6.9 but fails for versions 1.7.2 and 1.8.2 and 1.9.0 (all compiled with different options and library versions).
For example cdo 1.6.3 has these options:
Climate Data Operators version 1.6.3 (http://code.zmaw.de/projects/cdo)
Compiled: by eadmin on moore (x86_64-unknown-linux-gnu) Oct 22 2015 10:39:09
Compiler: mpicc -std=gnu99 -fPIC -O3 -march=native
version: unknown
Features: PTHREADS NC4 OPeNDAP Z
Libraries:
Filetypes: srv ext ieg grb nc nc2 nc4 nc4c
CDI library version : 1.6.3 of Oct 22 2015 10:39:00
CGRIBEX library version : 1.6.3 of Jan 8 2014 19:55:18
netCDF library version : 4.3.3.1 of Feb 10 2016 12:09:04 $
HDF5 library version : 1.8.14
SERVICE library version : 1.3.1 of Oct 22 2015 10:38:57
EXTRA library version : 1.3.1 of Oct 22 2015 10:38:56
IEG library version : 1.3.1 of Oct 22 2015 10:38:57
FILE library version : 1.8.2 of Oct 22 2015 10:38:56
The only thing I could find common to all the non working versions is that they are using HDF5 threadsafe whereas the working versions use "normal" HDF (even if all the versions are compiled with HDF5 1.8.14.
Thank you!
pbretonn@bsceslogin01:/esarchive/exp/CMIP5/decadal/miroc5/cmip5-decadal_i1p1/monthly_mean/tas$ ncdump -k tas_20110101.nc
classic
pbretonn@bsceslogin01:/esarchive/exp/CMIP5/decadal/miroc5/cmip5-decadal_i1p1/monthly_mean/tas$ ncdump -h tas_20110101.nc
netcdf tas_20110101 {
dimensions:
latitude = 128 ;
longitude = 256 ;
time = UNLIMITED ; // (120 currently)
ensemble = 6 ;
time_bnd = 2 ;
variables:
double latitude(latitude) ;
latitude:bounds = "lat_bnds" ;
latitude:units = "degrees_north" ;
latitude:axis = "Y" ;
latitude:long_name = "latitude" ;
latitude:standard_name = "latitude" ;
double longitude(longitude) ;
longitude:bounds = "lon_bnds" ;
longitude:units = "degrees_east" ;
longitude:axis = "X" ;
longitude:long_name = "longitude" ;
longitude:standard_name = "longitude" ;
float tas(time, ensemble, latitude, longitude) ;
tas:standard_name = "air_temperature" ;
tas:long_name = "Near-Surface Air Temperature" ;
tas:units = "K" ;
tas:original_name = "T2" ;
tas:cell_methods = "time: mean" ;
tas:cell_measures = "area: areacella" ;
tas:history = "2011-09-26T05:22:29Z altered by CMOR: Treated scalar dimension: \'height\'. 2011-09-26T05:22:29Z altered by CMOR: replaced missing value flag (-999) with standard missing value (1e+20). 2011-09-26T05:22:29Z altered by CMOR: Inverted axis: lat." ;
tas:coordinates = "height" ;
tas:missing_value = 1.e+20f ;
tas:_FillValue = 1.e+20f ;
tas:associated_files = "baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_atmos_fx_MIROC5_decadal2010_r0i0p0.nc areacella: areacella_fx_MIROC5_decadal2010_r0i0p0.nc" ;
double time(time) ;
time:bounds = "time_bnds" ;
time:units = "days since 2011-1-1" ;
time:calendar = "gregorian" ;
time:axis = "T" ;
time:long_name = "time" ;
time:standard_name = "time" ;
double height ;
height:units = "m" ;
height:axis = "Z" ;
height:positive = "up" ;
height:long_name = "height" ;
height:standard_name = "height" ;
double lat_bnds(latitude, time_bnd) ;
double lon_bnds(longitude, time_bnd) ;
double time_bnds(time, time_bnd) ;
// global attributes:
:history = "Mon Mar 12 17:01:06 2012: ncpdq -a -latitude ensemble_tas2010_tris.nc ensemble_tas2010_four.nc\n",
"Mon Mar 12 17:01:06 2012: ncrename -O -d bnds,time_bnd ensemble_tas2010_tris.nc ensemble_tas2010_tris.nc\n",
"Mon Mar 12 17:01:05 2012: ncks -A -v height,lon_bnds,lat_bnds,time_bnds out.nc ensemble_tas2010_tris.nc\n",
"Mon Mar 12 17:01:04 2012: ncks -O -x -v height,lon_bnds,lat_bnds,time_bnds ensemble_tas2010_tris.nc ensemble_tas2010_tris.nc\n",
"Mon Mar 12 17:01:04 2012: ncrename -d lat,latitude -d lon,longitude -v lat,latitude -v lon,longitude ensemble_tas2010_bis.nc ensemble_tas2010_tris.nc\n",
"Mon Mar 12 17:01:02 2012: ncpdq -O -a time,ensemble ensemble_tas2010.nc ensemble_tas2010_bis.nc\n",
"Mon Mar 12 17:00:55 2012: ncecat -M -u ensemble tas_Amon_MIROC5_decadal2010_r1i1p1_201101-202012.nc tas_Amon_MIROC5_decadal2010_r2i1p1_201101-202012.nc tas_Amon_MIROC5_decadal2010_r3i1p1_201101-202012.nc tas_Amon_MIROC5_decadal2010_r4i1p1_201101-202012.nc tas_Amon_MIROC5_decadal2010_r5i1p1_201101-202012.nc tas_Amon_MIROC5_decadal2010_r6i1p1_201101-202012.nc ensemble_tas2010.nc" ;
:nco_openmp_thread_number = 1 ;
:NCO = "4.0.8" ;
Replies (12)
RE: cdo info failing - Error (cdfGetSliceSlapDescription) : Internal error, variable tas has an unsupported array structure! - Added by Pierre-Antoine Bretonnière over 5 years ago
Hi,
It seems that we have the same issue with fldmean. And the threadsafe compilation doesn't seem to be the reason of the failure, we've tried other combinations of compilation and the issue appears when we go from 1.6.* to 1.7.*
RE: cdo info failing - Error (cdfGetSliceSlapDescription) : Internal error, variable tas has an unsupported array structure! - Added by Karin Meier-Fleischer over 5 years ago
Hi,
I just saw that nobody has answered your question. I'm not an expert in how to install CDO but I would recommend to install only the current version 1.9.5.
The C Compiler, CXX Compiler and Features lists in the info below are different to yours especially -D_THREAD_SAFE -pthread is missing.
cdo -V
Climate Data Operators version 1.9.5 (http://mpimet.mpg.de/cdo)
System: x86_64-apple-darwin16.7.0
CXX Compiler: /usr/bin/clang++ -std=gnu++11 -pipe -Os -stdlib=libc++ -arch x86_64 -D_THREAD_SAFE -pthread
CXX version : unknown
C Compiler: /usr/bin/clang -pipe -Os -arch x86_64 -D_THREAD_SAFE -pthread -D_THREAD_SAFE -D_THREAD_SAFE -pthread
C version : unknown
F77 Compiler: -pipe -Os
Features: 16GB 8threads C++11 DATA PTHREADS HDF5 NC4/HDF5 OPeNDAP SZ UDUNITS2 PROJ.4 XML2 MAGICS CURL FFTW3 SSE4_1
Libraries: HDF5/1.10.2 proj/5.1 xml2/2.9.7 curl/7.61.1
Filetypes: srv ext ieg grb1 grb2 nc1 nc2 nc4 nc4c nc5
CDI library version : 1.9.5
GRIB_API library version : 2.8.2
NetCDF library version : 4.6.1 of Aug 12 2018 14:32:21 $
HDF5 library version : 1.10.2
EXSE library version : 1.4.0
FILE library version : 1.8.3
Maybe this can help a little bit.
-Karin
RE: cdo info failing - Error (cdfGetSliceSlapDescription) : Internal error, variable tas has an unsupported array structure! - Added by Ralf Mueller over 5 years ago
I hdf5-threadsafty is a possible issue, you can add the option -L to your call. This serializes any IO operation on a stream and should help with non-threadsafe hdf5 installations
hth
ralf
RE: cdo info failing - Error (cdfGetSliceSlapDescription) : Internal error, variable tas has an unsupported array structure! - Added by Pierre-Antoine Bretonnière over 5 years ago
Hi,
Thanks for the replies. I've tried with the latest version and I still have the same issue. The "-L" option doesn't change anything (I'm doing only one operation anyway).
I've uploaded my file here (https://drive.google.com/file/d/1GL7Y403Tuf8vudACmDwvMFm-FI7H9x5m/view?usp=sharing) to see if someone can reproduce my error.
Thanks again for your help,
Best regards,
Pierre-Antoine
RE: cdo info failing - Error (cdfGetSliceSlapDescription) : Internal error, variable tas has an unsupported array structure! - Added by Karin Meier-Fleischer over 5 years ago
I've got the same error so the file is the problem. I'll have a look at the file.
cdo info tas_20110101.nc
-1 : Date Time Level Gridsize Miss : Minimum Mean Maximum : Parameter ID
Error (cdfGetSliceSlapDescription): Internal error, variable tas has an unsupported array structure!
-Karin
RE: cdo info failing - Error (cdfGetSliceSlapDescription) : Internal error, variable tas has an unsupported array structure! - Added by Karin Meier-Fleischer over 5 years ago
The attribute coordinates of the variable tas is the problem. Delete it from the netCDF file and cdo info works fine.
ncatted -O -a coordinates,tas,d,, tas_20110101.nc
cdo info tas_20110101.nc
-1 : Date Time Level Gridsize Miss : Minimum Mean Maximum : Parameter ID
1 : 2011-01-16 12:00:00 1 32768 0 : 234.58 277.52 309.37 : -1
2 : 2011-01-16 12:00:00 2 32768 0 : 222.56 277.85 309.47 : -1
3 : 2011-01-16 12:00:00 3 32768 0 : 234.83 277.59 308.60 : -1
4 : 2011-01-16 12:00:00 4 32768 0 : 231.43 277.30 309.67 : -1
5 : 2011-01-16 12:00:00 5 32768 0 : 228.88 277.65 307.12 : -1
6 : 2011-01-16 12:00:00 6 32768 0 : 228.02 277.71 308.00 : -1
7 : 2011-02-15 00:00:00 1 32768 0 : 227.90 277.03 308.79 : -1
8 : 2011-02-15 00:00:00 2 32768 0 : 230.90 277.26 306.51 : -1
9 : 2011-02-15 00:00:00 3 32768 0 : 228.74 276.76 307.27 : -1
10 : 2011-02-15 00:00:00 4 32768 0 : 229.84 276.79 308.75 : -1
...
-Karin
RE: cdo info failing - Error (cdfGetSliceSlapDescription) : Internal error, variable tas has an unsupported array structure! - Added by Pierre-Antoine Bretonnière over 5 years ago
Thank you very much!!! It tried removing the coordinate and it indeed works!
However, I don't understand why it didn't work in the first place. Doesn't cdo allow to have a coordinates attribute? That's quite standard in netcdf. Or is it because the variable height (which is the coordinate associated to tas) is a scalar and not a (1D) array?
RE: cdo info failing - Error (cdfGetSliceSlapDescription) : Internal error, variable tas has an unsupported array structure! - Added by Karin Meier-Fleischer over 5 years ago
In my opinion CDO allows to have the coordinates latitude and longitude as coordinates attribute. If tas:coordinates would be e.g. "lat lon" CDO will work fine.
RE: cdo info failing - Error (cdfGetSliceSlapDescription) : Internal error, variable tas has an unsupported array structure! - Added by Pierre-Antoine Bretonnière over 5 years ago
I think having the possibility to have other coordinates than lat or lon would be quite useful.
Do you know if latitude or longitude work too (instead of lat/lon)? For example in the CMIP6 CMOR conventions, some variables have coordinates: "rlon rlat". Is there a list of authorized coordinates?
Thank you.
RE: cdo info failing - Error (cdfGetSliceSlapDescription) : Internal error, variable tas has an unsupported array structure! - Added by Karin Meier-Fleischer over 5 years ago
You can set the attribute coordinates to "latitude longitude" or "rlat rlon".
RE: cdo info failing - Error (cdfGetSliceSlapDescription) : Internal error, variable tas has an unsupported array structure! - Added by Uwe Schulzweida over 5 years ago
- 1 dimension is for the time.
- 1-2 dimesions are for the horizontal grid, this depends on the type of the grid.
- the last available dimension is reserved for z:level
If z:level is not found and one extra dimension is left, then this extra dimension is treated like z:level. For your example data this is the dimension: ensemble.
Since CDO is supporting scalar coordinates, one dimension is reserved for the scalar coordinate.
Your example file contains the scalar coordinate: height. In this case there is no dimension left for the extra dimension: exsemble.
RE: cdo info failing - Error (cdfGetSliceSlapDescription) : Internal error, variable tas has an unsupported array structure! - Added by Uwe Schulzweida over 5 years ago
We will improve the error message in the next CDO release!