Project

General

Profile

netCDF4 problems

Added by Andreas Hilboll over 11 years ago

Hi,

I have some problems with cdo-1.5.8 which I compiled on my Ubuntu 12.04 system yesterday. This is my configuration:

$ cdo -V
Climate Data Operators version 1.5.8 (http://code.zmaw.de/projects/cdo)
Compiler: gcc -std=gnu99 -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security -fPIC -pthread
version: gcc (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3
with: PTHREADS Z JASPER UDUNITS2 PROJ.4
Compiled: by pbuilder on elara.iup.uni-bremen.de (x86_64-unknown-linux-gnu) Nov 1 2012 17:13:27
CDI library version : 1.5.8 of Nov 1 2012 17:13:23
GRIB_API library version : 1.9.9
netCDF library version : 4.1.1 of Nov 7 2011 11:35:16 $
HDF5 library version : 1.8.4
SERVICE library version : 1.3.1 of Nov 1 2012 17:13:21
EXTRA library version : 1.3.1 of Nov 1 2012 17:13:19
IEG library version : 1.3.1 of Nov 1 2012 17:13:20
FILE library version : 1.8.0 of Nov 1 2012 17:13:19

When I try to open a netCDF4 file, I get this error:

$ cdo info /mnt/data/tph/tph_2011.nc 
cdo info: Open failed on >/mnt/data/tph/tph_2011.nc<
Unsupported file type (library support not compiled in)

The netCDF4 file was created using the Python bindings for netCDF4:

ncFile = netCDF4.Dataset(start.strftime("/misc/decagon/data/tropopause/tph_%Y.nc"), 'w', clobber=True, format='NETCDF4')
tph_v = ncFile.createVariable('tropopause_altitude', 'f4', ('time', 'lon', 'lat',), zlib=True)

Any help resolving this issue is greatly appreciated ...

Cheers, Andreas.


Replies (36)

RE: netCDF4 problems - Added by Uwe Schulzweida over 11 years ago

Hi Andreas,

It seems that the netCDF library linked to CDO was not build with netCDF4 support.
Otherwise the with: line of "cdo -V" should contain the flag NC4.

Cheers, Uwe

RE: netCDF4 problems - Added by Andreas Hilboll over 11 years ago

Hi Uwe,

thanks for your answer. However, I'm using netcdf libraries as shipped with Ubuntu 12.04. This is version 4.1.1, and the configure flags do have --enable-netcdf4. So I believe that my netcdf library does indeed have netCDF4 support. Is there any way I can explicitly check for this?

Do you have any other ideas?

Cheers, Andreas.

RE: netCDF4 problems - Added by Uwe Schulzweida over 11 years ago

Hi Andreas,

Yes, you can check it with the netCDF tool nc-config:

nc-config --has-nc4
This tool will be installed with the netCDF library and is located in the bin directory of your netCDF installation.

Cheers, Uwe

RE: netCDF4 problems - Added by Andreas Hilboll over 11 years ago

Thanks, Uwe, NetCDF4 is indeed activated:

$ nc-config --has-nc4
yes

Do you have any idea what could be going wrong? I mean, I can read the file with ncdump, and I can access them with python-netcdf4. Only thing not working with them is the cdo I compiled =(

Thanks for your help,
Andreas.

RE: netCDF4 problems - Added by Uwe Schulzweida over 11 years ago

Sorry, I have no more ideas. Maybe the nc-config is not found during the configuration. Could you please attach the config.log file?

Cheers,
Uwe

RE: netCDF4 problems - Added by Andreas Hilboll over 11 years ago

Uwe, here's the config.log. Strange thing: In the file, it says

configure:18717: checking netcdf's nc4 support
configure:18723: result: yes

I'm lost.

Thanks for your help!
Andreas.

RE: netCDF4 problems - Added by Uwe Schulzweida over 11 years ago

Yes, netCDF4 support should be available in this case. Have you checked the output of "cdo -V". Could you please attach the netCDF file?

Cheers, Uwe

RE: netCDF4 problems - Added by Andreas Hilboll over 11 years ago

I posted the output of cdo -V in my original post (see above). The .nc file is available at www.iup.physik.uni-bremen.de/~hilboll/tph_2011.nc.

Thanks for your help!
Cheers, Andreas.

RE: netCDF4 problems - Added by Uwe Schulzweida over 11 years ago

Hi Andreas,

The result of cdo -V and the config.log doesn't fit together. Try to reconfigure CDO after a 'make distclean'.

Cheers, Uwe

RE: netCDF4 problems - Added by Andreas Hilboll over 11 years ago

Thanks a lot for your help, Uwe! I recompiled after a make distclean, and now everything works. Still not sure what happened here, but I'm happy everything's working for me now. Thanks again!

I'll post a note here on the Forum once my packages are uploaded to an Ubuntu PPA.

Cheers, Andreas.

RE: netCDF4 problems - Added by Sanita Dhaubanjar about 7 years ago

Hi,

I am facing a similar problem compiling cdo-1.8.1 for .nc4 files in my Mac. I used homebrew to compile cdo from the source. I have tried all the steps you mentioned here but I am unable to solve my issue. I am able to view my files using ncdump and ncview but not with cdo.

Here are my configurations:

$ nc-config --has-nc4
yes
$ cdo -V
Climate Data Operators version 1.8.1rc2 (http://mpimet.mpg.de/cdo)
Compiled: by sdhauban on Sanitas-MacBook-Pro.local (x86_64-apple-darwin13.4.0) Mar 31 2017 14:00:31
Compiler: gcc -g -O2 -D_THREAD_SAFE -pthread
version: Apple LLVM version 6.0 (clang-600.0.57) (based on LLVM 3.5svn)
Features: DATA PTHREADS HDF5 SSE3
Libraries: HDF5/1.10.0
Filetypes: srv ext ieg grb1 nc1
CDI library version : 1.8.1 of Mar 31 2017 13:59:23
CGRIBEX library version : 1.7.6 of Dec 20 2016 19:55:24
NetCDF library version : 4.4.1.1 of Mar 31 2017 11:34:28 $
SERVICE library version : 1.4.0 of Mar 31 2017 13:59:08
EXTRA library version : 1.4.0 of Mar 31 2017 13:59:01
IEG library version : 1.4.0 of Mar 31 2017 13:59:05
FILE library version : 1.8.2 of Mar 31 2017 13:59:02

This is the error I get when I try to use cdo:
@$ cdo info Desktop/TRMM2/3B42_Daily.20161104.7.nc4

cdo info: Open failed on >Desktop/TRMM2/3B42_Daily.20161104.7.nc4<
Unsupported file type (library support not compiled in)
CDO was build with a NetCDF version which doesn't support NetCDF4 data!@

Attached is the .nc4 file as well.

Any suggestions?
Thank for the help!

Andreas Hilboll wrote:

Hi,

I have some problems with cdo-1.5.8 which I compiled on my Ubuntu 12.04 system yesterday. This is my configuration:

$ cdo -V
Climate Data Operators version 1.5.8 (http://code.zmaw.de/projects/cdo)
Compiler: gcc -std=gnu99 -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security -fPIC -pthread
version: gcc (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3
with: PTHREADS Z JASPER UDUNITS2 PROJ.4
Compiled: by pbuilder on elara.iup.uni-bremen.de (x86_64-unknown-linux-gnu) Nov 1 2012 17:13:27
CDI library version : 1.5.8 of Nov 1 2012 17:13:23
GRIB_API library version : 1.9.9
netCDF library version : 4.1.1 of Nov 7 2011 11:35:16 $
HDF5 library version : 1.8.4
SERVICE library version : 1.3.1 of Nov 1 2012 17:13:21
EXTRA library version : 1.3.1 of Nov 1 2012 17:13:19
IEG library version : 1.3.1 of Nov 1 2012 17:13:20
FILE library version : 1.8.0 of Nov 1 2012 17:13:19

When I try to open a netCDF4 file, I get this error:

$ cdo info /mnt/data/tph/tph_2011.nc
cdo info: Open failed on >/mnt/data/tph/tph_2011.nc<
Unsupported file type (library support not compiled in)

The netCDF4 file was created using the Python bindings for netCDF4:

ncFile = netCDF4.Dataset(start.strftime("/misc/decagon/data/tropopause/tph_%Y.nc"), 'w', clobber=True, format='NETCDF4')
tph_v = ncFile.createVariable('tropopause_altitude', 'f4', ('time', 'lon', 'lat',), zlib=True)

Any help resolving this issue is greatly appreciated ...

Cheers, Andreas.

RE: netCDF4 problems - Added by Ralf Mueller about 7 years ago

have you tried macports??

RE: netCDF4 problems - Added by Ralf Mueller about 7 years ago

the point is, I don't know waht homebrew is doing. nc-config could be part of a different netcdf installation. your cdo binary does not have netcdf4 support build in (at least that's clear)

RE: netCDF4 problems - Added by Sanita Dhaubanjar about 7 years ago

I haven't tried macports but I have followed Uwe's suggestions to "make distclean" and then I compiled CDO manually by downloading the package from the website.

RE: netCDF4 problems - Added by Ralf Mueller about 7 years ago

ok, then upload your config.log file, pls

RE: netCDF4 problems - Added by Sanita Dhaubanjar about 7 years ago

where can i find the config.log file on a Mac OSX?

RE: netCDF4 problems - Added by Ralf Mueller about 7 years ago

in the directory, where you called CDOs configure

RE: netCDF4 problems - Added by Sanita Dhaubanjar about 7 years ago

I had already deleted the directory where I ran CDO last time. So I tried to follow my steps again. Attached is the config.log file from this.

RE: netCDF4 problems - Added by Ralf Mueller about 7 years ago

configure:19185: checking nc-config script
configure:19189: checking for nc-config
configure:19219: result: no

configure does not find you nc-config. where is it? could you return the output of

 type nc-config
or
 where nc-config

RE: netCDF4 problems - Added by Sanita Dhaubanjar about 7 years ago


$ type nc-config
nc-config is hashed (/usr/local/bin/nc-config)
$ where nc-config
-bash: where: command not found

RE: netCDF4 problems - Added by Ralf Mueller about 7 years ago

BTW: your compiler is from 09.2014. Could you update?

RE: netCDF4 problems - Added by Sanita Dhaubanjar about 7 years ago

Sorry! I have not used command line based tools before. How do update that?

RE: netCDF4 problems - Added by Sanita Dhaubanjar about 7 years ago

I just noticed I did not put the right addresses for netcdf and hdf when calling ./configure. Maybe that is the problem? I am compiling again.

RE: netCDF4 problems - Added by Ralf Mueller about 7 years ago

please try

./configure --with-netcdf=/usr/local

in your configure call, you're refering to a netcdf installation below /usr while you real netcdf (the one, which has nc-config) is below /usr/local

RE: netCDF4 problems - Added by Sanita Dhaubanjar about 7 years ago

I tried the following:

$./configure --with-netcdf=/usr/local/Cellar/netcdf/4.4.1.1_4 --with-hdf5=/usr/local/Cellar/hdf5/1.10.0-patch1
$make
$make install

Attached is the config.log file again. Now it is giving me many error messages, bottom ones are:

clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [cdi] Error 1
make[1]: *** [install-recursive] Error 1
make: *** [install-recursive] Error 1

(1-25/36)