Compilation issues grib_api

Added by Brendan DeTracey almost 10 years ago

Hi,
I am trying to compile 1.8.0rc4 under 32 bit cygwin and having issues with grib api (Yes, I know there is a 64 bit cygwin executable, but for now I am stuck in 32 bit cygwin land). The last version I successfully compiled was 1.6.8. Since then the --with-jasper configure option has been removed , which might be contributing to my problem. First attempt was to configure exactly as I did for 1.6.8, but excluding the --with-jasper option.

$ ./configure --with-netcdf --with-hdf5 --with-proj --with-szlib=/usr/local --with-grib_api=/usr/local/grib_api --with-fftw3 --with-curl --with-udunits --with-libxml2
.
.
configure: error: Could not link to grib_api library

The log is attached below as config_default.log

Next I tried including the jasper library. On cygwin the jasper library is installed in /lib, the includes in /usr/include/jasper.

LDFLAGS="-ljasper" CPPFLAGS="-I/usr/include/jasper" ./configure --with-netcdf --with-hdf5 --with-proj --with-szlib=/usr/local --with-grib_api=/usr/local/grib_api --with-fftw3 --with-curl --with-udunits --with-libxml2
.
.
configure: error: Could not link to grib_api library

The log is attached below as config_jasper.log

(grib api version is 1.13.1)

Download all files

config_default.log (87.5 KB) config_default.log
config_jasper.log (90 KB) config_jasper.log

Replies (9)

RE: Compilation issues grib_api - Added by Ralf Mueller almost 10 years ago

I may try

LDFLAGS="-L/lib -ljasper"

or use the LIBS environment variable instead

hth
ralf

RE: Compilation issues grib_api - Added by Brendan DeTracey almost 10 years ago

LDFLAGS did not work, but LIBS did. Strange. So my working configure line was:

LIBS="-ljasper" CPPFLAGS="-I/usr/include/jasper" ./configure --with-netcdf --with-hdf5 --with-proj --with-szlib=/usr/local --with-grib_api=/usr/local/grib_api --with-fftw3 --with-curl --with-udunits --with-libxml2

and I have attached the log for the successful configure as config.log

Thanks Ralf.
Now I must try those fancy new cdo colours!

config.log (114 KB) config.log

RE: Compilation issues grib_api - Added by Brendan DeTracey almost 10 years ago

Spoke too soon. When I make I get:

*** Warning: This system can not link to static lib archive /usr/local/grib_api/lib/libgrib_api.la.
*** I have the capability to make that library automatically link in when
*** you link to this library.  But I can only do this if you have a
*** shared version of the library, which you do not appear to have.

Don't remember having this problem before. I tried adding

--enable-all-static

to configure but the warning persists.

RE: Compilation issues grib_api - Added by Brendan DeTracey almost 10 years ago

Okay the following configure did successfully make:

LIBS="-ljasper" CPPFLAGS="-I/usr/include/jasper" ./configure --with-netcdf --with-hdf5 --with-proj --with-szlib=/usr/local --with-grib_api=/usr/local/grib_api --with-fftw3 --with-curl --with-udunits --with-libxml2 --enable-all-static --disable-shared

But make check failed one test.

.
.
ok 7 - chaining set 1 with netCDF2
PASS: tsformat.test 7 - chaining set 1 with netCDF2
Running test: 8 - chaining set 1 with netCDF4
cdo setgrid: Processed 2160 values from 3 variables over 5 timesteps ( 0.02s )
cdo fldmin: Started child process "timmean -select,code=130 ifile7236 (pipe1.1)".
cdo(2) timmean: Started child process "select,code=130 ifile7236 (pipe2.1)".
./tsformat.test: line 59:  6208 Segmentation fault      (core dumped) $CDOCOMMAND
cdo fldmin: Started child process "timmean -select,code=130 ifile7236 (pipe1.1)".
cdo(2) timmean: Started child process "select,code=130 ifile7236 (pipe2.1)".
cdo fldmin: Started child process "timmean -select,code=130 ifile7236 (pipe1.1)".
cdo(2) timmean: Started child process "select,code=130 ifile7236 (pipe2.1)".
cdo fldmin: Started child process "timmean -select,code=130 ifile7236 (pipe1.1)".
cdo(2) timmean: Started child process "select,code=130 ifile7236 (pipe2.1)".
./tsformat.test: line 59:  6220 Segmentation fault      (core dumped) $CDOCOMMAND
cdo fldmin: Started child process "timmean -select,code=130 ifile7236 (pipe1.1)".
cdo(2) timmean: Started child process "select,code=130 ifile7236 (pipe2.1)".
cdo(3) select: Processed 1440 values from 3 variables over 5 timesteps ( 0.03s )
cdo(2) timmean: Processed 1440 values from 1 variable over 5 timesteps ( 0.03s )
cdo fldmin: Processed 288 values from 1 variable over 1 timestep ( 0.03s )
../src/cdo -fldmin -timmean -select,code=130 ../test/data/pl_data(nc4) thread8_res
cdo diff: Started child process "selcode,130 ../test/data/tsformat1_ref (pipe1.1)".
cdo(2) selcode: Processed 4 values from 3 variables over 1 timestep ( 0.01s )
cdo diff: Processed 8 values from 2 variables over 2 timesteps ( 0.01s )
not ok 8 - chaining set 1 with netCDF4
FAIL: tsformat.test 8 - chaining set 1 with netCDF4

My nc-config:

$ nc-config --all

This netCDF 4.3.3.1 has been built with the following features:

  --cc        -> gcc
  --cflags    ->  -I/usr/include
  --libs      -> -L/usr/lib -lnetcdf

  --has-c++   -> no
  --cxx       ->
  --has-c++4  -> yes
  --cxx4      -> g++

  --fc        -> gfortran
  --fflags    -> -I/usr/include
  --flibs     -> -L/usr/lib -lnetcdff -lnetcdf -lnetcdf
  --has-f90   -> no

  --has-dap   -> yes
  --has-nc2   -> yes
  --has-nc4   -> yes
  --has-hdf5  -> yes
  --has-hdf4  -> no
  --has-pnetcdf-> no

  --prefix    -> /usr
  --includedir-> /usr/include
  --version   -> netCDF 4.3.3.1

RE: Compilation issues grib_api - Added by Brendan DeTracey almost 10 years ago

Forgot to attach core dump. Not very useful.

Exception: STATUS_ACCESS_VIOLATION at eip=6588894A
eax=800CD258 ebx=FFD7CBCC ecx=00000000 edx=00000001 esi=800D10C8 edi=04000003
ebp=00000000 esp=FFD7CB70 program=C:\cygwin\usr\src\cdo-1.8.0rc4\src\cdo.exe, pid 6220, thread unknown (0x216C)
cs=0023 ds=002B es=002B fs=0053 gs=002B ss=002B
Stack trace:
Frame     Function  Args
End of stack trace

I think there is a race condition when the operators are chained. When I run fldmin alone it does not crash, but when chained as in the test it fails at random.

$ src/cdo -O fldmin -timmean -select,code=130 test0.nc test1.nc
cdo fldmin: Started child process "timmean -select,code=130 test0.nc (pipe1.1)".
cdo(2) timmean: Started child process "select,code=130 test0.nc (pipe2.1)".
Segmentation fault (core dumped)
$ src/cdo -O fldmin -timmean -select,code=130 test0.nc test1.nc
cdo fldmin: Started child process "timmean -select,code=130 test0.nc (pipe1.1)".
cdo(2) timmean: Started child process "select,code=130 test0.nc (pipe2.1)".
Segmentation fault (core dumped)
$ src/cdo -O fldmin -timmean -select,code=130 test0.nc test1.nc
cdo fldmin: Started child process "timmean -select,code=130 test0.nc (pipe1.1)".
cdo(2) timmean: Started child process "select,code=130 test0.nc (pipe2.1)".
cdo(3) select: Processed 1440 values from 3 variables over 5 timesteps ( 0.02s )
cdo(2) timmean: Processed 1440 values from 1 variable over 5 timesteps ( 0.02s )
cdo fldmin: Processed 288 values from 1 variable over 1 timestep ( 0.02s )

RE: Compilation issues grib_api - Added by Ralf Mueller almost 10 years ago

Brendan DeTracey wrote:

Okay the following configure did successfully make:
[...]
But make check failed one test.
[...]

My nc-config:
[...]

netcdf is not the problem, it is the hdf5 used by netcdf. netcdf4 uses hdf5 as IO backend, but by default hdf5 is installed non-thread-safe. you can

install hdf5 in a thread safe mode, but here you will have to --enable-unsuported
use the CDO option for serialized IO: -L,e.g.
```
make check CDO='./src/cdo -L'
```

RE: Compilation issues grib_api - Added by Ralf Mueller almost 10 years ago

some info about it directly from hdf5 developers: https://support.hdfgroup.org/hdf5-quest.html#tsafe

RE: Compilation issues grib_api - Added by Brendan DeTracey almost 10 years ago

Thanks once again Ralf. All tests passed with flying colours. No longer bored, sad and lonely. Will cdo always require thread safe hdf?

RE: Compilation issues grib_api - Added by Ralf Mueller almost 10 years ago

Brendan DeTracey wrote:

Thanks once again Ralf. All tests passed with flying colours. No longer bored, sad and lonely.

GREAT!!

Will cdo always require thread safe hdf?

in essence: yes.
reason: you can easily write down operation chains, that read from files in a parallel way. operators chains are already threaded unless you disable it explicitely (use option ... no, I won't tell you - CDO is great the way it is)

hence, just because CDO does not know, how netcdf/hdf5 is installed on the user's machine, the '-L' is needed. if not sure, just use it. if hdf5 is thread-safe, it's not needed.

(1-9/9)

Project

General

Profile

CDO