Compilation issues grib_api

Added by Brendan DeTracey over 7 years ago

Hi,
I am trying to compile 1.8.0rc4 under 32 bit cygwin and having issues with grib api (Yes, I know there is a 64 bit cygwin executable, but for now I am stuck in 32 bit cygwin land). The last version I successfully compiled was 1.6.8. Since then the --with-jasper configure option has been removed , which might be contributing to my problem. First attempt was to configure exactly as I did for 1.6.8, but excluding the --with-jasper option.

$ ./configure --with-netcdf --with-hdf5 --with-proj --with-szlib=/usr/local --with-grib_api=/usr/local/grib_api --with-fftw3 --with-curl --with-udunits --with-libxml2
.
.
configure: error: Could not link to grib_api library

The log is attached below as config_default.log

Next I tried including the jasper library. On cygwin the jasper library is installed in /lib, the includes in /usr/include/jasper.

LDFLAGS="-ljasper" CPPFLAGS="-I/usr/include/jasper" ./configure --with-netcdf --with-hdf5 --with-proj --with-szlib=/usr/local --with-grib_api=/usr/local/grib_api --with-fftw3 --with-curl --with-udunits --with-libxml2
.
.
configure: error: Could not link to grib_api library

The log is attached below as config_jasper.log

(grib api version is 1.13.1)

Download all files

config_default.log (87.5 KB) config_default.log
config_jasper.log (90 KB) config_jasper.log

Replies (9)

RE: Compilation issues grib_api - Added by Ralf Mueller over 7 years ago

I may try

LDFLAGS="-L/lib -ljasper"

or use the LIBS environment variable instead

hth
ralf

RE: Compilation issues grib_api - Added by Brendan DeTracey over 7 years ago

LDFLAGS did not work, but LIBS did. Strange. So my working configure line was:

LIBS="-ljasper" CPPFLAGS="-I/usr/include/jasper" ./configure --with-netcdf --with-hdf5 --with-proj --with-szlib=/usr/local --with-grib_api=/usr/local/grib_api --with-fftw3 --with-curl --with-udunits --with-libxml2

and I have attached the log for the successful configure as config.log

Thanks Ralf.
Now I must try those fancy new cdo colours!

config.log (114 KB) config.log

RE: Compilation issues grib_api - Added by Brendan DeTracey over 7 years ago

Spoke too soon. When I make I get:

*** Warning: This system can not link to static lib archive /usr/local/grib_api/lib/libgrib_api.la.
*** I have the capability to make that library automatically link in when
*** you link to this library.  But I can only do this if you have a
*** shared version of the library, which you do not appear to have.

Don't remember having this problem before. I tried adding

--enable-all-static

to configure but the warning persists.

RE: Compilation issues grib_api - Added by Brendan DeTracey over 7 years ago

Okay the following configure did successfully make:

LIBS="-ljasper" CPPFLAGS="-I/usr/include/jasper" ./configure --with-netcdf --with-hdf5 --with-proj --with-szlib=/usr/local --with-grib_api=/usr/local/grib_api --with-fftw3 --with-curl --with-udunits --with-libxml2 --enable-all-static --disable-shared

But make check failed one test.

.
.
ok 7 - chaining set 1 with netCDF2
PASS: tsformat.test 7 - chaining set 1 with netCDF2
Running test: 8 - chaining set 1 with netCDF4
cdo setgrid: Processed 2160 values from 3 variables over 5 timesteps ( 0.02s )
cdo fldmin: Started child process "timmean -select,code=130 ifile7236 (pipe1.1)".
cdo(2) timmean: Started child process "select,code=130 ifile7236 (pipe2.1)".
./tsformat.test: line 59:  6208 Segmentation fault      (core dumped) $CDOCOMMAND
cdo fldmin: Started child process "timmean -select,code=130 ifile7236 (pipe1.1)".
cdo(2) timmean: Started child process "select,code=130 ifile7236 (pipe2.1)".
cdo fldmin: Started child process "timmean -select,code=130 ifile7236 (pipe1.1)".
cdo(2) timmean: Started child process "select,code=130 ifile7236 (pipe2.1)".
cdo fldmin: Started child process "timmean -select,code=130 ifile7236 (pipe1.1)".
cdo(2) timmean: Started child process "select,code=130 ifile7236 (pipe2.1)".
./tsformat.test: line 59:  6220 Segmentation fault      (core dumped) $CDOCOMMAND
cdo fldmin: Started child process "timmean -select,code=130 ifile7236 (pipe1.1)".
cdo(2) timmean: Started child process "select,code=130 ifile7236 (pipe2.1)".
cdo(3) select: Processed 1440 values from 3 variables over 5 timesteps ( 0.03s )
cdo(2) timmean: Processed 1440 values from 1 variable over 5 timesteps ( 0.03s )
cdo fldmin: Processed 288 values from 1 variable over 1 timestep ( 0.03s )
../src/cdo -fldmin -timmean -select,code=130 ../test/data/pl_data(nc4) thread8_res
cdo diff: Started child process "selcode,130 ../test/data/tsformat1_ref (pipe1.1)".
cdo(2) selcode: Processed 4 values from 3 variables over 1 timestep ( 0.01s )
cdo diff: Processed 8 values from 2 variables over 2 timesteps ( 0.01s )
not ok 8 - chaining set 1 with netCDF4
FAIL: tsformat.test 8 - chaining set 1 with netCDF4

My nc-config:

$ nc-config --all

This netCDF 4.3.3.1 has been built with the following features:

  --cc        -> gcc
  --cflags    ->  -I/usr/include
  --libs      -> -L/usr/lib -lnetcdf

  --has-c++   -> no
  --cxx       ->
  --has-c++4  -> yes
  --cxx4      -> g++

  --fc        -> gfortran
  --fflags    -> -I/usr/include
  --flibs     -> -L/usr/lib -lnetcdff -lnetcdf -lnetcdf
  --has-f90   -> no

  --has-dap   -> yes
  --has-nc2   -> yes
  --has-nc4   -> yes
  --has-hdf5  -> yes
  --has-hdf4  -> no
  --has-pnetcdf-> no

  --prefix    -> /usr
  --includedir-> /usr/include
  --version   -> netCDF 4.3.3.1

RE: Compilation issues grib_api - Added by Brendan DeTracey over 7 years ago

Forgot to attach core dump. Not very useful.

Exception: STATUS_ACCESS_VIOLATION at eip=6588894A
eax=800CD258 ebx=FFD7CBCC ecx=00000000 edx=00000001 esi=800D10C8 edi=04000003
ebp=00000000 esp=FFD7CB70 program=C:\cygwin\usr\src\cdo-1.8.0rc4\src\cdo.exe, pid 6220, thread unknown (0x216C)
cs=0023 ds=002B es=002B fs=0053 gs=002B ss=002B
Stack trace:
Frame     Function  Args
End of stack trace

I think there is a race condition when the operators are chained. When I run fldmin alone it does not crash, but when chained as in the test it fails at random.

$ src/cdo -O fldmin -timmean -select,code=130 test0.nc test1.nc
cdo fldmin: Started child process "timmean -select,code=130 test0.nc (pipe1.1)".
cdo(2) timmean: Started child process "select,code=130 test0.nc (pipe2.1)".
Segmentation fault (core dumped)
$ src/cdo -O fldmin -timmean -select,code=130 test0.nc test1.nc
cdo fldmin: Started child process "timmean -select,code=130 test0.nc (pipe1.1)".
cdo(2) timmean: Started child process "select,code=130 test0.nc (pipe2.1)".
Segmentation fault (core dumped)
$ src/cdo -O fldmin -timmean -select,code=130 test0.nc test1.nc
cdo fldmin: Started child process "timmean -select,code=130 test0.nc (pipe1.1)".
cdo(2) timmean: Started child process "select,code=130 test0.nc (pipe2.1)".
cdo(3) select: Processed 1440 values from 3 variables over 5 timesteps ( 0.02s )
cdo(2) timmean: Processed 1440 values from 1 variable over 5 timesteps ( 0.02s )
cdo fldmin: Processed 288 values from 1 variable over 1 timestep ( 0.02s )

RE: Compilation issues grib_api - Added by Ralf Mueller over 7 years ago

Brendan DeTracey wrote:

Okay the following configure did successfully make:
[...]
But make check failed one test.
[...]

My nc-config:
[...]

netcdf is not the problem, it is the hdf5 used by netcdf. netcdf4 uses hdf5 as IO backend, but by default hdf5 is installed non-thread-safe. you can

install hdf5 in a thread safe mode, but here you will have to --enable-unsuported
use the CDO option for serialized IO: -L,e.g.
```
make check CDO='./src/cdo -L'
```

RE: Compilation issues grib_api - Added by Ralf Mueller over 7 years ago

some info about it directly from hdf5 developers: https://support.hdfgroup.org/hdf5-quest.html#tsafe

RE: Compilation issues grib_api - Added by Brendan DeTracey over 7 years ago

Thanks once again Ralf. All tests passed with flying colours. No longer bored, sad and lonely. Will cdo always require thread safe hdf?

RE: Compilation issues grib_api - Added by Ralf Mueller over 7 years ago

Brendan DeTracey wrote:

Thanks once again Ralf. All tests passed with flying colours. No longer bored, sad and lonely.

GREAT!!

Will cdo always require thread safe hdf?

in essence: yes.
reason: you can easily write down operation chains, that read from files in a parallel way. operators chains are already threaded unless you disable it explicitely (use option ... no, I won't tell you - CDO is great the way it is)

hence, just because CDO does not know, how netcdf/hdf5 is installed on the user's machine, the '-L' is needed. if not sure, just use it. if hdf5 is thread-safe, it's not needed.

(1-9/9)

Project

General

Profile

CDO