Project

General

Profile

Build cdo with Magics

Added by Yanchun He about 4 years ago

Hi,

Is there a easy way to build cdo with Magics. I can install Magics with pip, but it can not be used for cdo

./configure --with-magics=/usr/local/lib/python3.7/site-packages/Magics
...
checking for library containing mag_open... no
configure: error: Could not link to magics library

Thanks for the help!

Regards,
Yanchun


Replies (6)

RE: Build cdo with Magics - Added by Yanchun He about 4 years ago

If I compile Magic from source:

$ which python
/usr/local/bin/python
$ python --version
Python 3.7.7

$ pip3 install Jinja2
Requirement already satisfied: Jinja2 in /usr/local/lib/python3.7/site-packages (2.10.3)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.7/site-packages (from Jinja2) (1.1.1)

$ cmake -ECCODES_PATH=/usr/local/Cellar/eccodes/2.17.0/ -DCMAKE_PREFIX_PATH=/usr/local/Magics/4.3.0 ~/local/src/Magics-4.3.0-Source/

gives:


-- Feature NETCDF enabled
CMake Error at cmake/FindJinja2.cmake:38 (message):
  Jinja2 import failure:

  Traceback (most recent call last):

    File "<string>", line 1, in <module>

  ImportError: No module named jinja2

Call Stack (most recent call first):
  CMakeLists.txt:87 (find_package)

-- Configuring incomplete, errors occurred!
See also "/tmp/build/CMakeFiles/CMakeOutput.log".

But Jinja2 has been actually installed at: /usr/local/lib/python3.7/site-packages/jinja2/

RE: Build cdo with Magics - Added by Ralf Mueller about 4 years ago

hi!
whats you operating system?

-ralf

RE: Build cdo with Magics - Added by Yanchun He about 4 years ago

Thanks Ralf!

I use MacOS Catalina 10.15.3

I cleaned up the /tmp/build directory, and then it recognized the installed jinja2. And now Magc 4.2.0 is successfully installed, with the configuration flags/options as:

cmake -DCMAKE_INSTALL_PREFIX=${HOME}/local/magics/4.2.0 \
-DECCODES_PATH=/usr/local/Cellar/eccodes/2.17.0  \
-DENABLE_NETCDF=on \
-DNETCDF_PATH=/usr/local/Cellar/netcdf/4.7.3_2 \
${HOME}/local/src/Magics-4.2.0-Source

$ ls ~/local/magics/4.2.0/
bin/  include/  lib/  share/

The next step is to compile and install cdo 1.9.8 with some necessary libraries installed.

I configure the cdo 1.9.8 as:

./configure --prefix=${HOME}/local/cdo/1.9.8 \
--with-hdf5=/usr/local/Cellar/hdf5/1.12.0 \
--with-netcdf=/usr/local/Cellar/netcdf/4.7.3_2 \
--with-udunits2=/usr/local/Cellar/udunits/2.2.27.6 \
--with-fftw3 \
--with-proj=/usr/local/Cellar/proj/7.0.0 \
--with-magics=${HOME}/local/magics/4.2.0 \
--with-eccodes=/usr/local/Cellar/eccodes/2.17.0

the 'configure', 'make' and 'make install' all seems successful.
and the cdo 1.9.8 is installed at ~/local/cdo/1.9.8/bin

However, when I run ./cdo, it can not find the magic library:

$ ./cdo
dyld: Library not loaded: @rpath/libMagPlus.dylib
  Referenced from: /Users/heyc/local/cdo/1.9.8/bin/./cdo
  Reason: image not found
Abort trap: 6

I can post more detailed compile and build logs if you need.

Thank you very much!

Yanchun

RE: Build cdo with Magics - Added by Ralf Mueller about 4 years ago

you might consider macports: https://github.com/macports/macports-ports/blob/master/science/cdo/Portfile

it has a magics variant that should provide plotting with CDO

RE: Build cdo with Magics - Added by Yanchun He about 4 years ago

Thanks for the suggestion!

macports is kind of clumsy, but luckily I find a quick fix to this problem.

The problem is, by unknown reasons for me, the library path to magics is set as dynamic runtime path @rpath, and it only searches the default paths:

$ otool -L cdo
cdo:
    /usr/local/opt/libxml2/lib/libxml2.2.dylib (compatibility version 12.0.0, current version 12.10.0)
    @rpath/libMagPlus.dylib (compatibility version 0.0.0, current version 0.0.0)
    /usr/local/opt/proj/lib/libproj.19.dylib (compatibility version 20.0.0, current version 20.0.0)
    /usr/lib/libsqlite3.dylib (compatibility version 9.0.0, current version 308.4.0)
    /usr/local/opt/libtiff/lib/libtiff.5.dylib (compatibility version 11.0.0, current version 11.0.0)
    /usr/lib/libcurl.4.dylib (compatibility version 7.0.0, current version 9.0.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.0.0)
    /usr/local/opt/eccodes/lib/libeccodes.dylib (compatibility version 0.0.0, current version 0.0.0)
    /usr/local/opt/udunits/lib/libudunits2.dylib (compatibility version 0.0.0, current version 0.0.0)
    /usr/local/opt/netcdf/lib/libnetcdf.15.dylib (compatibility version 15.0.0, current version 15.0.0)
    /usr/local/opt/hdf5/lib/libhdf5_hl.200.dylib (compatibility version 201.0.0, current version 201.0.0)
    /usr/local/opt/hdf5/lib/libhdf5.200.dylib (compatibility version 201.0.0, current version 201.0.0)
    /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 800.7.0)

This can be fixed to install magics to default path (without prefix=/new/path/)
or set path:

$ install_name_tool -change @rpath/libMagPlus.dylib ~/local/magics/4.2.0/lib/libMagPlus.dylib ~/local/cdo/1.9.8/bin/cdo

Then otool -L cdo will show the full path to magics library:

/Users/heyc/local/magics/4.2.0/lib/libMagPlus.dylib (compatibility version 0.0.0, current version 0.0.0)

And now cdo works well with magics operators!

Thanks again :)

Cheers,
Yanchun

RE: Build cdo with Magics - Added by Yanchun He about 4 years ago

or directly set DYLD_LIBRARY_PATH

export DYLD_LIBRARY_PATH=${HOME}/local/magics/4.2.0/lib

    (1-6/6)