Python syntax for cdo commands
Added by Romain LE LAMER over 4 years ago
Hi,
I am trying to include cdo commands in my Python script but I am having a problem ...
[...]
if c_tm == 6:
c_tm006_cnv = join(path_c_tz_cnv, 'GFS1_20200809_12_006_cnv.grb2')
c_tm006_10uv006 = join(path_b_tm003, 'GFS1_20200809_12_006_10uv006.nc')
# Rename 10u & 10v
cdo.setpartabn('10uv006',
option='-f nc',
input=c_tm006_cnv,
output=c_tm006_10uv006)
[...]
File 10uv006 are in the same directory that the script and contains
¶meter
name = 10u
out_name = 10u006
/
¶meter
name = 10v
out_name = 10v006
/
The file is created and in the correct directory but the parameters are not renamed.
What I don't understand is that this structure works perfectly in this case (cnvUV is also in the same location as the file above)
[...]
gribfile = 'GFS1_20200809_12_006.grb2'
gribfile_cnv = 'GFS1_20200809_12_006_cnv.grb2'
cdo.exprf('cnvUV',
input=join(path_c_tz, gribfile),
output=join(path_c_tz_cnv, gribfile_cnv))
print('Convert OK')
[...]
File cnvUV
10u = ((10u > 0) ? 1 : -1) * sqr(nint(sqrt(abs(10u * 230.4)))) / 230.4;
10v = ((10v > 0) ? 1 : -1) * sqr(nint(sqrt(abs(10v * 230.4)))) / 230.4;
I also have the controls
cdo.merge / cdo.setreftime & cdo.mergetime to translate with "python syntax"
PS: Is there documentation somewhere on the Python syntax for commanced cdo ?
If yes I did not find it.
Thanks
Replies (20)
RE: Python syntax for cdo commands - Added by Romain LE LAMER over 4 years ago
I understood my mistake ...
(thanks cdo.debug = True)
Do not forget this "s" to options=
...
RE: Python syntax for cdo commands - Added by Ralf Mueller over 4 years ago
hi Romain!
yeah, unfortunately there is no warning for unknown keywords. Maybe I should implement this.
On Cdo{rbpy} you can find some examples for python usage.
hth
ralf
RE: Python syntax for cdo commands - Added by Kyle Lesinger over 4 years ago
Romain:
May I ask how you were even able to get cdo working in Python? Are you using a specific IDE or was there a website that helped? I have been trying for days and I have not yet found a good source for even getting it started. I do have it installed as version 1.5.3.
Any help would be appreciated.
Thanks,
KL
RE: Python syntax for cdo commands - Added by Romain LE LAMER over 4 years ago
Hi Kyle,
I'm using PyCharm CE and my "bible" is [[https://code.mpimet.mpg.de/projects/cdo/embedded/index.html]].
I have installed cdo with
pip install cdo netcdf4 xarray
Your file.py must be contains
from cdo import *
# OR
# from cdo import Cdo
cdo = Cdo()
#cdo.debug = True """ can be useful to help for solve errors """
# Commands
cdo.operator('parameters',
options=XXX, """ if need """
input=infile,
output=outfile)
I don't know much more for now
RE: Python syntax for cdo commands - Added by Kyle Lesinger over 4 years ago
Thank you for your assistance. I use spyder and unfortunately that pip install cdo netcdf4 xarray did not solve my issue. For some reason I get this error:
signal.siginterrupt(signal.SIGINT, False)
AttributeError: module 'signal' has no attribute 'siginterrupt'
Thanks for your help though. I'll figure it out one day!
RE: Python syntax for cdo commands - Added by Ralf Mueller over 4 years ago
@Kyle: Which version of python do you use?
signal.siginterrup() was introduced in python 2.6 and is still there in 3.8
RE: Python syntax for cdo commands - Added by Ralf Mueller over 4 years ago
Romain LE LAMER wrote:
Hi Kyle,
I'm using PyCharm CE and my "bible" is [[https://code.mpimet.mpg.de/projects/cdo/embedded/index.html]].
Amen!! ;-)
RE: Python syntax for cdo commands - Added by Kyle Lesinger over 4 years ago
@Ralf Thehos: I am running Python 3.7.6.
RE: Python syntax for cdo commands - Added by Romain LE LAMER over 4 years ago
I am on macOS and a friend also has the same problem (with my script) as you Kyle, and he is on Windows ...
My question is therefore are you under Windows also ?
Me:
macOS v10.15.6
python v3.8.5
cdo v1.9.8
$ cdo --version
Climate Data Operators version 1.9.8 (https://mpimet.mpg.de/cdo)
System: x86_64-apple-darwin19.4.0
CXX Compiler: clang++ -std=gnu++11 -g -O2 -D_THREAD_SAFE -pthread
CXX version : Apple clang version 11.0.3 (clang-1103.0.32.59)
C Compiler: clang -g -O2 -D_THREAD_SAFE -pthread -D_THREAD_SAFE -D_THREAD_SAFE -pthread
C version : Apple clang version 11.0.3 (clang-1103.0.32.59)
F77 Compiler: gfortran -g -O2
F77 version : GNU Fortran (Homebrew GCC 9.3.0_1) 9.3.0
Features: 8GB 4threads C++11 Fortran DATA PTHREADS HDF5 NC4/HDF5 OPeNDAP SZ SSE4_2
Libraries: HDF5/1.12.0
Filetypes: srv ext ieg grb1 grb2 nc1 nc2 nc4 nc4c nc5
CDI library version : 1.9.8
cgribex library version : 1.9.4
ecCodes library version : 2.18.0
NetCDF library version : 4.7.4 of Jul 2 2020 22:40:24 $
hdf5 library version : 1.12.0
exse library version : 1.4.1
FILE library version : 1.8.3
A friend :
Windows 10
python v3.8.5
cdo --version not possible
RE: Python syntax for cdo commands - Added by Romain LE LAMER over 4 years ago
Oop's it is not good ...
You must have at least these packages
pip list
Package Version
--------------- ---------
cdo 1.5.3
cftime 1.2.1
netCDF4 1.5.4
numpy 1.19.1
pandas 1.1.0
pip 20.1.1
python-dateutil 2.8.1
pytz 2020.1
six 1.15.0
wheel 0.34.2
xarray 0.16.0
RE: Python syntax for cdo commands - Added by Romain LE LAMER over 4 years ago
I come back to the error encountered by Kyle and a friend (who uses my script)
As I indicated above it is under Windows 10, at Python v3.8.5 and via pip it installed cdo netcdf4 and xarray
C:\BIN\Python\python.exe D:/Temp/main.py
Traceback (most recent call last):
File "D:/Temp/main.py", line 112, in <module>
cdo = Cdo()
File "C:\BIN\Python\lib\site-packages\cdo.py", line 170, in _init_
signal.siginterrupt(signal.SIGINT, False)
AttributeError: module 'signal' has no attribute 'siginterrupt'
Process finished with exit code 1
My conclusion (after testing) is that it is necessary to have the cdo binary installed locally.
• For UNIX systems, it must be compiled or installed via the system's package manager (me on macOS I used Homebrew (brew install cdo))
• For Windows systems, it is necessary to do this via Cygwin.
I think a cdo developer will confirm this conclusion.
RE: Python syntax for cdo commands - Added by Kyle Lesinger over 4 years ago
@Romain:
Yes I am also using windows. I've tried to get CDO to work with Spyder, Pycharm, and cygwin now but to no avail. I did update all my packages to be exactly the same way that you did Romain, but for some reason I still get the error even in Pycharm
Traceback (most recent call last):
File "C:\Users\kylel\Anaconda3\envs\cdo\lib\site-packages\IPython\core\interactiveshell.py", line 3417, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-3-237a1757623c>", line 1, in <module>
cdo = Cdo()
File "C:\Users\kylel\Anaconda3\envs\cdo\lib\site-packages\cdo.py", line 170, in init
signal.siginterrupt(signal.SIGINT, False)
AttributeError: module 'signal' has no attribute 'siginterrupt'.
I tried to install Ubuntu through the VM Virtual Machine, but unfortunately computers really dislike me and there are errors in the download :(.
RE: Python syntax for cdo commands - Added by Romain LE LAMER over 4 years ago
Hi Kyle
Use this tutorial to install correctly cygwin + cdo
=> [[https://opencpn.org/wiki/dokuwiki/doku.php?id=opencpn:supplementary_software:cdo:windows_8.1#:~:text='cdo.exe'%20is%20extracted,running%20the%20batch%20file%20'Cygwin.]]
RE: Python syntax for cdo commands - Added by Ralf Mueller over 4 years ago
hi!
There is not CDO binary for the native Windows system. Hence you cannot use python/anaconda/pip to call CDO through cdo.py, because there is no CDO on plain Windows.
For Windows there are two option
- cygwin (Thx for the link, Roman)
- the Windows Subsystem for Linux and install cdo with
sudo apt-get install cdo
The windows version of python doesn't seem to have this part of the signal
module - sorry for missing the fact, that you work under windows directly.
RE: Python syntax for cdo commands - Added by Romain LE LAMER over 4 years ago
Hi Ralf & Kyle
I did not know Windows Subsystem for Linux (WSL) at all but I find it to be THE solution to adopt.
It is extremely easy to implement (compared to Cygwin)
Thanks for that
RE: Python syntax for cdo commands - Added by Ralf Mueller over 4 years ago
hi Roman!
yes, it's a lot easier IMO. the only disadvantage is, that you won't get the latest release out-of-the-box. But for most applications this will not be needed.
The other question is, whether Microsoft is willing to support this on the long run. Many years ago there was a Unix Subsystem, but totally outdated and a Microsoft employer told me, he would use cygwin instead.
Maybe this has changed now - debian/ubuntu is well maintained, Microsoft updates the ubuntu version.
In case of problems using WSL please let me know. I am curious how a real world setup works in there.
cheers
ralf
RE: Python syntax for cdo commands - Added by Romain LE LAMER over 4 years ago
I think you have to do it in a simple way, that is to say you should not try to make Windows and Linux communicate, you have to run everything from Linux.
If Windows gives the possibility of running a Linux so easily it is a bit of the death of Windows (and it's not me that will bother )
RE: Python syntax for cdo commands - Added by Kyle Lesinger over 4 years ago
Thank you all for your input. After many hours, I have finally been able to run CDO through the Ubuntu Desktop app!
Here were my steps for anyone else in the future who tries to install it:
After first opening and creating your username:
sudo apt-get update and sudo apt-get upgrade
Install CDO
sudo apt-get install cdo
Try cdo - v if you get this error:
cdo: error while loading shared libraries: libQt5Core.so.5: cannot open shared object file: No such file or directory
Then:
sudo strip --remove-section=.note.ABI-tag /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
Then you have to move your file you want to edit into the actual Desktop folder. Then cdo operators will work.
RE: Python syntax for cdo commands - Added by Kyle Lesinger over 4 years ago
I maybe shouldn't have posted this here considering the title of the post is Python syntax for cdo commands, but this may be much quicker for others if they have a hard time figuring it out.
RE: Python syntax for cdo commands - Added by Ralf Mueller over 4 years ago
hi Kyle!
no worries - it's good that we have it here! thx for the info
best wishes
ralf