Rules for piping operators
Added by Pablo Larraondo over 6 years ago
Hello,
I'm trying to combine a selection and statistical operators into one command to avoid creating intermediate files:
$ cdo timsum -seldate,2018-03-09,2018-03-12 http://www.bom.gov.au/jsp/awra/thredds/dodsC/AWRACMS/values/day/rain_day_2018.nc 4days_accum.nc
Cannot create cookie file
cdo(2) seldate: Processed 2290884 values from 1 variable over 72 timesteps ( 0.21s )
cdo timsum: Processed 2290884 values from 1 variable over 4 timesteps ( 0.21s 99MB )
Segmentation fault
But applying the same operators independently works fine:
$ cdo seldate,2018-03-09,2018-03-12 http://www.bom.gov.au/jsp/awra/thredds/dodsC/AWRACMS/values/day/rain_day_2018.nc 4days.nc
Cannot create cookie file
cdo seldate: Processed 2290884 values from 1 variable over 72 timesteps ( 0.15s 82MB )
$ cdo timsum 4days.nc 4days_accum.nc
cdo timsum: Processed 2290884 values from 1 variable over 4 timesteps ( 0.09s 49MB )
I might be doing something wrong but I cannot find the rules of piping CDO commands in the documentation. Any help is greatly appreciated.
Thanks,
Pablo
Replies (4)
RE: Rules for piping operators - Added by Ralf Mueller over 6 years ago
hi Pablo!
thx for the report - could you give the CDO release, please? the output of
cdo -Vshould do it
cheers
ralf
RE: Rules for piping operators - Added by Pablo Larraondo over 6 years ago
Hi Ralf,
Thanks for looking at this. Here is the output of the cdo version command in my system:
[pl5189@raijin4]$ cdo -V
Climate Data Operators version 1.9.2 (http://mpimet.mpg.de/cdo)
Compiled: by mma900 on raijin7 (x86_64-unknown-linux-gnu) Dec 24 2017 00:42:45
CXX Compiler: g++ -std=gnu++11 -g -O2 -fopenmp
CXX version : g++ (GCC) 4.9.0
C Compiler: gcc -std=gnu99 -g -O2 -fopenmp
C version : gcc (GCC) 4.9.0
F77 Compiler: gfortran -g -O2
F77 version : GNU Fortran (GCC) 4.9.0
Features: 94GB Fortran DATA PTHREADS OpenMP4 HDF5 NC4/HDF5 OPeNDAP UDUNITS2 FFTW3 CMOR SSE2
Libraries: HDF5/1.8.14 CMOR/3.2.8
Filetypes: srv ext ieg grb1 grb2 nc1 nc2 nc4 nc4c
CDI library version : 1.9.2 of Dec 24 2017 00:38:36
CGRIBEX library version : 1.9.0 of Sep 29 2017 10:16:02
GRIB_API library version : 1.12.3
NetCDF library version : 4.3.3.1 of Mar 19 2015 11:24:25 $
HDF5 library version : 1.8.14
SERVICE library version : 1.4.0 of Dec 24 2017 00:38:17
EXTRA library version : 1.4.0 of Dec 24 2017 00:38:08
IEG library version : 1.4.0 of Dec 24 2017 00:38:13
FILE library version : 1.8.3 of Dec 24 2017 00:38:08
This NCO is run in an HPC system and loaded with a 'module load nco' command. I'll report back to the maintainers of the module if it's a problem in the compilation or in the version of the libraries used.
Thanks,
Pablo
RE: Rules for piping operators - Added by Ralf Mueller over 6 years ago
this is definitely no problem with rule of operators chaining - I cannot see anything wrong in your call. It could be
- an issue in CDO
- an issue in the netcdf-library used by the CDO binary.
with the current release 1.9.5 this doesn't because of other reasons :-/
sorry for the inconvenience
RE: Rules for piping operators - Added by Ralf Mueller over 6 years ago
this will be fixed in the next release 1.9.6