Rules for piping operators

Added by Pablo Larraondo over 5 years ago

Hello,

I'm trying to combine a selection and statistical operators into one command to avoid creating intermediate files:

$ cdo timsum -seldate,2018-03-09,2018-03-12 http://www.bom.gov.au/jsp/awra/thredds/dodsC/AWRACMS/values/day/rain_day_2018.nc 4days_accum.nc Cannot create cookie file cdo(2) seldate: Processed 2290884 values from 1 variable over 72 timesteps ( 0.21s ) cdo timsum: Processed 2290884 values from 1 variable over 4 timesteps ( 0.21s 99MB ) Segmentation fault

But applying the same operators independently works fine:

$ cdo seldate,2018-03-09,2018-03-12 http://www.bom.gov.au/jsp/awra/thredds/dodsC/AWRACMS/values/day/rain_day_2018.nc 4days.nc Cannot create cookie file cdo seldate: Processed 2290884 values from 1 variable over 72 timesteps ( 0.15s 82MB ) $ cdo timsum 4days.nc 4days_accum.nc cdo timsum: Processed 2290884 values from 1 variable over 4 timesteps ( 0.09s 49MB )

I might be doing something wrong but I cannot find the rules of piping CDO commands in the documentation. Any help is greatly appreciated.

Thanks,
Pablo

Replies (4)

RE: Rules for piping operators - Added by Ralf Mueller over 5 years ago

hi Pablo!

thx for the report - could you give the CDO release, please? the output of

cdo -V

should do it

cheers
ralf

RE: Rules for piping operators - Added by Pablo Larraondo over 5 years ago

Hi Ralf,

Thanks for looking at this. Here is the output of the cdo version command in my system:

[pl5189@raijin4]$ cdo -V Climate Data Operators version 1.9.2 (http://mpimet.mpg.de/cdo) Compiled: by mma900 on raijin7 (x86_64-unknown-linux-gnu) Dec 24 2017 00:42:45 CXX Compiler: g++ -std=gnu++11 -g -O2 -fopenmp CXX version : g++ (GCC) 4.9.0 C Compiler: gcc -std=gnu99 -g -O2 -fopenmp C version : gcc (GCC) 4.9.0 F77 Compiler: gfortran -g -O2 F77 version : GNU Fortran (GCC) 4.9.0 Features: 94GB Fortran DATA PTHREADS OpenMP4 HDF5 NC4/HDF5 OPeNDAP UDUNITS2 FFTW3 CMOR SSE2 Libraries: HDF5/1.8.14 CMOR/3.2.8 Filetypes: srv ext ieg grb1 grb2 nc1 nc2 nc4 nc4c CDI library version : 1.9.2 of Dec 24 2017 00:38:36 CGRIBEX library version : 1.9.0 of Sep 29 2017 10:16:02 GRIB_API library version : 1.12.3 NetCDF library version : 4.3.3.1 of Mar 19 2015 11:24:25 $ HDF5 library version : 1.8.14 SERVICE library version : 1.4.0 of Dec 24 2017 00:38:17 EXTRA library version : 1.4.0 of Dec 24 2017 00:38:08 IEG library version : 1.4.0 of Dec 24 2017 00:38:13 FILE library version : 1.8.3 of Dec 24 2017 00:38:08

This NCO is run in an HPC system and loaded with a 'module load nco' command. I'll report back to the maintainers of the module if it's a problem in the compilation or in the version of the libraries used.

Thanks,
Pablo

RE: Rules for piping operators - Added by Ralf Mueller over 5 years ago

this is definitely no problem with rule of operators chaining - I cannot see anything wrong in your call. It could be

an issue in CDO
an issue in the netcdf-library used by the CDO binary.

with the current release 1.9.5 this doesn't because of other reasons :-/

sorry for the inconvenience

RE: Rules for piping operators - Added by Ralf Mueller over 5 years ago

this will be fixed in the next release 1.9.6

(1-4/4)

Project

General

Profile

CDO