smooth operator not working
Added by Guillaume S about 5 years ago
Hello,
I am trying to use the "smooth" operator (with and without options) with files on different regular grids (curvilinear ORCA025, lonlat025 and gaussian F128), but the computation is never ending with all the different files. I also tried the "smooth9" operator without any problem.
I have the same problem with 2 different cdo versions:
Climate Data Operators version 1.9.3 (http://mpimet.mpg.de/cdo) Compiled: by modules on px-114.mercator-ocean.fr (x86_64-unknown-linux-gnu) Feb 7 2018 11:35:53 CXX Compiler: g++ -std=gnu++11 -g -O2 -fopenmp CXX version : g++ (GCC) 4.8.2 20140120 (Red Hat 4.8.2-16) C Compiler: gcc -std=gnu99 -g -O2 -fopenmp C version : gcc (GCC) 4.8.2 20140120 (Red Hat 4.8.2-16) F77 Compiler: gfortran -g -O2 F77 version : GNU Fortran (GCC) 4.8.2 20140120 (Red Hat 4.8.2-16) Features: 31GB C++11 Fortran DATA PTHREADS OpenMP3 NC4/HDF5 SSE2 Libraries: Filetypes: srv ext ieg grb1 grb2 nc1 nc2 nc4 nc4c CDI library version : 1.9.3 of Feb 7 2018 11:32:34 CGRIBEX library version : 1.9.0 of Jan 22 2018 09:24:03 GRIB_API library version : 1.13.1 NetCDF library version : 4.3.0 of Mar 8 2015 17:17:32 $ HDF5 library version : 1.8.11 EXSE library version : 1.4.0 of Feb 7 2018 11:32:10 FILE library version : 1.8.3 of Feb 7 2018 11:32:03
And:
Climate Data Operators version 1.9.6 (http://mpimet.mpg.de/cdo)
System: x86_64-pc-linux-gnu
CXX Compiler: /home/conda/feedstock_root/build_artifacts/cdo_1550155197086/_build_env/bin/x86_64-conda_cos6-linux-gnu-c++ fPIC -DPIC -g -O2 -std=c++11 -fopenmp -fPIC -DPIC -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -I/home/ext/mr/smer/samsong/miniconda3/include -fdebug-prefix-map=${SRC_DIR}=/usr/local/src/conda/${PKG_NAME}${PKG_VERSION} fdebug-prefix-map=${PREFIX}=/usr/local/src/conda-prefix -fopenmp${PKG_VERSION}
CXX version : unknown
C Compiler: /home/conda/feedstock_root/build_artifacts/cdo_1550155197086/_build_env/bin/x86_64-conda_cos6-linux-gnu-cc -fPIC -DPIC -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -I/home/ext/mr/smer/samsong/miniconda3/include -fdebug-prefix-map=${SRC_DIR}=/usr/local/src/conda/${PKG_NAME}fdebug-prefix-map=${PREFIX}=/usr/local/src/conda-prefix -fopenmp${PKG_VERSION} -fdebug-prefix-map=${PREFIX}=/usr/local/src/conda-prefix
C version : unknown
F77 Compiler: /home/conda/feedstock_root/build_artifacts/cdo_1550155197086/_build_env/bin/x86_64-conda_cos6-linux-gnu-gfortran -fopenmp -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -I/home/ext/mr/smer/samsong/miniconda3/include -fdebug-prefix-map=${SRC_DIR}=/usr/local/src/conda/${PKG_NAME}
F77 version : unknown
Features: 62GB 40threads C++11 Fortran DATA PTHREADS OpenMP45 HDF5 NC4/HDF5/threadsafe OPeNDAP UDUNITS2 PROJ.4 XML2 CURL FFTW3 SSE3
Libraries: HDF5/1.10.4 proj/5.2 xml2/2.9.8 curl/7.64.0
Filetypes: srv ext ieg grb1 grb2 nc1 nc2 nc4 nc4c nc5
CDI library version : 1.9.6
cgribex library version : 1.9.2
ecCodes library version : 2.12.0
NetCDF library version : 4.6.2 of Dec 17 2018 19:36:03 $
hdf5 library version : 1.10.4 threadsafe
exse library version : 1.4.1
FILE library version : 1.8.3
Did I missed something with the smooth operator ? Thank you for your help.
Guillaume
Replies (2)
RE: smooth operator not working - Added by Guillaume S about 5 years ago
I add a few details:
it is maybe related to a performance issue because I was able to use "smooth" with 4 timesteps of a ERAI (F128) file with 8 2D variables, but it took more than half an hour to process it:
cdo smooth: Processed 4194304 values from 8 variables over 4 timesteps [1878.05s 66MB]
For comparaison, with the "smooth9" operator, it took less than a second to treat the same file:
cdo smooth9: Processed 4194304 values from 8 variables over 4 timesteps [0.27s 52MB]
Do you have any advice about smooth operator usage or options to get it work faster ? Thanks !
Guillaume
RE: smooth operator not working - Added by Uwe Schulzweida about 5 years ago
The performance of the smooth operator depends on the option maxpoints. The default value of maxpoints is the number of gridpoints. That means all grid points are checked to see if they are within the search radius. If you think there are less than 1000 point inside the search radius (default 1deg) multiply this number by 10 to be on the safe side.
cdo smooth,maxpoints=10000 infile outfileAnother performance key is the number of used cores. The CDO operator smooth is parallelized with OpenMP. Use as much threads as you can. Here is an example to use smooth with 16 threads:
cdo -P 16 smooth,maxpoints=10000 infile outfile