Reducing memory usage
Added by Vladimir Vovo about 11 years ago
Hello.
I've used remapping operator for small test grids and it works great!
But now i am trying to remap data from 360x180 to 3600x1800 grid on 50 levels. And for every level CDO allocates memory and deplete it all on about 30 levels.
Is there any option to ask CDO make interpolation as pipe - level per level - ([read]-[remap]-[write]-[read]-[remap]-[write] etc) without accumulating it?
Command i use:
cdo remapbil,./data/OCN_3600x1800_cdo.nc -selvar,t_an ./data/temperature_annual_1deg.nc ./data/OCN_3600x1800_T_AN.nc
My version:
cdo -V
Climate Data Operators version 1.5.3 (http://code.zmaw.de/projects/cdo)
Compiler: mpicc -std=gnu99 -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security -fPIC
version: gcc: error: unrecognized option '-V'
with: PTHREADS Z JASPER PROJ.4
Compiled: by buildd on allspice (x86_64-unknown-linux-gnu) Dec 12 2011 14:32:18
Replies (2)
RE: Reducing memory usage - Added by Uwe Schulzweida about 11 years ago
Hello Vladimir,
CDO holds the remap weights and grid information for all levels in memory if they differ. They differ if the number of missing values changed from level to level. The weights will be reused for the next time steps in the file. This is only useful if the file contains more than one timestep.
If you set the environment variable MAX_REMAPS to 1 then different weights for different levels are not stored in memory. The default for MAX_REMAPS is the number of levels.
Cheers,
Uwe