remapcon with high resolution 0.00833 deg global map
Added by Andrew Schuh almost 6 years ago
Having some trouble remapping a file. The file is at: https://landcover.usgs.gov/documents/average.tif.zip
I converted it to netcdf w/ "gdal_translate". I end up w/ ubyte type (Band1 variable) which I use NCO to convert to "float".
It's a regular lat/lon grid at approximately 1km that I'm trying to regrid up to 5X or 10X. I believe the command should be something like:
cdo -P 14 -f nc remapcon,r720x360 -setgrid,r43200x21600 GVF.nc GVF_0.5deg.nc
I'm working on a node w/ 28 cores and 3GB/core. The input file is just under 1GB. It is unclear whether my compute node is properly using the 14 cores I'm trying to use, but I'm not even sure if that should be necessary. It IS using all the node's memory. The memory useage blows up to 120GB or more, CPU useage goes way down and it seems to "hang", or at least takes a LONG time. remapbil seems to work fine, presumably it has a smaller memory footprint. I'm unsure why the memory useage has to be so high and I'm assuming something has to be going wrong. I can do the basic operations on my laptop so I'm unsure of whether I'm running the code wrong.
Can anybody successfully convert this file to coarser resolution w/ remapcon? and am I doing something wrong?
Replies (1)
RE: remapcon with high resolution 0.00833 deg global map - Added by Uwe Schulzweida almost 6 years ago
The CDO operator remapcon is obsolete and not optimized for regular 2D grids. Please try it with the new implementation remapycon:
gdal_translate -of netcdf Average.tif Average.nc cdo -P 4 -b F32 remapycon,r720x360 Average.nc result.ncIt takes 8 minutes and uses 35GB of memory on our cluster.