regrid never completes...help
Added by Erik Noble almost 12 years ago
Could I please have some help? When I try to use the remapbil command (or any of the other remapping tools) on one file, CDO keeps looping infinitely. When I use the same method on other files, I don't have nay propblems/
For instance when I run the following command, CDO just loops and loops and loops infinitely. It never creates the regridded file.
cdo sellonlatbox,-35,35,-20,35 -remapbil,$REGRID/1x1_grid.griddes MERRA300.prod.assim.inst3_3d_asm_Cp.20060902.SUB.nc 1x1_out.nc
cdo sellonlatbox: Started child process "remapbil,/Volumes/DATA_1_Terabyte/POST_PROCESSING/regridding_tools/1x1_grid.griddes MERRA300.prod.assim.inst3_3d_asm_Cp.20060902.SUB.nc (pipe1.1)".
Warning (scanVarAttributes) : Gridtype linear unsupported!
cdo(2) remapbil: 53%
[snip]
5 minutes later....it is still trying to regrid.
here are the contents of the file:
[eunoble@GS611-Noble] data $ ncdump -h MERRA300.prod.assim.inst3_3d_asm_Cp.20060902.SUB.nc
netcdf MERRA300.prod.assim.inst3_3d_asm_Cp.20060902.SUB {
dimensions:
time = UNLIMITED ; // (8 currently)
levels = 42 ;
longitude = 288 ;
latitude = 144 ;
variables:
double time(time) ;
time:units = "hours since 2006-9-2 0" ;
double levels(levels) ;
levels:units = "hPa" ;
levels:description = "Pressure level" ;
levels:type = "plev" ;
levels:long_name = "Level" ;
levels:positive = "down" ;
double longitude(longitude) ;
longitude:units = "degrees_east" ;
longitude:long_name = "Longitude" ;
double latitude(latitude) ;
latitude:units = "degrees_north" ;
latitude:long_name = "Latitude" ;
float qv(time, levels, latitude, longitude) ;
qv:comments = "Unknown1 variable comment" ;
qv:long_name = "Specific humidity" ;
qv:units = "" ;
qv:grid_name = "grid-1" ;
qv:grid_type = "linear" ;
qv:time_statistic = "instantaneous" ;
qv:missing_value = 1.e+15f ;
float rh(time, levels, latitude, longitude) ;
rh:comments = "Unknown1 variable comment" ;
rh:long_name = "Relative humidity" ;
rh:units = "" ;
rh:grid_name = "grid-1" ;
rh:grid_type = "linear" ;
rh:time_statistic = "instantaneous" ;
rh:missing_value = 1.e+15f ;
float t(time, levels, latitude, longitude) ;
t:comments = "Unknown1 variable comment" ;
t:long_name = "Air Temperature" ;
t:units = "" ;
t:grid_name = "grid-1" ;
t:grid_type = "linear" ;
t:time_statistic = "instantaneous" ;
t:missing_value = 1.e+15f ;
float u(time, levels, latitude, longitude) ;
u:comments = "Unknown1 variable comment" ;
u:long_name = "Eastward wind component" ;
u:units = "" ;
u:grid_name = "grid-1" ;
u:grid_type = "linear" ;
u:time_statistic = "instantaneous" ;
u:missing_value = 1.e+15f ;
float v(time, levels, latitude, longitude) ;
v:comments = "Unknown1 variable comment" ;
v:long_name = "Northward wind component" ;
v:units = "" ;
v:grid_name = "grid-1" ;
v:grid_type = "linear" ;
v:time_statistic = "instantaneous" ;
v:missing_value = 1.e+15f ;
// global attributes:
:Conventions = "COARDS" ;
:calendar = "standard" ;
:comments = "file created by grads using lats4d available from http://dao.gsfc.nasa.gov/software/grads/lats4d/" ;
:model = "geos/das" ;
:center = "gsfc" ;
Replies (5)
RE: regrid never completes...help - Added by Jaison-Thomas Ambadan almost 12 years ago
Hi Erik,
I couldn't reproduce the problem; for me it worked fine! Here is the command sequence that I used:
1. cdo -f nc -sellonlatbox,-35,35,-20,35 -random,r360x180 template.nc
This create a "template" of the required lat/lon box with 1 degree resolution (random numbers, just for the grid info)
2. cdo -f nc -remapbil,template.nc MERRA300.prod.assim.inst3_3d_asm_Cp.20060914.SUB.nc outfile.nc
DONE!! Here is the "cdo -sinfov outfile.nc"
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
File format: netCDF
-1 : Institut Source Name Time Typ Grid Size Num Levels Num
1 : unknown unknown qv var F32 3905 1 42 1
2 : unknown unknown rh var F32 3905 1 42 1
3 : unknown unknown t var F32 3905 1 42 1
4 : unknown unknown u var F32 3905 1 42 1
5 : unknown unknown v var F32 3905 1 42 1
Horizontal grids :
1 : lonlat > size : dim = 3905 nlon = 71 nlat = 55
lon : first = -35 last = 35 inc = 1 degrees_east
lat : first = -19.5 last = 34.5 inc = 1 degrees_north
Vertical grids :
1 : pressure hPa : 1000 975 950 925 900 875 850 825 800 775 750
725 700 650 600 550 500 450 400 350 300 250 200
150 100 70 50 40 30 20 10 7 5 4 3 2 1 0.699999988
0.5 0.400000006 0.300000012 0.100000001
Time axis : 8 steps
RefTime = 2006-09-14 00:00:00 Units = hours Calendar = STANDARD
YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss
2006-09-14 00:00:00 2006-09-14 03:00:00 2006-09-14 06:00:00 2006-09-14 09:00:00
2006-09-14 12:00:00 2006-09-14 15:00:00 2006-09-14 18:00:00 2006-09-14 21:00:00
cdo sinfon: Processed 5 variables over 8 timesteps. ( 0.05s )
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If you've any doubts/questions, please do not hesitate to ask!
Cheers,
J.
PS: It is always good to use the latest version of CDO (I'm using v1.5.4).
RE: regrid never completes...help - Added by Uwe Schulzweida almost 12 years ago
Hi Erik,
I tried it with a global 1 degree grid and can't reproduce your problem. The remapping takes less than 2 minutes on my machine:
cdo sellonlatbox,-35,35,-20,35 -remapbil,global_1 MERRA300.prod.assim.inst3_3d_asm_Cp.20060914.SUB.nc resultCould you please also attach your target grid description file 1x1_grid.griddes?
And your CDO version number (cdo -V) could also be interested.
Cheers,
Uwe
RE: regrid never completes...help - Added by Erik Noble almost 12 years ago
Thank you. Almost there.
The first step worked. Thank you for this.
[eunoble@GS611-Noble] data $ time cdo -f nc -sellonlatbox,-35,35,-20,35 -random,r360x180 template.nc
cdo sellonlatbox: Started child process "random,r360x180 (pipe1.1)".
cdo(2) random: Processed 0 variable. ( 0.00s )
cdo sellonlatbox: Processed 64800 values from 1 variable over 1 timestep. ( 0.00s )
real 0m0.034s
user 0m0.006s
sys 0m0.011s
But the second step ( and this time I am using remapcon instead of remapbil) still loops over & over again, using this new template.
As you can see, when I halt this step, 9 minutes have already past...
[eunoble@GS611-Noble] data $ time cdo -f nc -remapcon,template.nc MERRA300.prod.assim.inst3_3d_asm_Cp.20060914.SUB.nc outfile.nc
Warning (scanVarAttributes) : Gridtype linear unsupported!
cdo remapcon: 19%^C
real 9m55.035s
user 9m54.092s
sys 0m0.915s
This is the CDO version I am using on this computer (last installed on this Mac computer using Macports). Could this be the problem?
[eunoble@GS611-Noble] ~ $ cdo -V
Climate Data Operators version 1.5.2 (http://code.zmaw.de/projects/cdo)
Compiler: /Developer/usr/bin/llvm-gcc-4.2 -std=gnu99 -pipe -O2 -arch x86_64 -D_THREAD_SAFE -pthread
version: i686-apple-darwin11-llvm-gcc-4.2 (GCC) 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)
with: PTHREADS SZ Z PROJ.4
Compiled: by root on gs611-noble.local (x86_64-apple-darwin11.1.0) Sep 22 2011 11:41:57
CDI library version : 1.5.2 of Sep 22 2011 11:41:53
CGRIBEX library version : 1.5.0 of Feb 15 2011 10:23:39
netCDF library version : 4.1.3 of Oct 4 2011 16:37:15 $
HDF5 library version : 1.8.7
SERVICE library version : 1.3.0 of Sep 22 2011 11:41:52
EXTRA library version : 1.3.0 of Sep 22 2011 11:41:51
IEG library version : 1.3.0 of Sep 22 2011 11:41:52
FILE library version : 1.7.1 of Sep 22 2011 11:41:51
RE: regrid never completes...help - Added by Jaison-Thomas Ambadan almost 12 years ago
Hi Erik,
Uwe's method SHOULD WORK, even for conservative remapping, "rempacon" (I did it with your data set). However, it took ~8 minutes on a single Quad-Core AMD Opteron(tm) Processor 8356 (2293.905MHz), because it is global conservative remaping. So naturally, it will take longer time on a laptop - you just have to wait for the output, I think 
However, if your CDO is compiled with OpenMP support then you can use more cpu/cores using the "-P" option on a cluster/higher end machine; like:
cdo -P 8 -sellonlatbox,-35,35,-20,35 -remapcon,global_1 MERRA300.prod.assim.inst3_3d_asm_Cp.20060914.SUB.nc outfile.nc
On the other hand, instead of remaping the global data, you can first select the lon/lat box from your original input file, and then do the remap (assuming that you have the template.nc):
cdo -remapcon,template.nc -sellonlatbox,-35,35,-20,35 MERRA300.prod.assim.inst3_3d_asm_Cp.20060914.SUB.nc outfile.nc
It took only <10 SECONDS! on the same machine (on the laptop, may be you've to wait for couple of minutes)
Cheers,
Jaison.
PS: I guess, the looping that you mentioned is the remaping process at every levels, for every variables, for every time steps i.e. you will see 5*42*8=1680 loops!
RE: regrid never completes...help - Added by Erik Noble almost 12 years ago
Hi. Thank you for the help.
Things started working once I upgraded to the latest version of CDO.
Uwe, thank your or the help.
Jaison, your last suggestion of switching the order of commands helped immensely after I upgraded to the latest version. It didn't seem to make a difference when I was using CDO 5.2. Not sure why.
Again, thank you,
Erik