cdo remap with weights computed outside cdo ?
Added by Olivier Marti about 10 years ago
cdo remap with weights computed outside cdo ?¶
I'm trying to use cdo to interpolates files using weights computed outside cdo, but at the SCRIP format. The wieghts file use works properly with OASIS MCT.
I run the command :
cdo -v remap,torc_grid.nc,rmp_tlmd_to_torc_mosaic.nc tlmd_infile.nc t2m_mosaic_from_tlmd_to_torc.nc
What seems to append : cdo read the global attribute "map_method", and recompute all weights using the method. It does not use the weights inside the file. If I use cdo gen* to compute my weights, I obtain exactly the same results ("genbil" matching the result off map_method="Bilinear", etc ...)
Is it possible to use external weights ? How ?
Thanks
Olivier
ncdump -h torc_grid.nc
netcdf torc_grid {
dimensions:
jpjo = 149 ;
jpio = 182 ;
corners = 4 ;
variables:
byte dummy(jpjo, jpio) ;
dummy:coordinates = "torc.lat torc.lon" ;
double torc.cla(corners, jpjo, jpio) ;
torc.cla:units = "degrees_north" ;
torc.cla:title = "corners of orca-t latitudes" ;
torc.cla:grid_type = "P" ;
torc.cla:overlap = 2 ;
double torc.clo(corners, jpjo, jpio) ;
torc.clo:units = "degrees_east" ;
torc.clo:title = "corners of orca-t longitudes" ;
torc.clo:grid_type = "P" ;
torc.clo:overlap = 2 ;
double torc.lat(jpjo, jpio) ;
torc.lat:units = "degrees_north" ;
torc.lat:title = "orca-t latitudes" ;
torc.lat:grid_type = "P" ;
torc.lat:overlap = 2 ;
torc.lat:bounds = "torc.cla" ;
double torc.lon(jpjo, jpio) ;
torc.lon:units = "degrees_east" ;
torc.lon:title = "orca-t longitudes" ;
torc.lon:grid_type = "P" ;
torc.lon:overlap = 2 ;
torc.lon:bounds = "torc.clo" ;
// global attributes:
:Conventions = "CF-1.1" ;
:Model = "IPSLCM6" ;
:NCO = "4.1.0" ;
:history = "Wed Mar 5 17:08:23 2014: ncatted -a coordinates,dummy,o,c,torc.lat torc.lon torc_grid.nc\n",
"Wed Mar 5 17:08:23 2014: ncap2 -O -s dummy[jpjo,jpio]=1b torc_grid.nc torc_grid.nc\n",
"Wed Mar 5 17:08:23 2014: ncatted -a bounds,torc.lon,o,c,torc.clo -a bounds,torc.lat,o,c,torc.cla torc_grid.nc\n",
"Wed Mar 5 17:08:23 2014: ncks -O -v torc.lon,torc.lat,torc.clo,torc.cla ../MOZAIC/grids.nc torc_grid.nc\n",
"Tue Feb 25 13:48:24 2014: ncks -A orca_grids_runoff_conserv.nc ../MOZAIC/grids.nc\n",
"Tue Feb 25 13:48:24 2014: ncks -A lmdz_grids_runoff_conserv.nc ../MOZAIC/grids.nc\n",
"Tue Feb 25 13:48:04 2014: ncks -A orca_grids_runoff_conserv.nc ../MOZAIC/grids.nc\n",
"Tue Feb 25 13:48:04 2014: ncks -A lmdz_grids_runoff_conserv.nc ../MOZAIC/grids.nc\n",
"Tue Feb 25 13:47:37 2014: ncks -A orca_grids_runoff_conserv.nc ../MOZAIC/grids.nc\n",
"Tue Feb 25 13:47:37 2014: ncks -A lmdz_grids_runoff_conserv.nc ../MOZAIC/grids.nc\n",
"Tue Feb 25 13:45:09 2014: ncks -A orca_grids_runoff_conserv.nc ../MOZAIC/grids.nc\n",
"Tue Feb 25 13:45:09 2014: ncks -A lmdz_grids_runoff_conserv.nc ../MOZAIC/grids.nc" ;
:nco_openmp_thread_number = 1 ;
}
ncdump -h tlmd_infile.nc
netcdf tlmd_infile {
dimensions:
lat = 96 ;
lon = 96 ;
time_counter = UNLIMITED ; // (12 currently)
variables:
float lat(lat) ;
lat:axis = "Y" ;
lat:standard_name = "latitude" ;
lat:units = "degrees_north" ;
lat:valid_min = -90.f ;
lat:valid_max = 90.f ;
lat:long_name = "Latitude" ;
lat:nav_model = "Default grid" ;
float lon(lon) ;
lon:axis = "X" ;
lon:standard_name = "longitude" ;
lon:units = "degrees_east" ;
lon:valid_min = -180.f ;
lon:valid_max = 176.25f ;
lon:long_name = "Longitude" ;
lon:nav_model = "Default grid" ;
float t2m(time_counter, lat, lon) ;
t2m:units = "K" ;
t2m:standard_name = "Temperature 2m" ;
t2m:_FillValue = 9.96921e+36f ;
t2m:long_name = "Temperature 2m" ;
t2m:online_operation = "ave(X)" ;
t2m:interval_operation = 1800.f ;
t2m:interval_write = 2678400.f ;
t2m:coordinates = "time_counter lat lon" ;
double time_counter(time_counter) ;
time_counter:axis = "T" ;
time_counter:standard_name = "time" ;
time_counter:units = "seconds since 1800-01-01 00:00:00" ;
time_counter:calendar = "noleap" ;
time_counter:title = "Time" ;
time_counter:long_name = "Time axis" ;
time_counter:time_origin = " 1800-JAN-01 00:00:00" ;
time_counter:bounds = "time_counter_bnds" ;
// global attributes:
:Conventions = "CF-1.1" ;
:file_name = "histmth.nc" ;
:production = "An IPSL model" ;
:TimeStamp = "2011-JAN-14 15:42:31 GMT-0100" ;
:history = "Wed Mar 5 17:08:23 2014: ncks -O -v t2m -d time_counter,0,11 /ccc/store/cont003/dsm/p86caub/IGCM_OUT/IPSLCM5A/PROD/piControl/piControl2/ATM/Analyse/TS_MO/piControl2_28000101_36991231_1M_t2m.nc tlmd_infile.nc" ;
:NCO = "4.1.0" ;
}
ncdump -h rmp_tlmd_to_torc_mosaic.nc
netcdf rmp_tlmd_to_torc_mosaic {
dimensions:
src_grid_size = 9216 ;
dst_grid_size = 27118 ;
src_grid_corners = 4 ;
dst_grid_corners = 4 ;
src_grid_rank = 2 ;
dst_grid_rank = 2 ;
num_links = 44142 ;
num_wgts = 1 ;
variables:
int src_grid_dims(src_grid_rank) ;
int dst_grid_dims(dst_grid_rank) ;
double src_grid_center_lat(src_grid_size) ;
src_grid_center_lat:units = "radians" ;
double src_grid_center_lon(src_grid_size) ;
src_grid_center_lon:units = "radians" ;
double dst_grid_center_lat(dst_grid_size) ;
dst_grid_center_lat:units = "radians" ;
double dst_grid_center_lon(dst_grid_size) ;
dst_grid_center_lon:units = "radians" ;
double src_grid_corner_lat(src_grid_size, src_grid_corners) ;
src_grid_corner_lat:units = "radians" ;
double src_grid_corner_lon(src_grid_size, src_grid_corners) ;
src_grid_corner_lon:units = "radians" ;
double dst_grid_corner_lat(dst_grid_size, dst_grid_corners) ;
dst_grid_corner_lat:units = "radians" ;
double dst_grid_corner_lon(dst_grid_size, dst_grid_corners) ;
dst_grid_corner_lon:units = "radians" ;
int src_grid_imask(src_grid_size) ;
src_grid_imask:units = "unitless" ;
src_grid_imask:land_value = 0 ;
src_grid_imask:sea_value = 1 ;
int dst_grid_imask(dst_grid_size) ;
dst_grid_imask:units = "unitless" ;
dst_grid_imask:land_value = 0 ;
dst_grid_imask:sea_value = 1 ;
double src_grid_area(src_grid_size) ;
src_grid_area:units = "square radians" ;
double dst_grid_area(dst_grid_size) ;
dst_grid_area:units = "square radians" ;
double src_grid_frac(src_grid_size) ;
src_grid_frac:units = "unitless" ;
double dst_grid_frac(dst_grid_size) ;
dst_grid_frac:units = "unitless" ;
int dst_address(num_links) ;
int src_address(num_links) ;
double remap_matrix(num_links, num_wgts) ;
// global attributes:
:Conventions = "CF-1.1" ;
:title = "rmp_tlmd_to_torc_MOSAIC.nc" ;
:normalization = "none" ;
:map_method = "Nearest" ;
:history = "Wed Mar 5 16:58:18 2014: ncatted -a map_method,global,o,c,Nearest rmp_tlmd_to_torc_mosaic.nc\n",
"Created: 2014-03-05 14h57 GMT+0100" ;
:conventions = "SCRIP" ;
:method = "MOSAIC" ;
:source_grid = "lonlat" ;
:dest_grid = "curvilinear" ;
:Institution = "IPSL" ;
:Model = "IPSL CM6" ;
:Max_nei_num = 96 ;
:HOSTNAME = "curie71" ;
:LOGNAME = "p86mart" ;
}
Replies (6)
RE: cdo remap with weights computed outside cdo ? - Added by Uwe Schulzweida about 10 years ago
Hi Oliver,
It should be possible to use the weights generate by SCRIP with the CDO operator remap. I can't reproduce the problem with my testdata. Could you please upload your netCDF file? That would help me to verify this problem.
Cheers,
Uwe
RE: cdo remap with weights computed outside cdo ? - Added by Olivier Marti about 10 years ago
Uwe,
Files are uploaded.
Thanks for caring,
Olivier
| rmp_tlmd_to_torc_mosaic.nc (4.14 MB) rmp_tlmd_to_torc_mosaic.nc | Weights, computed by an external program | ||
| torc_grid.nc (2.1 MB) torc_grid.nc | Target grid file (NEMO/ORCA2 grid) | ||
| tlmd_infile.nc (38.4 KB) tlmd_infile.nc | Input file (Lat/Lon regulagrid) |
RE: cdo remap with weights computed outside cdo ? - Added by Uwe Schulzweida about 10 years ago
Hi Olivier,
Your weight file contains a sea-land mask for the source grid. In this case CDO has to compare this mask with the input data. If this mask differ then new weights have to be computed because the weights are only valid for the same masks.
CDO generates a mask from the input data via missing values. Your input data does not have missing data.
Here is an example which apply the source grid mask selected from your weight file to the data file:
cdo selname,src_grid_imask rmp_tlmd_to_torc_mosaic.nc src_mask.nc cdo remap,torc_grid.nc,rmp_tlmd_to_torc_mosaic.nc -ifthen src_mask.nc tlmd_infile.nc result.nc
Best wishes,
Uwe
RE: cdo remap with weights computed outside cdo ? - Added by Olivier Marti about 10 years ago
Uwe,
Thank you for you help. It works now.
However, I'm still puzzled by the behavior of cdo. I'm successfull with :
cdo selname,src_grid_imask rmp_tlmd_to_torc_mosaic.nc tlmd_mask.nc cdo selname: Processed 9216 values from 19 variables over 1 timestep ( 0.03s ) + my_opt='-ifthen tlmd_mask.nc' + cdo remap,torc_grid.nc,rmp_tlmd_to_torc_mosaic.nc -ifthen tlmd_mask.nc tlmd_infile.nc t2m_mosaic_from_tlmd_to_torc.nc cdo remap: Started child process "ifthen tlmd_mask.nc tlmd_infile.nc (pipe1.1)". cdo(2) ifthen (Warning): Grids have different types! First grid: generic; second grid: lonlat cdo remap: Using remapnn cdo(2) ifthen: Processed 18432 values from 2 variables over 2 timesteps ( 0.08s ) cdo remap: Processed 9216 values from 1 variable over 1 timestep ( 0.21s )
It works and the result seems fine. The message "cdo remap: Using remapnn" is somewhat misleading for the user, because this is not a "remapnn" interpolation.
If I change the attribute "map_method" to "Conservative Remapping", I got :
cdo selname,src_grid_imask rmp_tlmd_to_torc_mosaic.nc tlmd_mask.nc cdo selname: Processed 9216 values from 19 variables over 1 timestep ( 0.03s ) + my_opt='-ifthen tlmd_mask.nc' + cdo remap,torc_grid.nc,rmp_tlmd_to_torc_mosaic.nc -ifthen tlmd_mask.nc tlmd_infile.nc t2m_mosaic_from_tlmd_to_torc.nc cdo remap: Started child process "ifthen tlmd_mask.nc tlmd_infile.nc (pipe1.1)". cdo(2) ifthen (Warning): Grids have different types! First grid: generic; second grid: lonlat cdo remap: Using remapcon cdf_put_vara_double : ncid = 196608 varid = 5 val0 = 25185619547684088346776155756601312018432.000000 cdf_put_vara_double : varname = t2m Error (cdf_put_vara_double) : NetCDF: Numeric conversion not representable
The result is totally wrong. It seems to be a problem with missing values.
This mean that cdo performs different computations, depending on the map_method attribute. This let me a little insecure about what computations are made by cdo, and what differs in this two cases. Can I trust the result ?
However, if I set "map_method" to "Bilinear", I obtain the same result than in the first case, which is a good news. And results is different to any interpolation made with cdo computed weights.
Best regards,
Olivier
RE: cdo remap with weights computed outside cdo ? - Added by Uwe Schulzweida about 10 years ago
Hi Olivier,
Yes, the message "cdo remap: Using remapnn" really misleading. I have removed it now!
In principle you are right, the map_method should not influence the result of the remap operator. But the conservative remapping is an exception. The remapping itself is the same but afterwards a normalization takes place. This normalization depends on the attribute normalization. If this attribute is set to "none" the normalization is dst_grid_frac*dst_grid_area.
Best regards,
Uwe
RE: cdo remap with weights computed outside cdo ? - Added by Olivier Marti about 10 years ago
Ok, thanks ! I think all is clear now.
Best regards,
Olivier