remapcon failing?
Added by Brendan DeTracey over 4 years ago
With all the help I have graciously received, I finally reached my destination, but with a couple bumps. I have four polygons I have encoded as SCRIP unstructured grids to be used with remapcon
. Two of them work, and two of them fail.
$ cdo remapcon,cdo_scrip_grid_gom.nc -reducegrid,zos_Omon_ACCESS-CM2_historical_r1i1p1f1_mask_185008.nc [ zos_Omon_ACCESS-CM2_historical_r1i1p1f1_gn_185008.nc ] test.nc cdo(1) reducegrid: Process started cdo remapcon: YAC first order conservative weights from unstructured (69782) to unstructured (1) grid cdo(1) reducegrid: Processed 108000 values from 1 variable over 1 timestep. cdo remapcon: Processed 69782 values from 1 variable over 1 timestep [0.25s 79MB]. (cmip6) [brendan@hecla:remapcon_demo]$ cdo remapcon,cdo_scrip_grid_wss.nc -reducegrid,zos_Omon_ACCESS-CM2_historical_r1i1p1f1_mask_185008.nc [ zos_Omon_ACCESS-CM2_historical_r1i1p1f1_gn_185008.nc ] test.nc cdo(1) reducegrid: Process started cdo remapcon: YAC first order conservative weights from unstructured (69782) to unstructured (1) grid cdo(1) reducegrid: Processed 108000 values from 1 variable over 1 timestep. cdo remapcon: Processed 69782 values from 1 variable over 1 timestep [0.23s 79MB]. (cmip6) [brendan@hecla:remapcon_demo]$ cdo remapcon,cdo_scrip_grid_css.nc -reducegrid,zos_Omon_ACCESS-CM2_historical_r1i1p1f1_mask_185008.nc [ zos_Omon_ACCESS-CM2_historical_r1i1p1f1_gn_185008.nc ] test.nc cdo(1) reducegrid: Process started cdo remapcon: YAC first order conservative weights from unstructured (69782) to unstructured (1) grid ERROR: invalid cell Aborting in file clipping.c, line 1305 ... (cmip6) [brendan@hecla:remapcon_demo]$ cdo remapcon,cdo_scrip_grid_ess.nc -reducegrid,zos_Omon_ACCESS-CM2_historical_r1i1p1f1_mask_185008.nc [ zos_Omon_ACCESS-CM2_historical_r1i1p1f1_gn_185008.nc ] test.nc cdo(1) reducegrid: Process started cdo remapcon: YAC first order conservative weights from unstructured (69782) to unstructured (1) grid ERROR: invalid cell Aborting in file clipping.c, line 1305 ...
All four unstructured grids were produced with the exact same code. Corner points are counter-clockwise. The plot below shows all four polygons. The black dots are the source grid corners. The red dots are the source grid centers of unmasked cells.
Why does the remap fail for two of the target grids? There are viable source grid cells that intersect.
Replies (15)
RE: remapcon failing? - Added by Brendan DeTracey over 4 years ago
Ah. I see that the relevant code is YAC not cdo.
RE: remapcon failing? - Added by Brendan DeTracey over 4 years ago
I compiled cdo setting #define YAC_VERBOSE_CLIPPING
in src/lib/yac/clipping.c
. I hoped it would help me debug but no extra information was printed.
I noticed by mistake that an older cdo (1.9.3) worked:
$ cdo -V Climate Data Operators version 1.9.3 (http://mpimet.mpg.de/cdo) CXX Compiler: g++ -g -O2 -fdebug-prefix-map=/build/cdo-g3Qjnd/cdo-1.9.3+dfsg.1=. -fstack-protector-strong -Wformat -Werror=format-security -fopenmp CXX version : g++ (Ubuntu 7.3.0-1ubuntu1) 7.3.0 C Compiler: gcc -g -O2 -fdebug-prefix-map=/build/cdo-g3Qjnd/cdo-1.9.3+dfsg.1=. -fstack-protector-strong -Wformat -Werror=format-security -Wall -pedantic -fPIC -fopenmp C version : gcc (Ubuntu 7.3.0-1ubuntu1) 7.3.0 F77 Compiler: f77 -g -O2 -fdebug-prefix-map=/build/cdo-g3Qjnd/cdo-1.9.3+dfsg.1=. -fstack-protector-strong F77 version : unknown Features: 62GB C++14 Fortran DATA PTHREADS OpenMP45 HDF5 NC4/HDF5/threadsafe OPeNDAP SZ UDUNITS2 PROJ.4 MAGICS CURL FFTW3 SSE2 Libraries: HDF5/1.10.0 proj/4.93 curl/7.58.0 Filetypes: srv ext ieg grb1 grb2 nc1 nc2 nc4 nc4c nc5 CDI library version : 1.9.3 GRIB_API library version : 2.6.0 NetCDF library version : 4.6.0 of Feb 9 2018 19:21:24 $ HDF5 library version : library undefined EXSE library version : 1.4.0 FILE library version : 1.8.3 $ cdo reducegrid,zos_Omon_ACCESS-CM2_historical_r1i1p1f1_mask_185008.nc zos_Omon_ACCESS-CM2_historical_r1i1p1f1_gn_185008.nc reduce.nc cdo reducegrid: Processed 108000 values from 1 variable over 1 timestep [0.08s 85MB] $ cdo remapcon,cdo_scrip_grid_ess.nc reduce.nc test.nc cdo remapcon: SCRIP first order conservative weights from unstructured (69782) to unstructured (1) grid cdo remapcon: Map weight > 1! grid1idx=55259 grid2idx=0 nlink=0 wts=1.13276 cdo remapcon: Map weight > 1! grid1idx=55430 grid2idx=0 nlink=3 wts=1.14521 cdo remapcon: Map weight > 1! grid1idx=55431 grid2idx=0 nlink=4 wts=1.153 cdo remapcon: Map weight < 0! grid1idx=55600 grid2idx=0 nlink=6 wts=-0.154891 cdo remapcon: Map weight < 0! grid1idx=55762 grid2idx=0 nlink=7 wts=-0.367269 cdo remapcon: Map weight < 0! grid1idx=55761 grid2idx=0 nlink=8 wts=-0.832767 cdo remapcon: Map weight < 0! grid1idx=55923 grid2idx=0 nlink=9 wts=-0.358429 cdo remapcon: Map weight < 0! grid1idx=55922 grid2idx=0 nlink=10 wts=-0.960929 cdo remapcon: Map weight < 0! grid1idx=55760 grid2idx=0 nlink=11 wts=-0.236372 cdo remapcon: Map weight < 0! grid1idx=55596 grid2idx=0 nlink=12 wts=-1.16865 cdo remapcon: Processed 69782 values from 1 variable over 1 timestep [0.17s 66MB]
RE: remapcon failing? - Added by Brendan DeTracey over 4 years ago
Looks like my target polygons were too complicated. I simplified them and the problem went away.
RE: remapcon failing? - Added by Brendan DeTracey over 4 years ago
I spoke too soon. YAC still fails with "invalid cell" on some very simply polygons. The relevant function in YAC clipping.c
is:
static int get_cell_points_ordering(struct point_list * cell) { if ((cell->start == NULL) || (cell->start == cell->start->next)) yac_internal_abort_message("ERROR: invalid cell\n", __FILE__, __LINE__); double norm_vec[3]; struct point_list_element * curr = cell->start; compute_norm_vector(curr->vec_coords, curr->next->vec_coords, norm_vec); curr = curr->next; if (curr->next == cell->start) yac_internal_abort_message("ERROR: invalid cell\n", __FILE__, __LINE__); do { curr = curr->next; double dot = dotproduct(curr->vec_coords, norm_vec); if (fabs(dot) > tol) return dot > 0; } while (curr != cell->start); return -1; }
Any advice on what this means and/or how to follow this issue up with YAC maintainers?
RE: remapcon failing? - Added by Brendan DeTracey over 4 years ago
I forgot to include the simplified polygon and its ncdump:
]$ ncdump cdo_scrip_grid_ess.nc netcdf cdo_scrip_grid_ess { dimensions: grid_size = 1 ; grid_corners = 6 ; grid_rank = 1 ; variables: int grid_dims(grid_rank) ; double grid_center_lat(grid_size) ; grid_center_lat:units = "degrees" ; double grid_center_lon(grid_size) ; grid_center_lon:units = "degrees" ; double grid_corner_lat(grid_size, grid_corners) ; grid_corner_lat:units = "degrees" ; double grid_corner_lon(grid_size, grid_corners) ; grid_corner_lon:units = "degrees" ; int grid_imask(grid_size) ; grid_imask:units = "unitless" ; // global attributes: :title = "cdo_scrip_grid_ess.nc" ; data: grid_dims = 1 ; grid_center_lat = 44.5752793657899 ; grid_center_lon = -59.8509057269732 ; grid_corner_lat = 44.9239310821268, 42.9987809907331, 43.7583337517241, 44.3104173896552, 43.9447807410383, 44.5916675448276 ; grid_corner_lon = -62.0002590886872, -62.0003759902259, -59.1340888451411, -59.3749980689655, -58.8416644413793, -57.1945735855501 ; grid_imask = 1 ; }
cdo_scrip_grid_ess.nc (684 Bytes) cdo_scrip_grid_ess.nc | Simplified version |
RE: remapcon failing? - Added by Brendan DeTracey over 4 years ago
clipping.c
is failing on the target grid in this part of the code:
curr = curr->next; if (curr->next == cell->start) yac_internal_abort_message("ERROR: invalid cell\n", __FILE__, __LINE__);
I do not understand why the pointers are the same.
RE: remapcon failing? - Added by Brendan DeTracey over 4 years ago
Error originates in a call to yac_cell_clipping
in cdo_compute_concave_overlap_areas
in remap_conserv.cc
RE: remapcon failing? - Added by Brendan DeTracey over 4 years ago
Somebody please shoot me... or take mercy on me and delete this thread...
I screwed up a path to my target grids. It was reusing the old ones which led me on this merry chase today.
RE: remapcon failing? - Added by Ralf Mueller about 4 years ago
hi Brendan! no problem, I can delete whatever you want here. But take a little pity on yourself - you cannot imaging how often I tricked myself with mistakes by this.
IMO the only way to make this beneficial is to think about, how to prevent it in the future. Any have some coffee in the meantime
RE: remapcon failing? - Added by Brendan DeTracey about 4 years ago
Thanks Ralf. My darth mater(pun intended) thanks you too...
The internet, where mistakes live forever, but only if one bothers to search for them. A metaphor for life.
RE: remapcon failing? - Added by Ralf Mueller about 4 years ago
alright - then for your peace and violation of the 3rd law of thermodynamics I will reduce entropy by removing this thread later today
RE: remapcon failing? - Added by Brendan DeTracey about 4 years ago
Interesting. I was suggesting that the thread not be deleted...
Why should bots have all the fun?
RE: remapcon failing? - Added by Ralf Mueller about 4 years ago
you are so damn right.
So whats your conclusion of 2020: Alt-F4 or Ctrl-Alt-Del ?