Date range selection causes remap failure
Added by Brendan DeTracey almost 4 years ago
I can't get my head around this one:
$ cdo -f nc4 -z zip --cmor --reduce_dim remap,grid_Omon_IPSL-CM6A-LR_t-grid_ess.nc,weights_Omon_IPSL-CM6A-LR_t-grid_ess.nc -reducegrid,mask_Omon_IPSL-CM6A-LR_t-grid.nc -select,startdate=2015-01-01T00:00:00,enddate=2049-12-31T23:59:59 zos_Omon_IPSL-CM6A-LR_ssp370_r4i1p1f1_gn_201501-201505.nc test.nc cdo(1) reducegrid: Process started cdo(2) select: Process started cdo remap: YAC first order conservative weights from unstructured (120184) to unstructured (1) grid cdo(2) select: Processed 721104 values from 2 variables over 5 timesteps. cdo(1) reducegrid: Processed 600920 values from 2 variables over 5 timesteps. cdo remap (Warning): Remap weights from weights_Omon_IPSL-CM6A-LR_t-grid_ess.nc not used, unstructured (120184) grid not found! cdo remap: Processed 600920 values from 1 variable over 5 timesteps [0.41s 121MB]. $ cdo -f nc4 -z zip --cmor --reduce_dim remap,grid_Omon_IPSL-CM6A-LR_t-grid_ess.nc,weights_Omon_IPSL-CM6A-LR_t-grid_ess.nc -reducegrid,mask_Omon_IPSL-CM6A-LR_t-grid.nc zos_Omon_IPSL-CM6A-LR_ssp370_r4i1p1f1_gn_201501-201505.nc test.nc cdo(1) reducegrid: Process started cdo remap: YAC first order conservative weights from unstructured (120184) to unstructured (1) grid cdo(1) reducegrid: Processed 721104 values from 2 variables over 5 timesteps. cdo remap: Processed 721104 values from 2 variables over 5 timesteps [0.37s 117MB].
Why does the first example fail?
Replies (5)
RE: Date range selection causes remap failure - Added by Ralf Mueller almost 4 years ago
hi Brendan!
I can confirm this with cdo 1.9.9 and 1.9.10. select
seems to miss some meta data in the stream. A work around is, that you create an intermediate file with
cdo -select,startdate=2015-01-01T00:00:00,enddate=2049-12-31T23:59:59 zos_Omon_IPSL-CM6A-LR_ssp370_r4i1p1f1_gn_201501-201505.nc seldate.ncand use this file in the original call as a replacement:
cdo -f nc4 -z zip --cmor --reduce_dim remap,grid_Omon_IPSL-CM6A-LR_t-grid_ess.nc,weights_Omon_IPSL-CM6A-LR_t-grid_ess.nc -reducegrid,mask_Omon_IPSL-CM6A-LR_t-grid.nc seldate.nc test.nc
If you use python or ruby bindings you would not need to handle this yourself. tempfile creates is included - just an idea
hth
ralf
RE: Date range selection causes remap failure - Added by Uwe Schulzweida almost 4 years ago
Hi Brendan,
The input file contains 2 variables (area and zos). From CDO's point of view, both have different grid information, because the variable area is missing the coordinate attribute.
The operator reducegrid makes two equal grids out of it. Since the two grids are the same, the operator select makes one grid out of them. This leads to the different messages.
In both cases the weights are recalculated. The grid in the weight file does not seem to match the grid in the data.
The maskfile also contains 2 variables. reducegrid uses only the first variable to reduce the data. The first variable is area and all values are 1, so nothing is reduced here.
Cheers,
Uwe
RE: Date range selection causes remap failure - Added by Brendan DeTracey almost 4 years ago
Thanks guys. I'll strip area out before processing. I wish data publishers would do better at following standards instead of producing edge cases
RE: Date range selection causes remap failure - Added by Brendan DeTracey almost 4 years ago
I spoke too soon. Deleting the area did not do it:
$ cdo -f nc4 -z zip --cmor --reduce_dim remap,grid_Omon_IPSL-CM6A-LR_t-grid_ess.nc,weights_Omon_IPSL-CM6A-LR_t-grid_ess.nc -reducegrid,mask_Omon_IPSL-CM6A-LR_t-grid.nc -delete,name=area -select,startdate=2015-01-01T00:00:00,enddate=2049-12-31T23:59:59 zos_Omon_IPSL-CM6A-LR_ssp370_r4i1p1f1_gn_201501-201505.nc test.nc cdo(1) reducegrid: Process started cdo(2) delete: Process started cdo(3) select: Process started cdo remap: YAC first order conservative weights from unstructured (120184) to unstructured (1) grid cdo(3) select: Processed 721104 values from 2 variables over 5 timesteps. cdo(2) delete: Processed 600920 values from 2 variables over 5 timesteps. cdo(1) reducegrid: Processed 600920 values from 1 variable over 5 timesteps. cdo remap (Warning): Remap weights from weights_Omon_IPSL-CM6A-LR_t-grid_ess.nc not used, unstructured (120184) grid not found! cdo remap: Processed 600920 values from 1 variable over 5 timesteps [0.50s 122MB].
RE: Date range selection causes remap failure - Added by Brendan DeTracey almost 4 years ago
Please ignore last post. All good now.