Project

General

Profile

Date range selection causes remap failure

Added by Brendan DeTracey about 3 years ago

I can't get my head around this one:

$ cdo -f nc4 -z zip --cmor --reduce_dim remap,grid_Omon_IPSL-CM6A-LR_t-grid_ess.nc,weights_Omon_IPSL-CM6A-LR_t-grid_ess.nc -reducegrid,mask_Omon_IPSL-CM6A-LR_t-grid.nc -select,startdate=2015-01-01T00:00:00,enddate=2049-12-31T23:59:59 zos_Omon_IPSL-CM6A-LR_ssp370_r4i1p1f1_gn_201501-201505.nc test.nc
cdo(1) reducegrid: Process started
cdo(2) select: Process started
cdo    remap: YAC first order conservative weights from unstructured (120184) to unstructured (1) grid
cdo(2) select: Processed 721104 values from 2 variables over 5 timesteps.
cdo(1) reducegrid: Processed 600920 values from 2 variables over 5 timesteps.
cdo    remap (Warning): Remap weights from weights_Omon_IPSL-CM6A-LR_t-grid_ess.nc not used, unstructured (120184) grid not found!
cdo    remap: Processed 600920 values from 1 variable over 5 timesteps [0.41s 121MB].

$ cdo -f nc4 -z zip --cmor --reduce_dim remap,grid_Omon_IPSL-CM6A-LR_t-grid_ess.nc,weights_Omon_IPSL-CM6A-LR_t-grid_ess.nc -reducegrid,mask_Omon_IPSL-CM6A-LR_t-grid.nc zos_Omon_IPSL-CM6A-LR_ssp370_r4i1p1f1_gn_201501-201505.nc test.nc
cdo(1) reducegrid: Process started
cdo    remap: YAC first order conservative weights from unstructured (120184) to unstructured (1) grid
cdo(1) reducegrid: Processed 721104 values from 2 variables over 5 timesteps.
cdo    remap: Processed 721104 values from 2 variables over 5 timesteps [0.37s 117MB].

Why does the first example fail?


Replies (5)

RE: Date range selection causes remap failure - Added by Ralf Mueller about 3 years ago

hi Brendan!

I can confirm this with cdo 1.9.9 and 1.9.10. select seems to miss some meta data in the stream. A work around is, that you create an intermediate file with

cdo -select,startdate=2015-01-01T00:00:00,enddate=2049-12-31T23:59:59 zos_Omon_IPSL-CM6A-LR_ssp370_r4i1p1f1_gn_201501-201505.nc seldate.nc
and use this file in the original call as a replacement:
cdo -f nc4 -z zip --cmor --reduce_dim remap,grid_Omon_IPSL-CM6A-LR_t-grid_ess.nc,weights_Omon_IPSL-CM6A-LR_t-grid_ess.nc -reducegrid,mask_Omon_IPSL-CM6A-LR_t-grid.nc seldate.nc test.nc

If you use python or ruby bindings you would not need to handle this yourself. tempfile creates is included - just an idea

hth
ralf

RE: Date range selection causes remap failure - Added by Uwe Schulzweida about 3 years ago

Hi Brendan,

The input file contains 2 variables (area and zos). From CDO's point of view, both have different grid information, because the variable area is missing the coordinate attribute.
The operator reducegrid makes two equal grids out of it. Since the two grids are the same, the operator select makes one grid out of them. This leads to the different messages.
In both cases the weights are recalculated. The grid in the weight file does not seem to match the grid in the data.
The maskfile also contains 2 variables. reducegrid uses only the first variable to reduce the data. The first variable is area and all values are 1, so nothing is reduced here.

Cheers,
Uwe

RE: Date range selection causes remap failure - Added by Brendan DeTracey about 3 years ago

Thanks guys. I'll strip area out before processing. I wish data publishers would do better at following standards instead of producing edge cases :(

RE: Date range selection causes remap failure - Added by Brendan DeTracey about 3 years ago

I spoke too soon. Deleting the area did not do it:

$ cdo -f nc4 -z zip --cmor --reduce_dim remap,grid_Omon_IPSL-CM6A-LR_t-grid_ess.nc,weights_Omon_IPSL-CM6A-LR_t-grid_ess.nc -reducegrid,mask_Omon_IPSL-CM6A-LR_t-grid.nc -delete,name=area -select,startdate=2015-01-01T00:00:00,enddate=2049-12-31T23:59:59 zos_Omon_IPSL-CM6A-LR_ssp370_r4i1p1f1_gn_201501-201505.nc test.nc
cdo(1) reducegrid: Process started
cdo(2) delete: Process started
cdo(3) select: Process started
cdo    remap: YAC first order conservative weights from unstructured (120184) to unstructured (1) grid
cdo(3) select: Processed 721104 values from 2 variables over 5 timesteps.
cdo(2) delete: Processed 600920 values from 2 variables over 5 timesteps.
cdo(1) reducegrid: Processed 600920 values from 1 variable over 5 timesteps.
cdo    remap (Warning): Remap weights from weights_Omon_IPSL-CM6A-LR_t-grid_ess.nc not used, unstructured (120184) grid not found!
cdo    remap: Processed 600920 values from 1 variable over 5 timesteps [0.50s 122MB].

RE: Date range selection causes remap failure - Added by Brendan DeTracey about 3 years ago

Please ignore last post. All good now.

    (1-5/5)