Float precision loss in coordinate variables
Added by Felix Dietzsch about 6 years ago
Hi,
I try to map a dataset on a regular lonlat grid of 0.5° resolution onto a grid of 0.05° resolution using the remapycon operator. However, after regridding the new coordinate variables 'lon' and 'lat' look like this:
[..., -10.975, -10.925, -10.875, -10.825, -10.775, -10.725, -10.675, -10.625, -10.575, -10.525, -10.475, -10.425, -10.375, -10.325, -10.275, -10.225, -10.175, -10.125, -10.075, -10.025, -9.97499999999999, -9.92499999999998, -9.87499999999997, -9.82499999999999, -9.77499999999998, -9.72499999999999, -9.67499999999998, -9.62499999999997, -9.57499999999999, -9.52499999999998, -9.47499999999999, -9.42499999999998, -9.37499999999997, -9.32499999999999, -9.27499999999998, -9.22499999999999, -9.17499999999998, -9.12499999999997, -9.07499999999999, -9.02499999999998, -8.97499999999999, -8.92499999999998, -8.87499999999997, -8.82499999999999, -8.77499999999998, -8.72499999999999, -8.67499999999998, -8.62499999999997, -8.57499999999999, -8.52499999999998, -8.47499999999999, -8.42499999999998, -8.37499999999997, -8.32499999999999, -8.27499999999998, -8.22499999999999, -8.17499999999998, -8.12499999999997, -8.07499999999999, -8.02499999999998, ...]
using ncdump -v lat,lon ... The coordinate variables are supposed to have the 'double' type. Using C_format or FORTRAN_format seems not suitable for me. I'd prefer to have the plain data to be correct. Is there any way to get the precise values without manually overriding the data?
Replies (6)
RE: Float precision loss in coordinate variables - Added by Karin Meier-Fleischer about 6 years ago
Hi Felix,
I tried to reproduce the problem but if I use a 0.5 degree grid as input and remap it to a 0.05 degree grid the values are ok.
Generate a 0.5 degrees file using the topo operator and remapbil
cdo -f nc -remapbil,r720x360 -topo topo_720x360.ncncdump -v lon topo_720x360.nc
lon = 0, 0.5, 1, 1.5, 2, 2.5, 3, 3.5, 4, 4.5, 5, 5.5, 6, 6.5, 7, 7.5, 8, 8.5, 9, 9.5, 10, 10.5, 11, 11.5, 12, 12.5, 13, 13.5, 14, 14.5, 15, 15.5, 16, 16.5, 17, 17.5, 18, 18.5, 19, 19.5, 20, 20.5, 21, 21.5, 22, 22.5, 23, 23.5, 24, 24.5, 25, 25.5, 26, 26.5, 27, 27.5, 28, 28.5, 29, 29.5, 30, 30.5, 31, 31.5, 32, 32.5, 33, 33.5, 34, 34.5, 35, 35.5, 36, 36.5, 37, 37.5, 38, 38.5, 39, 39.5, 40, 40.5, 41, 41.5, 42, 42.5, 43, 43.5, 44, 44.5, 45, 45.5, 46, 46.5, 47, 47.5, 48, 48.5, 49, 49.5, 50, 50.5, 51, 51.5, 52, 52.5, 53, 53.5, 54, 54.5, 55, 55.5, 56, 56.5, 57, 57.5, 58, ...
Remap the grid to 0.05 degrees grid
cdo -f nc -remapycon,r7200x3600 topo_720x360.nc topo_7200x3600.ncncdump -v lon topo_7200x3600.nc
lon = 0, 0.05, 0.1, 0.15, 0.2, 0.25, 0.3, 0.35, 0.4, 0.45, 0.5, 0.55, 0.6, 0.65, 0.7, 0.75, 0.8, 0.85, 0.9, 0.95, 1, 1.05, 1.1, 1.15, 1.2, 1.25, 1.3, 1.35, 1.4, 1.45, 1.5, 1.55, 1.6, 1.65, 1.7, 1.75, 1.8, 1.85, 1.9, 1.95, 2, 2.05, 2.1, 2.15, 2.2, 2.25, 2.3, 2.35, 2.4, 2.45, 2.5, 2.55, 2.6, 2.65, 2.7, 2.75, 2.8, 2.85, 2.9, 2.95, 3, 3.05, 3.1, 3.15, 3.2, 3.25, 3.3, 3.35, 3.4, 3.45, 3.5, 3.55, 3.6, 3.65, 3.7, 3.75, 3.8, 3.85, 3.9, 3.95, 4, 4.05, 4.1, 4.15, 4.2, 4.25, 4.3, 4.35, 4.4, 4.45, 4.5, 4.55, 4.6, 4.65, 4.7, 4.75, 4.8, 4.85, 4.9, 4.95, 5, 5.05, 5.1, 5.15, 5.2, 5.25, 5.3, 5.35, 5.4, 5.45, 5.5, 5.55, 5.6, 5.65, 5.7, 5.75, 5.8, 5.85, 5.9, 5.95, 6, 6.05, 6.1, 6.15, 6.2, 6.25, 6.3, 6.35, 6.4, 6.45, 6.5, 6.55, 6.6, 6.65, 6.7, 6.75, 6.8, 6.85, 6.9, 6.95, 7, 7.05, 7.1, 7.15, 7.2, 7.25, 7.3, 7.35, 7.4, 7.45, 7.5, 7.55, 7.6, 7.65, 7.7, 7.75, ...
What kind of values have the coordinate variables lon and lat? Can you upload the file?
-Karin
RE: Float precision loss in coordinate variables - Added by Felix Dietzsch about 6 years ago
Hi Karin, see the lonlat description file attached. The 0.5° grid looks the same, but - as said - with 0.5° differences.
RE: Float precision loss in coordinate variables - Added by Karin Meier-Fleischer about 6 years ago
The grid description is not enough, please, upload the input data file and the CDO call you use for remapping.
RE: Float precision loss in coordinate variables - Added by Felix Dietzsch about 6 years ago
Okay, here are the data, the command used was:
`cdo -z zip_9 -P 6 remapycon,r3.grid hoaps-g.r30.d01.wvpa.2011-07-01.nc hoaps-g.r3.d01.wvpa.2011-07-01.nc`
RE: Float precision loss in coordinate variables - Added by Uwe Schulzweida about 6 years ago
The default number of significant digits for double precision floats in ncdump is 15.
That means you will see only the first 15 significant digits in the output of ncdump.
If you increase this number to e.g. 20, you will see that nearly all values are not exact because most floating point number can't be represented exactly.
You have to decrease the number of significant digits to e.g. 9 if you want to see only 9 digits:
ncdump -v lat,lon -p 7,9 infile
RE: Float precision loss in coordinate variables - Added by Berrel Tiper about 6 years ago
great post. thnank