Project

General

Profile

Problems with rounded numbers (?) when using divc/mulc...

Added by Morten Larsen over 13 years ago

Hello

I am calculating seasonal precipitation means from my climate model outputs over a region (252x252 cells) using the following series of formula:

selmon,1,2,12 - e.g. for choosing the dec-feb season.
selyear,2008,2009 - to choose a period
timsum - to sum the precipitation
divc,181(number of days in the period) - to have the average daily precipitation.

When mapping I found that all cells over my area of interest where nice looking number e.g. 2.000, 1.500, 1.225 etc. but most often the same over larger areas creating maps with virually no variation. I therefore went back and investigated the original output data, which for the hourly outputs had large variations with many decimals e.g. 0.093750, 0.15625, 0.12500, 0.21875, 0.37500, 0.28125 (storm event) for a specific cell I choose for investigation.

In the above example the result after 'timsum' is 384mm (reasonable result). However, when i divide by the number of days the result is = 2.000, which is wrong - it is really 2.1215. Also all the nabour cells had a value of 2.000. AND, when using the same row of calculations after selecting that one cell to begin with, the result is 2.1215.

Therefore; what is going on? Is there some sort of smoothing/interpolation over the area or?

The same pattern is visible when I use mulc,0.0055 (the same as divc,181).

Thanks

Morten


Replies (4)

RE: Problems with rounded numbers (?) when using divc/mulc... - Added by Ralf Mueller over 13 years ago

Hi Morten!
Could you provide the original data file?

regards
ralf

RE: Problems with rounded numbers (?) when using divc/mulc... - Added by Morten Larsen over 13 years ago

I have attached a file here. It is only 6 months and a smaller domain run, due to attachment size restrictions, but the problem should still be there...

I am now experimenting with taking smaller parts of the domain using the selindexbox, which seems to give non-rounded numbers.

Thanks!

Morten

RE: Problems with rounded numbers (?) when using divc/mulc... - Added by Uwe Schulzweida over 13 years ago

Thanks for the example file. Your GRIB records are very high compressed, there are only 6 bits for each value. If you modify this GRIB file with CDO then the result will also compressed with 6 bits per value. In general it is not recommended to use the GRIB format to store results of expressions. If you realy need GRIB to store the result you have to chose at least 16 bits for each value. I would recommed to use the CDO option '-b 32' for all GRIB records modified with CDO:

cdo -b 32 mulc,0.0055 ifile ofile
'-b 32' tries to store the result in 32 bit floating point if possible. This is not possible in GRIB1, so this option will be converted to packed 16 bit.

Best regards,
Uwe

RE: Problems with rounded numbers (?) when using divc/mulc... - Added by Morten Larsen over 13 years ago

Hey Uwe

That indeed solved the problem - thanks alot for pointing out where the error was.

Morten

    (1-4/4)