Time and date for paleo-climate applications
Added by Mario Krapp about 7 years ago
Hi,
I'm working with paleo-climate data where the time axis extends beyond the usual dates as used in present-day climate data, for example. This becomes problematic if the paleo-climate records are continuous and I would like to use cdo
to calculate averages, etc. For example, if I have paleo-climate data for the last 500,000 years with increments of 1000 years it would be nice to have the dates in a correct time/date format in my NetCDF files, e.g., -500000-01-01 00:00:00 -499000-01-01 00:00:00, etc.
Currently there is a limit in the allowed dates, which is set by INT_MAX/INT_MIN (see method encode_day
in libcdi/src/calendar.c
). On my machine INT_MAX
is 2147483647 which would translate into a largest allowed year to be 214,748 (the last 4 digits are reserved for month and day, ie., MM-DD). Find attached a NetCDF file which explores this limitation/overflow beyond INT_MAX.
The output of cdo sinfo test_date.nc
is
File format : NetCDF4 -1 : Institut Source T Steptype Levels Num Points Num Dtype : Parameter ID 1 : unknown unknown v instant 1 1 1 1 F32 : -1 Grid coordinates : 1 : generic : points=1 Vertical coordinates : 1 : surface : levels=1 Time coordinate : 12 steps RefTime = 0000-01-01 00:00:00 Units = years Calendar = 360_day YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss 0000-01-01 00:00:00 21474-01-01 00:00:00 42949-01-01 00:00:00 64424-01-01 00:00:00 85899-01-01 00:00:00 107374-01-01 00:00:00 128848-01-01 00:00:00 150323-01-01 00:00:00 171798-01-01 00:00:00 193273-01-01 00:00:00 214748-01-01 00:00:00 -214747-71-95 00:00:00
If this limit would be defined by LONG_MAX
(is 922337203685477 on my machine) instead of INT_MAX
that could allow dates up to year 92,233,720,368, or reversely about 80 billion years before the big bang ;-)).
Would it be possible to allow for a larger range in the dates (int
-> long
)?
Best
Mario
test_date.nc (14.8 KB) test_date.nc | A NetCDF file highlighting the overflow if years are beyond a certain threshold (e.g., INT_MAX) |
Replies (3)
RE: Time and date for paleo-climate applications - Added by Uwe Schulzweida about 7 years ago
Hi Mario,
We have this topic with high priority on our list. Milliseconds are also missing. This needs a refactoring of the date/time container.
It is planed for the next CDO release end of Q1/2018.
Best,
Uwe
RE: Time and date for paleo-climate applications - Added by Mario Krapp over 6 years ago
Hi Uwe,
Could you give a quick update about the current status of the time axis issue?
Thanks
Mario
RE: Time and date for paleo-climate applications - Added by Uwe Schulzweida over 6 years ago
Hi Mario,
Sorry, I had to switch the priorities a little bit.
Dates with 64-bit integer will be available in the next release. The implementation is finish and we already have a working branch for this. A CDO prerelease with this feature will be available next week. A CDO executable for testing is available on mistral:/pf/zmaw/m214003/local/bin/cdo.
Cheers,
Uwe