Project

General

Profile

Time and date for paleo-climate applications

Added by Mario Krapp over 6 years ago

Hi,

I'm working with paleo-climate data where the time axis extends beyond the usual dates as used in present-day climate data, for example. This becomes problematic if the paleo-climate records are continuous and I would like to use cdo to calculate averages, etc. For example, if I have paleo-climate data for the last 500,000 years with increments of 1000 years it would be nice to have the dates in a correct time/date format in my NetCDF files, e.g., -500000-01-01 00:00:00 -499000-01-01 00:00:00, etc.

Currently there is a limit in the allowed dates, which is set by INT_MAX/INT_MIN (see method encode_day in libcdi/src/calendar.c). On my machine INT_MAX is 2147483647 which would translate into a largest allowed year to be 214,748 (the last 4 digits are reserved for month and day, ie., MM-DD). Find attached a NetCDF file which explores this limitation/overflow beyond INT_MAX.

The output of cdo sinfo test_date.nc is

   File format : NetCDF4
    -1 : Institut Source   T Steptype Levels Num    Points Num Dtype : Parameter ID
     1 : unknown  unknown  v instant       1   1         1   1  F32  : -1            
   Grid coordinates :
     1 : generic                  : points=1
   Vertical coordinates :
     1 : surface                  : levels=1
   Time coordinate :  12 steps
     RefTime =  0000-01-01 00:00:00  Units = years  Calendar = 360_day
  YYYY-MM-DD hh:mm:ss  YYYY-MM-DD hh:mm:ss  YYYY-MM-DD hh:mm:ss  YYYY-MM-DD hh:mm:ss
  0000-01-01 00:00:00 21474-01-01 00:00:00 42949-01-01 00:00:00 64424-01-01 00:00:00
 85899-01-01 00:00:00 107374-01-01 00:00:00 128848-01-01 00:00:00 150323-01-01 00:00:00
 171798-01-01 00:00:00 193273-01-01 00:00:00 214748-01-01 00:00:00 -214747-71-95 00:00:00

If this limit would be defined by LONG_MAX (is 922337203685477 on my machine) instead of INT_MAX that could allow dates up to year 92,233,720,368, or reversely about 80 billion years before the big bang ;-)).

Would it be possible to allow for a larger range in the dates (int -> long)?

Best
Mario

test_date.nc (14.8 KB) test_date.nc A NetCDF file highlighting the overflow if years are beyond a certain threshold (e.g., INT_MAX)

Replies (3)

RE: Time and date for paleo-climate applications - Added by Uwe Schulzweida over 6 years ago

Hi Mario,

We have this topic with high priority on our list. Milliseconds are also missing. This needs a refactoring of the date/time container.
It is planed for the next CDO release end of Q1/2018.

Best,
Uwe

RE: Time and date for paleo-climate applications - Added by Mario Krapp almost 6 years ago

Hi Uwe,

Could you give a quick update about the current status of the time axis issue?

Thanks
Mario

RE: Time and date for paleo-climate applications - Added by Uwe Schulzweida almost 6 years ago

Hi Mario,

Sorry, I had to switch the priorities a little bit.
Dates with 64-bit integer will be available in the next release. The implementation is finish and we already have a working branch for this. A CDO prerelease with this feature will be available next week. A CDO executable for testing is available on mistral:/pf/zmaw/m214003/local/bin/cdo.

Cheers,
Uwe

    (1-3/3)