Project

General

Profile

sorting data based on time

Added by javad Babagolii over 3 years ago

Dear users,

I have a nc file while the data is not sorted by time. The data starts with one time step and then in the middle of the file another time step appears(please see the attached file) How can I sort lrf_time without any changes to other nc features?
I use this command cdo sorttimestamp gom_lwrad_era1.nc gom_lwrad_era2.nc but this led to a problem. This solve the problem but it damaged other variables in nc file.

please help me with your interesting comments.


Replies (12)

RE: sorting data based on time - Added by Ralf Mueller over 3 years ago

hi!

sorttimestamp is the only operator CDO offers for the job. So you ether use it and repair the the output afterwards or look for another tool.

sry for the bad news

cheers
ralf

RE: sorting data based on time - Added by javad Babagolii over 3 years ago

Thanks for your response.

You mean that there is not any solution for it in cdo?

RE: sorting data based on time - Added by javad Babagolii over 3 years ago

I use it for ocean modeling. After applying the revised file based on sorttimestamp, the model mentions that the grid is different for your domain. why does this change grid of the domain?

RE: sorting data based on time - Added by Ralf Mueller over 3 years ago

javad Babagolii wrote in RE: sorting data based on time:

Thanks for your response.

You mean that there is not any solution for it in cdo?

hard to say without the input file and the CDO version you use. older versions only took over the CF-compliant attributes. newer versions try to keep most of the stuff in place.

If you prove both I can say more

cheers
ralf

RE: sorting data based on time - Added by javad Babagolii over 3 years ago

Thank you so much for your response.

when it comes to version: Here are some features of my cdo in addition to my input file.

Climate Data Operators version 1.9.9rc1 (https://mpimet.mpg.de/cdo)
System: x86_64-pc-linux-gnu
CXX Compiler: g++ -g -O2 -fdebug-prefix-map=/build/cdo-51aH2m/cdo-1.9.9~rc1=. -fstack-protector-strong -Wformat -Werror=format-security -fopenmp
CXX version : g++ (Ubuntu 9.2.1-28ubuntu1) 9.2.1 20200203
C Compiler: gcc -g -O2 -fdebug-prefix-map=/build/cdo-51aH2m/cdo-1.9.9~rc1=. -fstack-protector-strong -Wformat -Werror=format-security -Wall -pedantic -fPIC -fopenmp
C version : gcc (Ubuntu 9.2.1-28ubuntu1) 9.2.1 20200203
F77 Compiler: f77 -g -O2 -fdebug-prefix-map=/build/cdo-51aH2m/cdo-1.9.9~rc1=. -fstack-protector-strong
F77 version : unknown
Features: 5GB 4threads C++14 Fortran DATA PTHREADS OpenMP HDF5 NC4/HDF5/threadsafe OPeNDAP SZ UDUNITS2 PROJ MAGICS CURL FFTW3 SSE2
Libraries: HDF5/1.10.4 proj/6.3.0 curl/7.68.0(h7.66.0)
Filetypes: srv ext ieg grb1 grb2 nc1 nc2 nc4 nc4c nc5
CDI library version : 1.9.9rc1
ecCodes library version : 2.16.0
NetCDF library version : 4.7.3 of Jan 23 2020 04:18:16 $
hdf5 library version : library undefined
exse library version : 1.4.1
FILE library version : 1.8.3

RE: sorting data based on time - Added by Ralf Mueller over 3 years ago

hi!

I sorted the file and uploaded it here. I cannot see any differences in terms of meta data, so you might download it and give it a try with the model.

RE: sorting data based on time - Added by Ralf Mueller over 3 years ago

in fact I could not find the time dimension un-sorted in your upload

RE: sorting data based on time - Added by javad Babagolii over 3 years ago

Thank you so much for your help.

Unfortunately, this could not solve the problem. I have checked it. Here is the error of the model.

REGRID - input gridded data does not contain model grid:

Gridded:  LonMin =   35.0000 LonMax =   41.0000
LatMin = 48.0000 LatMax = 55.0000
Model: LonMin = 48.5000 LonMax = 54.1875
LatMin = 36.0000 LatMax = 40.5133
Found Error: 04 Line: 269 Source: ROMS/Utility/get_2dfld.F
GET_2DFLD   - error while reading variable: lwrad_down   at TIME index =       1

RE: sorting data based on time - Added by Ralf Mueller over 3 years ago

I compared the gom_lwrad_era1.nc with my sorted version with ncdump. apart from the CDO version used and the history attribute, both files are identical in data and coordinates. So I really don't understand, why your model complains about a difference coming from the file. in-fact there is none.

are you sure the model reads the file you want it to read?

RE: sorting data based on time - Added by javad Babagolii over 3 years ago

Thanks for your help.

Yes. The model reads the files and then faces the problem. I think that it is predictable because the volume of the sorted file is about 20 Meg less than the original file.

Usually, I have followed this method to prepare data for the model. I have downloaded data from ecmwf for each month. Then I use megtime in cdo to merge files for 1 year. After that, I use Matlab code to make forcing files.

Is there any option to work with merge time in cdo to sort data based on time simultaneously?

cdo -b F64 -f nc2 mergetime .......

RE: sorting data based on time - Added by Ralf Mueller over 3 years ago

I think we talk about different thing:

  1. the file you uploaded (gom_lwrad_era1.nc) is 63MB
  2. I processed it with cdo -sorttimestamp... to a new file gom_lwrad_era1-sorted.nc also 63MB
  3. I compared both files with CDO: no differences
  4. I compared files using a text version printing all data with vimdiff
    • ncdump gom_lwrad_era1.nc > gom_lwrad_era1.dump
    • ncdump gom_lwrad_era1-sorted.nc > gom_lwrad_era1-sorted.dump
    • vimdiff gom_lwrad_era1.dump gom_lwrad_era1-sorted.dump shows no difference except CDO-version and history stuff (completely harmless)

So there is no difference as far as I can tell. I just uploaded a compressed version to save some bandwidth.

Before we go on I need to understand your problem. Currently I don't understand what's happening, sry.

RE: sorting data based on time - Added by javad Babagolii over 3 years ago

Dear Dr. Ralf Mueller
__

Thank you so much for your suggestions.

You are right!

I think that I have changed some parts of the file and then submit to you. Maybe, this is the main of the problem. Now, I created the original file again.

Please apply your method to this file.

Best,

    (1-12/12)