CDO

Hi Alonso,

without the data it is hard to say what is going wrong. Can you upload a test file with one or two time steps? Or, if not possible, send the output of 'ncdump -h'.

- You can have a look at the units attribute of lat (degrees_north) and lon (degrees_east).
- Select one variable and do the remapping, because you have 4 horizontal grids in your data file.

-Karin

Hi Alonso,

Your problem seems to be that lat and lon are variables, not coordinates.

There is a way to fix this with CDO, but I cannot remember it of the top of my head.

However, it is relatively easy to fix in NCO using ncatted. The following will probably solve your problem. You will need to adjust it to add coordinates for all variables.

ncatted -a coordinates,agno,c,c,'lon lat' infile outfile

Regards

Robert Wilson

Thanks Karin and Robert!

Karin: I attach a simplified version of the file, with just lat, lon, agno, mes and ET as variables.

Note that agno and mes corresponds to year and month in spanish.

I have no access to NCO, I have not install it yet, hope to do so!
(Robert: If you could show me a simple way of install it in Windows would be very helpful!)

Thank you very much!
Best!
Alonso A M

Hi Alonso

I think that setattribute is the CDO method you want to try. Something like:

cdo -setattribute,agno@coordinates="lat lon" infile outfile

Unfortunately I do not use Windows, so can offer not advice on setting up NCO.

Regards

Robert

Hi Robert,
Thanks for your response.

I have already use the "setattribute" operator, and a I get something like this:

File format : NetCDF
    -1 : Institut Source   T Steptype Levels Num    Points Num Dtype : Parameter name
     1 : unknown  unknown  c instant       1   1       444   1  F64  : agno     
     2 : unknown  unknown  c instant       1   1       444   1  F64  : mes      
     3 : unknown  unknown  c instant       1   2     88000   2  F32  : ET       
   Grid coordinates :
     1 : generic                  : points=444
     2 : generic                  : points=88000 (400x220)
   Vertical coordinates :
     1 : surface                  : levels=1
     2 : generic                  : levels=1
cdo sinfon: Processed 3 variables ( 0.03s 57MB )

Then when I try to use "sellonlatbox" operator, I get this error message:

cdo sellonlatbox: Unsupported grid type: generic

cdo sellonlatbox (Abort): Unsupported grid type!

Maybe it is a problem of the coordinates..?

I am trying to use NCO in Cygwin64 in Windows, but I have not being able to do so! :/

Thanks!

Regards,
Alonso.

My guess is that you now need to use setgrid to change the grid.

What does "cdo griddes infile" give you?

Robert

Ok, this took me a while but you need to install NCO (see http://nco.sourceforge.net/#Executables).

The netCDF file is not following the CF-convention, I would say it follows none. CDO is following the CF-convention and needs dimensions like time, lat, lon with the appropriate units. And if the dimension names are not lat and lon the variables have to have a coordinates attribute which leads CDO to the grid dimensions. Furthermore, the variable dimensions should be (time,lat,lon) or (time,lev,lat,lon).

ncdump -h regionalizacion_1979_2015_ET_sel.nc

netcdf regionalizacion_1979_2015_ET_sel {
dimensions:
    dim_lat = 400 ;
    dim_lon = 220 ;
    dim_time = 444 ;
    dim_time_2 = 1 ;
variables:
    double lat(dim_lat) ;
        lat:Latitud = "norte-sur" ;
    double lon(dim_lon) ;
        lon:Longitud = "oeste-este" ;
    double agno(dim_time) ;
        agno:inicio = "1979/01" ;
        agno:fin = "2015/12" ;
    double mes(dim_time) ;
    float ET(dim_time_2, dim_lon, dim_lat) ;
        ET:Long_name = "ET_sin_riego" ;
        ET:unidades = "[mm/d]" ;

// global attributes:
        :CDI = "Climate Data Interface version 1.9.2 (http://mpimet.mpg.de/cdi)" ;
        :Conventions = "CF-1.6" ;
        :history = "Mon Jan 06 15:41:25 2020: cdo select,name=lat,lon,agno,mes,ET,level=1 regionalizacion_1979_2015.nc regionalizacion_1979_2015_ET_sel.nc" ;
        :CDO = "Climate Data Operators version 1.9.2 (http://mpimet.mpg.de/cdo)" ;
}

1. Reorder the dimensions of ET with NCO's ncpdq

ncpdq -O -a dim_time_2,dim_lat,dim_lon $infile tmp.nc

Result:

dimensions:
    dim_lat = 400 ;
    dim_lon = 220 ;
    dim_time = 444 ;
    dim_time_2 = 1 ;
variables:
    double lat(dim_lat) ;
        lat:Latitud = "norte-sur" ;
    double lon(dim_lon) ;
        lon:Longitud = "oeste-este" ;
    double agno(dim_time) ;
        agno:inicio = "1979/01" ;
        agno:fin = "2015/12" ;
    double mes(dim_time) ;
    float ET(dim_time_2, dim_lat, dim_lon) ;
        ET:Long_name = "ET_sin_riego" ;
        ET:unidades = "[mm/d]" ;

2. To make it easier write the attribute settings to an ASCII file, first.

cat << EOF > attr_file.txt
ET@standard_name="ET_sin_riego" 
ET@coordinates="lon lat" 
ET@units="mm/d" 
lat@units="degrees_north" 
lat@standard_name="latitude" 
lat@long_name="latitude" 
lon@units="degrees_east" 
lon@standard_name="longitude" 
lon@long_name="longitude" 
EOF

3. Change the set of attributes for ET, lat, and lon.

cdo -setattribute,FILE=attr_file.txt tmp.nc tmp2.nc

Result:

dimensions:
    dim_lat = 400 ;
    dim_lon = 220 ;
    dim_time = 444 ;
    dim_time_2 = 1 ;
variables:
    double lat(dim_lat) ;
        lat:Latitud = "norte-sur" ;
        lat:units = "degrees_north" ;
        lat:standard_name = "latitude" ;
        lat:long_name = "latitude" ;
    double lon(dim_lon) ;
        lon:Longitud = "oeste-este" ;
        lon:units = "degrees_east" ;
        lon:standard_name = "longitude" ;
        lon:long_name = "longitude" ;
    double agno(dim_time) ;
        agno:inicio = "1979/01" ;
        agno:fin = "2015/12" ;
    double mes(dim_time) ;
    float ET(dim_time_2, dim_lat, dim_lon) ;
        ET:Long_name = "ET_sin_riego" ;
        ET:unidades = "[mm/d]" ;
        ET:standard_name = "ET_sin_riego" ;
        ET:coordinates = "lon lat" ;
        ET:units = "mm/d" ;

4. Select the correct grid, sellonlatbox, and change the time dimension name. In the original netCDF file
the dim_time_2 variable doesn't exist, that's why it's deleted, here.

cdo --reduce_dim -sellonlatbox,-75,-70,-30,-20 -selgrid,2 tmp2.nc tmp3.nc

Result:

netcdf tmp3 {
dimensions:
    lon = 100 ;
    lat = 200 ;
variables:
    double lon(lon) ;
        lon:standard_name = "longitude" ;
        lon:long_name = "longitude" ;
        lon:units = "degrees_east" ;
        lon:axis = "X" ;
    double lat(lat) ;
        lat:standard_name = "latitude" ;
        lat:long_name = "latitude" ;
        lat:units = "degrees_north" ;
        lat:axis = "Y" ;
    float ET(lat, lon) ;
        ET:standard_name = "ET_sin_riego" ;
        ET:units = "mm/d" ;
        ET:Long_name = "ET_sin_riego" ;
        ET:unidades = "[mm/d]" ;

5. Because time is missing that's why we have to set it to a correct time (you have to modify it!!). Change NaN to missing value.

cdo -setmissval,nan -invertlat -setreftime,2000-01-01,12:00:00,1day -settaxis,2000-01-01,12:00:00,1day tmp3.nc ${infile%.*}_attr_subregion.nc
cdo -setmissval,-9999.9 -setmissval,nan tmp4.nc ${infile%.*}_attr_subregion.nc

Result:

dimensions:
    time = UNLIMITED ; // (1 currently)
    lon = 100 ;
    lat = 200 ;
variables:
    double time(time) ;
        time:standard_name = "time" ;
        time:units = "days since 2000-1-1 12:00:00" ;
        time:calendar = "proleptic_gregorian" ;
        time:axis = "T" ;
    double lon(lon) ;
        lon:standard_name = "longitude" ;
        lon:long_name = "longitude" ;
        lon:units = "degrees_east" ;
        lon:axis = "X" ;
    double lat(lat) ;
        lat:standard_name = "latitude" ;
        lat:long_name = "latitude" ;
        lat:units = "degrees_north" ;
        lat:axis = "Y" ;
    float ET(time, lat, lon) ;
        ET:standard_name = "ET_sin_riego" ;
        ET:units = "mm/d" ;
        ET:Long_name = "ET_sin_riego" ;
        ET:unidades = "[mm/d]" ;

-Karin

Wow! Thanks Karin!

I will try to install NCO again, CDO was already a challenge to install in Windows, but I will do my best again with NCO!

Once I have it, I will follow your steps.

Unfortunately, the original NetCDF file does not follow the conventions you indicate, they are outputs of a hydrological model that I do not have access to, so I must modify the file on my own, in order to process and use the information in the file.

I will let those who generated the file know that it would be wise to follow the CF-convention for easier use with NCO and CDO.

Thanks a lot again!!

Regards,
Alonso.

Robert Wilson wrote:

My guess is that you now need to use setgrid to change the grid.

What does "cdo griddes infile" give you?

Robert

Hi Robert,

This gives me the "cdo griddes" operator with the original file:

$ cdo griddes regionalizacion_1979_2015.nc #

1. gridID 1 #
   gridtype = generic
   gridsize = 400
   xsize = 400 #
2. gridID 2 #
   gridtype = generic
   gridsize = 220
   xsize = 220 #
3. gridID 3 #
   gridtype = generic
   gridsize = 444
   xsize = 444 #
4. gridID 4 #
   gridtype = generic
   gridsize = 88000
   xsize = 400
   ysize = 220
   cdo griddes: Processed 11 variables ( 0.00s 8916KB )

The 444 gridsize corresponds to time, as Karin analisys shows it.

Regards,
Alonso.

Robert Wilson wrote:

My guess is that you now need to use setgrid to change the grid.

What does "cdo griddes infile" give you?

Robert

Hi Robert,

This gives me the "cdo griddes" operator for the original file:

$ cdo griddes regionalizacion_1979_2015.nc #

1. gridID 1 #
   gridtype = generic
   gridsize = 400
   xsize = 400 #
2. gridID 2 #
   gridtype = generic
   gridsize = 220
   xsize = 220 #
3. gridID 3 #
   gridtype = generic
   gridsize = 444
   xsize = 444 #
4. gridID 4 #
   gridtype = generic
   gridsize = 88000
   xsize = 400
   ysize = 220
   cdo griddes: Processed 11 variables ( 0.00s 8916KB )

The 444 gridsize grid corresponds to time, as Karin show on her analisys.

Regards,
Alonso.

Karin Meier-Fleischer wrote:

Ok, this took me a while but you need to install NCO (see http://nco.sourceforge.net/#Executables).

The netCDF file is not following the CF-convention, I would say it follows none. CDO is following the CF-convention and needs dimensions like time, lat, lon with the appropriate units. And if the dimension names are not lat and lon the variables have to have a coordinates attribute which leads CDO to the grid dimensions. Furthermore, the variable dimensions should be (time,lat,lon) or (time,lev,lat,lon).

ncdump -h regionalizacion_1979_2015_ET_sel.nc

[...]

1. Reorder the dimensions of ET with NCO's ncpdq

[...]

Result:
[...]

2. To make it easier write the attribute settings to an ASCII file, first.

[...]

3. Change the set of attributes for ET, lat, and lon.

[...]

Result:
[...]

4. Select the correct grid, sellonlatbox, and change the time dimension name. In the original netCDF file
the dim_time_2 variable doesn't exist, that's why it's deleted, here.

[...]

Result:
[...]

5. Because time is missing that's why we have to set it to a correct time (you have to modify it!!). Change NaN to missing value.

[...]

Result:
[...]

-Karin

Hi Karin!

I finally installed NCO with anaconda prompt, so I used your scheme to build the NetCDF file with the information of the "non-CF-convention" file that I must processes.

I have one question:
The original time dimension has 444 "steps" of time, with month as time scale or units. This means that from 1979/01 to 2015/12 there are 444 months that define the time scale of the series.
¿How can I assign to the new variable "Time" those characteristics, ie., 444 month from 1979/01 to 2015/12?

I have tried this, but I am not sure if it works fine:

cdo -setmissval,nan -invertlat -setreftime,1979-01,1month -settaxis,2015-12,1day tmp3.nc ${infile%.*}_attr_subregion.nc

Thank you very much for all your support!

Best,
Alonso.

You have to be careful because the variable ET (on horizontal grid) has just one time step which refers to the original time variable dim_time_2, which is missing. The variables agno and mes (both are time series) refer to the original dim_time variable, which is also missing. CDO accepts only one time dimension that means that you should seperate the variables in two different files.

The operator setreftime is a reference time which can be any you want (but use the same for all your files) and the settaxis sets the time you want.
But the example above uses the variable ET which has only one time step.

Thanks Karin!

I will be careful with time issues.

Regards,
Alonso.