Project

General

Profile

Extract a sub-region in a curvilinear grid and paste all parameter values on an identical file

Added by Orestis Speyer almost 5 years ago

Hi everyone,
this is my first post so I hope I am at the right place! Here is what I am trying to do.

I have a basecase file (emissions) "emiss2014012620_toPASTE.nc" which I want to change only for a particular geographical area.
To change it, I want to use the values from a different scenario (less emissions in the particular geographical area) found in emiss2014012620_toCUT.nc.
Both files are on an identical grid (it is in rotated coordinates), I am only trying to "exchange" the values of the parameters.

The basic idea is to "cut" a subset and put it/paste it in the basecase file.Could you please help me in this?
I looked at https://code.mpimet.mpg.de/boards/1/topics/1391 but couldn't make much use of it as I am completely new at CDO.

If this is accomplished, I would like to do it for multiple files, therefore I attach four files in total of two consecutive hours.

Thanks a lot for your time,
Orestis

I used sinfon to see what is in my file:
File format : netCDF4 ZIP
-1 : Institut Source Ttype Levels Num Points Num Dtype : Parameter name
1 : unknown unknown constant 7 1 2 1 F32 : level_bnds
2 : unknown unknown instant 7 1 131040 2 F32z : VSO4Je
3 : unknown unknown instant 7 1 131040 2 F32z : VSO4Ie
4 : unknown unknown instant 7 1 131040 2 F32z : VP25AIe
5 : unknown unknown instant 7 1 131040 2 F32z : VP25AJe
6 : unknown unknown instant 7 1 131040 2 F32z : VORGPAIe
7 : unknown unknown instant 7 1 131040 2 F32z : VORGPAJe
8 : unknown unknown instant 7 1 131040 2 F32z : VANTHAe
9 : unknown unknown instant 7 1 131040 2 F32z : VSOOTe
10 : unknown unknown instant 7 1 131040 2 F32z : ORA2e
11 : unknown unknown instant 7 1 131040 2 F32z : KETe
12 : unknown unknown instant 7 1 131040 2 F32z : ALDe
13 : unknown unknown instant 7 1 131040 2 F32z : HCHOe
14 : unknown unknown instant 7 1 131040 2 F32z : CSLe
15 : unknown unknown instant 7 1 131040 2 F32z : XYLe
16 : unknown unknown instant 7 1 131040 2 F32z : TOLe
17 : unknown unknown instant 7 1 131040 2 F32z : OLIe
18 : unknown unknown instant 7 1 131040 2 F32z : OLTe
19 : unknown unknown instant 7 1 131040 2 F32z : OL2e
20 : unknown unknown instant 7 1 131040 2 F32z : HC8e
21 : unknown unknown instant 7 1 131040 2 F32z : HC5e
22 : unknown unknown instant 7 1 131040 2 F32z : HC3e
23 : unknown unknown instant 7 1 131040 2 F32z : ETHe
24 : unknown unknown instant 7 1 131040 2 F32z : PM25e
25 : unknown unknown instant 7 1 131040 2 F32z : PM10e
26 : unknown unknown instant 7 1 131040 2 F32z : SO2e
27 : unknown unknown instant 7 1 131040 2 F32z : NOe
28 : unknown unknown instant 7 1 131040 2 F32z : NO2e
29 : unknown unknown instant 7 1 131040 2 F32z : NMVOCe
30 : unknown unknown instant 7 1 131040 2 F32z : NH3e
31 : unknown unknown instant 7 1 131040 2 F32z : COe
32 : unknown unknown instant 7 1 131040 2 F32z : CH4e
33 : unknown unknown instant 7 1 131040 2 F32z : COUNTRYMASK
Grid coordinates :
1 : generic : points=2
2 : lonlat : points=131040 (390x336)
rlon : -4.875 to 4.85 by 0.025 degrees
rlat : -4.075 to 4.3 by 0.0250001 degrees
Vertical coordinates :
1 : height : levels=7
level : 2 to 850 meters
Time coordinate : 1 step
RefTime = 2012-12-09 22:00:00 Units = seconds Calendar = proleptic_gregorian
YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss
2012-12-09 22:00:00
cdo sinfon: Processed 33 variables over 1 timestep ( 0.04s )


Replies (8)

RE: Extract a sub-region in a curvilinear grid and paste all parameter values on an identical file - Added by Uwe Schulzweida almost 5 years ago

You can use sellonlatbox or selindexbox to select a rectangular region and mergegrid to merge the data of the selected region with an other file. Here is an example:

cdo sellonlatbox,-2,2,-2,2 emiss2014012620_toCUT.nc select_region.nc
cdo mergegrid emiss2014012620_toPASTE.nc select_region.nc result
Unfortunately your netcdf files doesn't follow the CF standard. Therefor the coordinate variable level_bnds is handled as a data variable. You have to set the correct netCDF attributes or remove this variable:
cdo sellonlatbox,-2,2,-2,2 -delname,level_bnds emiss2014012620_toCUT.nc select_region.nc
cdo mergegrid -delname,level_bnds emiss2014012620_toPASTE.nc select_region.nc result

RE: Extract a sub-region in a curvilinear grid and paste all parameter values on an identical file - Added by Orestis Speyer almost 5 years ago

Uwe thanks a lot for your help.

Sorry I have not responded sooner but our HPC down for maintenance. I believe it will be up again some time today and I will try your solution.
With respect to level_bnds, I am not sure if this will create an issue in my pre-processor. I will delete it and see what happens next.

As far as the mergegrid operator is concerned, I read from the online CDO User Guide that "Only rectilinear grids are supported". Is this a problem for my case?

RE: Extract a sub-region in a curvilinear grid and paste all parameter values on an identical file - Added by Orestis Speyer almost 5 years ago

Uwe,

I am happy to report that the elegant procedure worked nicely!
However, the pre-processor exits with an error as it expects this (very small in size variable)level_bnds.

Can I put it back in somehow when I am finished with the merging?

Once again, thank you.

RE: Extract a sub-region in a curvilinear grid and paste all parameter values on an identical file - Added by Orestis Speyer almost 5 years ago

Hello again!

I used the merge operator and successfully re-introduced the level_bnds. However, the pre-processor still exits with the same error.
Till now, I've used:
cdo sellonlatbox
cdo mergegrid
cdo merge (I created a separate file of level_bnds and merged it in the final file)

I looked into the two files using ncdump -h and saw that some things are completely missed by CDO and also some smaller changes in some of the variables:
-The char rotated_pole(time, level, rlat, rlon) ;
rotated_pole:grid_mapping_name = "rotated_latitude_longitude" ;
rotated_pole:grid_north_pole_latitude = 52.3f ;
rotated_pole:grid_north_pole_longitude = -156.5f ;
rotated_pole:north_pole_grid_longitude = 0.f ;
is completely missing

-The original has
float level_bnds(bnds, level) ;
level_bnds:units = "meters" ;
while the processed has
float level_bnds(x, level) ;
level_bnds:units = "meters" ;

-Under Dimensions:
the processed has x = 2 ; instead of bnds =2 ;

-The processed under each variable has attributes added:
missing_value and axis for the rlat, rlon, level variables

RE: Extract a sub-region in a curvilinear grid and paste all parameter values on an identical file - Added by Uwe Schulzweida almost 5 years ago

The problem with the renaming of bnds is fixed since CDO release 1.8.0.
The variable rotated_pole is missing because the attribute

grid_mapping = "rotated_pole" ;
is missing on all data variables.

RE: Extract a sub-region in a curvilinear grid and paste all parameter values on an identical file - Added by Orestis Speyer almost 5 years ago

I am in the process of compiling version 1.8.0 with our IT guy.

With respect to the missing attribute, do you suggest I should add this attribute to all or some of the variables? Should I do this before the first cdo sellonlatbox? Could you provide an example on how to do so?

Thanks,
Orestis

RE: Extract a sub-region in a curvilinear grid and paste all parameter values on an identical file - Added by Orestis Speyer almost 5 years ago

I now have version 1.9.6!

I added the attribute grid_mapping in all data variables (both of the CUT file and the PASTE file) as follows:

ncatted -a grid_mapping,,a,c,rotated_pole emiss2014011900.nc (to add the attribute in all variables)

ncatted -O -a grid_mapping,rotated_pole,d,, emiss2014011900.nc
ncatted -O -a grid_mapping,rlon,d,, emiss2014011900.nc
ncatted -O -a grid_mapping,rlat,d,, emiss2014011900.nc
ncatted -O -a grid_mapping,level,d,, emiss2014011900.nc
ncatted -O -a grid_mapping,time,d,, emiss2014011900.nc (to leave it only in the data variables)

I then proceeded with cutting a region:
cdo sellonlatbox,-0.025,0.475,-0.025,0.425 -delname,level_bnds CUT/emiss2014011900.nc CUT/select_region.nc
and then merging it with the PASTE file into the final file:
cdo mergegrid -delname,level_bnds PASTE/emiss2014011900.nc CUT/select_region.nc FINAL/emiss2014011900_final.nc

but I get an ERROR:
cdo mergegrid (Abort): Unsupported grid type: projection

Indeed, when I look at the cdo sinfon of the original file before changing anything I get:
Grid coordinates :
1 : generic : points=2
2 : lonlat : points=131040 (390x336)
rlon : -4.875 to 4.85 by 0.025 degrees
rlat : -4.075 to 4.3 by 0.025 degrees

and after the ncatted:
Grid coordinates :
1 : projection : points=2
mapping : rotated_latitude_longitude
2 : projection : points=131040 (390x336)
mapping : rotated_latitude_longitude
rlon : -4.875 to 4.85 by 0.025 degrees
rlat : -4.075 to 4.3 by 0.025 degrees

Finally, I would like to note that the file, although it uses rlat and rlon, does not follow the format lat(rlat, rlon). It only has rlon(rlon) and rlat(rlat).

RE: Extract a sub-region in a curvilinear grid and paste all parameter values on an identical file - Added by Orestis Speyer almost 5 years ago

[SOLVED]

Hi Uwe, everyone, I would just like to let you know that I used both NCO and CDO and managed to solve my problem. A synopsis of what I did:

I use NCO to cut static (same variables for all files), "cdo-dirty" variables that cdo cannot handle onto an out.nc file to put back at the end of the procedure:

$ncks -v level_bnds emiss2014011900.nc out.nc
$ncks -v rotated_pole emiss2014011900.nc out.nc

Again with NCO, I remove these undesired variables (I do this for all my files with a bash do loop, the -O option answers the interactive question that pops up with an "overwrite"):

$ncks -x -O -v rotated_pole,level_bnds emiss2014011900.nc emiss2014011900.nc

Using CDO, I first cut out a desired region from the CUT files and temporarily store it in the select_region.nc. Then, I merge the PASTE file and the selected_region file into the FINAL file:

$cdo sellonlatbox,-0.025,0.475,-0.025,0.425 CUT/emiss2014011900.nc CUT/select_region.nc
$cdo mergegrid PASTE/emiss2014011900.nc CUT/select_region.nc FINAL/emiss2014011900.nc

Finally, again with NCO, I "return" the two variables I had cut in the beginning into the FINAL files:
$ncks -h -A out.nc FINAL/emiss2014011900.nc

I am sure there are more efficient ways to do this (e.g. using pipe) but this is the best I could manage.

Cheers to all,
Orestis

    (1-8/8)