CDO

Hi Clement,

We have an experimental operator called mergegrid for this task:

cdo setrtomiss,-1e33,1e33 file3.nc file4.nc  # create file4 with the grid of file3 and set all values to miss
cdo mergegrid file4.nc file1.nc file5.nc     # merge all grid points of file1 to file4 and store it to file5 
cdo mergegrid file5.nc file2.nc file6.nc     # merge all grid points of file2 to file5 and store it to file6

This operator is not documented, yet.

Regards,
Uwe

Excellent !

Thank you very much !

Regards,

Clement

Actually, there is seems to be a problem with mergegrid :

1. Example:
   cdo sellonlatbox,-11,10,30,40 file.nc file1.nc
   cdo sellonlatbox,-11,10,20,30 file.nc file2.nc
   cdo sellonlatbox,-11,10,20,40 file.nc file3.nc

1. Merge file1.nc and file2.nc into file4.nc.
   cdo mergegrid file1.nc file2.nc file.4.nc

1. Check grid characteristic for each file. Normally, file4.nc should be equal to file3.nc but it is not...
   clem@clem-laptop:/media/VERBATIM$ cdo griddes file1.nc #

1. gridID 0 #
   gridtype = lonlat
   gridsize = 40
   xname = lon
   xlongname = longitude
   xunits = degrees_east
   yname = lat
   ylongname = latitude
   yunits = degrees_north
   xsize = 8
   ysize = 5
   xfirst = -8.75
   xinc = 2.5
   xbounds = 350 352.5
   352.5 355
   355 357.5
   357.5 360
   0 2.5
   2.5 5
   5 7.5
   7.5 10
   yfirst = 31
   yinc = 2
   ybounds = 30 32
   32 34
   34 36
   36 38
   38 40
   cdo griddes: Processed 1 variable. ( 0.00s )
   clem@clem-laptop:/media/VERBATIM$ cdo griddes file2.nc #
2. gridID 0 #
   gridtype = lonlat
   gridsize = 40
   xname = lon
   xlongname = longitude
   xunits = degrees_east
   yname = lat
   ylongname = latitude
   yunits = degrees_north
   xsize = 8
   ysize = 5
   xfirst = -8.75
   xinc = 2.5
   xbounds = 350 352.5
   352.5 355
   355 357.5
   357.5 360
   0 2.5
   2.5 5
   5 7.5
   7.5 10
   yfirst = 21
   yinc = 2
   ybounds = 20 22
   22 24
   24 26
   26 28
   28 30
   cdo griddes: Processed 1 variable. ( 0.00s )
   clem@clem-laptop:/media/VERBATIM$ cdo griddes file3.nc #
3. gridID 0 #
   gridtype = lonlat
   gridsize = 80
   xname = lon
   xlongname = longitude
   xunits = degrees_east
   yname = lat
   ylongname = latitude
   yunits = degrees_north
   xsize = 8
   ysize = 10
   xfirst = -8.75
   xinc = 2.5
   xbounds = 350 352.5
   352.5 355
   355 357.5
   357.5 360
   0 2.5
   2.5 5
   5 7.5
   7.5 10
   yfirst = 21
   yinc = 2
   ybounds = 20 22
   22 24
   24 26
   26 28
   28 30
   30 32
   32 34
   34 36
   36 38
   38 40
   cdo griddes: Processed 1 variable. ( 0.00s )
   clem@clem-laptop:/media/VERBATIM$ cdo griddes file4.nc
   Warning (cdfInqContents) : time variable not found! #
4. gridID 0 #
   gridtype = lonlat
   gridsize = 40
   xname = lon
   xlongname = longitude
   xunits = degrees_east
   yname = lat
   ylongname = latitude
   yunits = degrees_north
   xsize = 8
   ysize = 5
   xfirst = -8.75
   xinc = 2.5
   xbounds = 350 352.5
   352.5 355
   355 357.5
   357.5 360
   0 2.5
   2.5 5
   5 7.5
   7.5 10
   yfirst = 31
   yinc = 2
   ybounds = 30 32
   32 34
   34 36
   36 38
   38 40
   cdo griddes: Processed 1 variable. ( 0.01s )

It seems that mergegrid is not the optimal name for this operator, because it does not merge two different grids to a new one!
The first input file have to contain the target grid. So you can add only the data from the second input file to the first grid.
That means the resulting output file has the same grid as the first input file.
Here again the example from above:

cdo setrtomiss,-1e33,1e33 file3.nc file4.nc  # create file4 with the grid of file3 and set all values to miss
cdo mergegrid file4.nc file1.nc file5.nc     # merge all grid points of file1 to file4 and store it to file5 
cdo mergegrid file5.nc file2.nc file6.nc     # merge all grid points of file2 to file5 and store it to file6

Regards,
Uwe

Ok thank you for the precisions.

So, to your knowledge there is no function in cdo to merge two files with different grids ? In my case study, the two files have exactly the same temporal structure and spatial resolution and they correspond to two spatially adjacent files.

Let me know if you have any suggestions...

Regards

Clem

Hello,

I have found a solution to my problem but I do not know if it is the most efficient way to do it. What do you think ?

## Create three files, as file3=file1+file2
cdo sellonlatbox,-11,10,30,40 ulwsfc.gdas.197901.grb2 file1.grb2
cdo sellonlatbox,-11,10,20,30 ulwsfc.gdas.197901.grb2 file2.grb2
cdo sellonlatbox,-11,10,20,40 ulwsfc.gdas.197901.grb2 file3.grb2

## Let us build a "No data" file6 which have the same structure than file3
cdo ifthenc,NA file3.grb2 file5.grb2
cdo ifnotthenc,NA, file3.grb2 file6.grb2

## Merging file1 and file2 into file8.
cdo mergegrid file6.grb2 file1.grb2 file7.grb2
cdo mergegrid file7.grb2 file2.grb2 file8.grb2

## Checking file3 and file8. After merging, file8 is identical to file3.
cdo info file8.grb2
cdo info file3.grb2

Cheers

Yes, this is the most efficient way for this task!
Regards,
Uwe

I have another solution for this, and it works very well.
Clement, in your example you still actually need the bigger "file3".

Here is what you do, without the file "file3" itself but only with its info.

• Make a file called "grid.txt" and put in it the right info about file3, e.g.:

gridtype = lonlat
gridsize = 420
xname = lon
xlongname = longitude
xunits = degrees east
yname = lat
ylongname = latitude
yunits = degrees north
xsize = 21
ysize = 20
xfirst = -11.0
xinc = 1
yfirst = -20.0
yinc = 1

• First make an enlarged file with the info of file3. Its contents are rubbish but that doesnt matter:

cdo enlarge,grid.txt file1.nc file4.nc

• Then merge the enlarged domain with the content of the first and consequently with the second file:

cdo mergegrid file4.nc file1.nc file5.nc
cdo mergegrid file5.nc file2.nc file6.nc

• Remove the temporary files:

rm file4.nc
rm file5.nc

• Now the spatially merged file is "file6.nc".

Good luck!

Hi Tom Akkermans,

Thanks very much , your procedure worked for my situation, but I have a question, how the Mergegrid.c make the calculation ?

My situation is as follows:

I have two datasets : file1.nc and file2.nc

file1: Latitude : 9N to 60N
Longitude: 100W to 19W

file2: Latitude : 9N to 60N
Longitude: 21W to 45E

My final file: file_final.nc have this coordinates:
Latitude : 9N to 60N
Longitude: 100W to 45E

The mergegrid works fine, but I have a doubt about the intersection, what the mergegrid command do in intersection? Average or another calculation?

Regards,
Rodrigo

Hello all,

First thank you Tom Akkermans for you solution, it has worked for me in the past.

However, I now need to merge several dataset along the spatial dimension that are adjacent, but have intersecting boundaries.

Example:

file1: Latitude: 42N to 46N
Longitude 120W to 110W

file2: Latitude: 42N to 46N
Longitude: 110W to 100W

For the intersecting Longitude 110W:
file1 has values for latitudes 42N to 44N and empty cells (NaN) for 45N to 46N
file2 has values for latidues 45N to 46N and empty cells (NaN) for 42N to 44N

When I use mergegrid the values in this intersecting longitude are all read from file2, overwriting the data from file1. Is there a way to have the function only append values that are not NaN?

I have searched extensively for a way (CDO, NCO, others) to merge files with adjacent and intersecting boundaries without finding a solution. Any ideas or other people to contact would be greatly appreciated!

Thank you for your time,
Kristen

Clement Tisseuil wrote:

Dear all,

Is there a possibility to merge several dataset along the spatial dimension (like mergetime but applied to the spatial coordinates). Typically, the dataset are identical in terms of variables and time dimensions but they represent different spatial area.

Example :
cdo sellonlatbox,-10,10,30,40 file.nc file1.nc
cdo sellonlatbox,-10,10,20,30 file.nc file2.nc
cdo sellonlatbox,-10,10,20,40 file.nc file3.nc

I would like to build file4.nc as the spatial merging of file1.nc and file2.nc, which should be identical to file3.nc

Thanks in advance.

Regards,

Clement

Is there a simpler way to merge nc files without first creating an empty nc file? It's not very efficient if the creation of the grid.txt needs to be done manually.

create an empty file first? I never did this ...

what exactly the problem, Noam?

cheers
ralf

I am trying to merge multiple nc files with the same variables and timesteps, but containing different sections of the world. Say I have africa.nc and india.nc

Can I merge them so I end up with a file africa-india.nc without manually creating the grid.txt file as suggested above? In R for example is possible to do merge(file1,file2,file3,etc.) and the function will figure out the extent of the combined file. Creating many empty files will require to write many grid.txt files, or am I missing something?

Hi Noam,

what about remapping the africa.nc and india.nc to a global grid?

Assume you want to have a 0.5x0.5 degrees global grid (720x360):

cdo -remapbil,r720x360 africa.nc africa_r720x360.nc
cdo -remapbil,r720x360 india.nc india_r720x360.nc

cdo -merge africa_r720x360.nc india_r720x360.nc africa_india_r720x360.nc

-Karin

Thank you Karin. Your suggestion works

It looks though, that if I want to merge multiple files I have to do pair by pair.

Because

cdo merge out-a.nc out-n.nc out-i.nc test.nc
Warning: Duplicate entry of parameter name myvar in out-n.nc!
cdo merge: Processed 3 variables over 3 timesteps [0.02s 9MB].

but the result only contains out-a.nc

and mergegrid only takes 2 input files. Maybe it's just not possible? Because I also tried
cdo merge out-* test.nc

but no luck.

Hello Karin,

I would like to know if when remapping a section of the globe into a global grid the actual data is interpolated or not, because remapbil does bilinear interpolation. Let's assume that the resolution is the same, say I have India at 0.5x0.5 and I remap to a global grid at 0.5x0.5, in theory there should be no interpolation, correct?

Thats correct, if the resolution and the coordinates are the same then no interpolation takes place.
If the environment variable REMAP_EXTRAPOLATE=off is set than remapbil and remapnn will give the same result in this case.

I am sorry to bring this up again but I'm getting inconsistent results with the remapbil function.

What I do is
cdo remapbil,mygrid popavg-africa.nc out-a.nc

mygrid is
gridtype = lonlat
xsize = 180
ysize = 90
xfirst = -179
xinc = 2
yfirst = -89
yinc = 2

But the resulting values in out-a.nc are wrong, it IS interpolating the values. I am uploading the starting file. If I do the merge with R than the result is correct. Do you know why?

You are right, remapbil can be used only if the data come without missing values. Your data contain missing values, in this case you have to use remapnn in combination with the environment variable REMAP_EXTRAPOLATE=off.

Thank you Uwe, I will try that.
So to clarify:
remapnn with REMAP_EXTRAPOLATE=off (with NAs) should give the same results as remapbil (when there are no NAs), assuming all other values are the same.

Hello,

i am also trying to merge multiple nc files with the same variables and timesteps, but containing different sections of the world. all of them are 0.5° resolution grid files.for exemple:

MLLO.1998.2019.cld.dat.nc
ASKS.1998.2019.cld.dat.nc
NY.1998.2019.cld.dat.nc

Then I would like to know if Noam find the best way or if someone got it.

Thanks

The solution suggested by Uwe works for me.

You should use set REMAP_EXTRAPOLATE=off and then use cdo remapbil,mygrid region1, region2 global.nc

The only problem I really have with this is that I cannot use more than two regions as an input so I have to use cdo remapbil,mygrid multiple times until I had all the regions.

Thank you, Noam

Please Noam, I did this:

$ REMAP_EXTRAPOLATE=off

$ cdo remapnn,mygrid MLLO.1998.2019.cld.dat.nc MLLOout.1998.2019.cld.dat.nc

but I have got this

cdo remapnn (Abort):

Did I miss something?

Thanks

Hi!
Can you upload the files, Audace? A single timestep should be sufficient.

cheers
ralf