Project

General

Profile

merge multi netcdf files, not base on time

Added by nazanin tavakoli over 3 years ago

Hello,
I want to convert 6 netcdf files to a NetCDF. My data are containing clay information, so all 6 files should have the same time, but they are different in layers.
longitude = 720 ;
latitude = 360 ;
layer = 6 ;
time = 1
I have converted them to a NetCDF file with the following code:
cdo merge p1.nc p2.nc p3.nc p4.nc p5.nc p6.nc out.nc
but the output file just shows one of my layer not 6 of them. Could anybody help me?
I want to merge netcdf file base on variable which isn't the time, and here is my data:
https://drive.google.com/file/d/1HzIE3NkJJ8WJydYg9kwiYUYavXoSRBL8/view

I would appreciate your help.


Replies (10)

RE: merge multi netcdf files, not base on time - Added by Ralf Mueller over 3 years ago

hi!

cdo merge p1.nc p2.nc p3.nc p4.nc p5.nc p6.nc out.nc does NOT convert any csv file into netcdf format. And your uploaded rar file does not contain the nc input:

UNRAR 6.00 freeware      Copyright (c) 1993-2020 Alexander Roshal

Archive: sand.rar
Details: RAR 5, solid

 Attributes      Size     Date    Time   Name
----------- ---------  ---------- -----  ----
    ..A....        92  2021-02-26 15:22  attributes.txt
    ..A....      2153  2021-02-26 15:59  code.txt
    ..A....       262  2021-02-26 10:11  gridfile.txt
    ..A.... 128553644  2021-02-26 15:38  sand0_5.csv
    ..A.... 128518646  2021-02-21 16:41  sand5_15.csv
    ..A.... 680417712  2021-02-26 15:53  out.nc
    ..A.... 128527083  2021-02-21 16:42  sand100_200.csv
    ..A.... 128527650  2021-02-21 16:41  sand15_30.csv
    ..A.... 128531094  2021-02-21 16:41  sand30_60.csv
    ..A.... 128542755  2021-02-21 16:41  sand60_100.csv
    ..A....  12450876  2005-02-07 13:11  soita.sand.nc
----------- ---------  ---------- -----  ----
           1464071967                    11

So it's hard to come up with a reasonable comment at the moment, sorry

cheers
ralf

RE: merge multi netcdf files, not base on time - Added by nazanin tavakoli over 3 years ago

yes, I know that cdo merge doesn't convert CSV file to NetCDF. This command will just merge NetCDF files. I put my output file in the rear I have uploaded. Sorry for making that mistake. Here is my input files:
https://drive.google.com/file/d/13KvgihCEGPyzz6osbz5iUqdX43PVs8Yi/view?usp=sharing

RE: merge multi netcdf files, not base on time - Added by Brendan DeTracey over 3 years ago

You do not want to merge. You need to create a new dimension and concatenate along that new dimension. I do not know if cdo can do this.
I do know that NCO (http://nco.sourceforge.net/) can, but I do not know how. Otherwise you will have to code your own solution in any of ncl, matlab, python, fortran, c etc.

RE: merge multi netcdf files, not base on time - Added by nazanin tavakoli over 3 years ago

Why did you say that I don't want to merge them? I want definitely want to merge 6 NetCDF files base on layer not time. Also, I created a new dimension and variable called layer in nco with the following code:
ncap2 -s 'defdim("layer",1);layer[layer]=1*1' input.nc output.nc

RE: merge multi netcdf files, not base on time - Added by Ralf Mueller over 3 years ago

nazanin tavakoli wrote:

Why did you say that I don't want to merge them? I want definitely want to merge 6 NetCDF files base on layer not time. Also, I created a new dimension and variable called layer in nco with the following code:
ncap2 -s 'defdim("layer",1);layer[layer]=1*1' input.nc output.nc

Hi nazanin!

I think it was not the intention to tell you what you want, but to tell you, that the term 'merge' already has a special meaning when it comes to CDO. There are operators called 'merge' and 'mergetime' which does certain things and these things are not, what you want.

For example there is also a 'cat' operators for concatenation of files - this could also be regarded as some kind of merge operation, it works mostly like the Unix tool 'cat'.

RE: merge multi netcdf files, not base on time - Added by nazanin tavakoli over 3 years ago

OK, Could you have any idea how can I merge my NetCDF files into a single one?
I would appreciate it if you help me.

RE: merge multi netcdf files, not base on time - Added by Ralf Mueller over 3 years ago

here is my solution: put the griddes file and the processing.sh script next to your input files l1.nc - l6.nc and run the shell script. Or you execute the commands directly

for i in $(seq 1 6); do ls l${i}.nc; cp griddes griddes_${i}; perl -pi.bak -e "s/AAA/${i}/" griddes_${i} ; cdo -setzaxis,griddes_${i} l${i}.nc out${i}.nc ;done

the result files out1.nc - out6.nc can be merged with cdo merge

hth
ralf

RE: merge multi netcdf files, not base on time - Added by nazanin tavakoli over 3 years ago

Thank you for your response.
I did what you said with your processing.sh and griddes, but I had this error:
cdo merge: Open failed on >out{1,2,3,4,5,6}.nc<
No such file or directory

Then, I changed griddes, and wrote it in a txt file( I have sent it here with processing.txt), but I had this error:
l1.nc
cp: cannot stat 'griddes': No such file or directory
Can't open griddes_1: No such file or directory.

cdo setzaxis (Abort): Open failed on griddes_1!
l2.nc
cp: cannot stat 'griddes': No such file or directory
Can't open griddes_2: No such file or directory.

cdo setzaxis (Abort): Open failed on griddes_2!
l3.nc
cp: cannot stat 'griddes': No such file or directory
Can't open griddes_3: No such file or directory.

cdo setzaxis (Abort): Open failed on griddes_3!
l4.nc
cp: cannot stat 'griddes': No such file or directory
Can't open griddes_4: No such file or directory.

cdo setzaxis (Abort): Open failed on griddes_4!
l5.nc
cp: cannot stat 'griddes': No such file or directory
Can't open griddes_5: No such file or directory.

cdo setzaxis (Abort): Open failed on griddes_5!
l6.nc
cp: cannot stat 'griddes': No such file or directory
Can't open griddes_6: No such file or directory.

cdo setzaxis (Abort): Open failed on griddes_6!

cdo merge: Open failed on >out{1,2,3,4,5,6}.nc<
No such file or directory

Also, when I used your griddes file, the variable level in out1.nc, out2.nc, and so on was 1 meter, 2 metres, and so on. I want to change them to 10,15,25,50,100,and 200 for out1.nc until out6.nc,respectively.
Could you please help me?

code.rar (1.72 MB) code.rar

RE: merge multi netcdf files, not base on time - Added by Ralf Mueller over 3 years ago

hi!

I added the levels to each individual griddes file. on my machine the result file OUT.nc looks like this

cdo infov OUT.nc                                                                                                                                          ─╯
    -1 :       Date     Time   Level Gridsize    Miss :     Minimum        Mean     Maximum : Parameter name
     1 : 2001-01-01 12:00:00      10   329478  100680 :      0.0000      58.243      88.700 : var1          
     2 : 2001-01-01 12:00:00      15   329478  100668 :      0.0000      56.481      90.000 : var1          
     3 : 2001-01-01 12:00:00      25   329478  100668 :      0.0000      53.452      88.600 : var1          
     4 : 2001-01-01 12:00:00      50   329478  100668 :      0.0000      49.192      89.800 : var1          
     5 : 2001-01-01 12:00:00     100   329478  100668 :      0.0000      48.579      86.100 : var1          
     6 : 2001-01-01 12:00:00     200   329478  100668 :      0.0000      48.706      86.700 : var1          

start the processing with

bash processing.txt

    (1-10/10)