LOOPING OF MULTIPLE FILES
Added by selam haftu about 4 years ago
i am new to CDO, i have gone through the manual and i can perform some operations. however, my work is too bulky, i have about 1500 files from ESGF with different variables. for each file, i have to remap, extract the study area, and calculate area mean. i can however perform this operation for individual file, but to perform a loop to ease this task is my difficulties. i cant find such operation in the manual. please provide me with the command on how to perform the loop operation for my multiple files so that i can perform at once. Also, can i perform all the three task together or it must be one after the other.
Thank you for your time and effort.
CDO version: 1.9.4rc1
Replies (4)
RE: LOOPING OF MULTIPLE FILES - Added by Ralf Mueller about 4 years ago
Hi Selam!
1500 files is a lot indeed. CDO does not offer looping itself, but this seem to be a perfect example for a shell loop. I posted some techniques how to speed up data analysis some time ago here
you might have a look at point 1) and 2). Remapping benefits from OpenMP-parallelization (1) and a proper for-loop is shown in (2). Appending
| parallel -v -j 12will process things in parallel, but for a first try I would leave this out (and also the "echo" stuff around the CDO command line).
hth
ralf
RE: LOOPING OF MULTIPLE FILES - Added by selam haftu about 4 years ago
thank you for your response. i saw a similar question by a user in the forum, i don't know if i can use the the same method she used for few files at a given time with her loop (for loop).
Her question can be found here https://code.mpimet.mpg.de/boards/2/topics/6396 if its possible, kindly assist me in defining it. thank you for all your support, we really appreciate.
RE: LOOPING OF MULTIPLE FILES - Added by Ralf Mueller about 4 years ago
sure, her solution is also fine
for i in $(ls); do cdo remapnn,grid2.txt ${i} regrid/${i}wgs.nc; doneItis the same that I pointed to, but without the extra parallelization command. But that's no problem - it will just take a bit longer.
cheers
ralf