-outputtab for multiple combinations of points
Added by Fabio Farinosi over 5 years ago
Hello,
I have a large netcdf providing global daily values with a high spatial resolution. I would like to extract the time series for a selection of points (some hundred thousands of them) using the -outputtab operator. Is there a way to do something like this?
cdo -outputtab,lon,lat,date,value -remapnn,lon_and_lat_combination_list.txt ifile.nc > ofile.txt
In case this is possible, what format should I give to the points coordinate list file? Should it be a .txt? a .csv?
(Please, note that the points I need to extract the data for are not in a regular grid)
I tried a couple of combinations, but I received the following error messages:
cdo(1) remapnn: Process started Namelist error: Invalid contents in pts_coords_list.txt! cdo(1) remapnn (Abort): Namelist error!
cdo(1) remapnn: Process started Namelist error: Invalid contents in pts_coords_list.csv! cdo(1) remapnn (Abort): Namelist error!
Thank you
Replies (11)
RE: -outputtab for multiple combinations of points - Added by Karin Meier-Fleischer over 5 years ago
Hi Fabio,
I would guess the answer is no. But you can write a short shell script using a loop to select all points.
Here is an example Korn-Shell script which selects 4 points (in your case you can read the x and y arrays
from file):
#!/bin/ksh #-- delete existing ASCII output file rm -rf outfile.txt #-- name of the input file infile=inputfile.nc #-- arrays of the points lon lat locations set -A lon 0.0 5.0 10.0 15.0 set -A lat 0.0 5.0 10.0 15.0 #-- loop to retrieve the data of the selected points for i in $(seq 0 3) do cdo -O remapnn,lon=${lon[$i]}/lat=${lat[$i]} $infile tmp.nc cdo -outputtab,lon,lat,lev,date,value tmp.nc >> outfile.txt done #-- delete the multiple entries of the header line in output file header="#" awk '!header[$0]++' outfile.txt > tmp2.txt mv tmp2.txt outfile.txt #-- remove temporary file rm tmp.nc exit
Take care whether your data file has levels or not.
-Karin
RE: -outputtab for multiple combinations of points - Added by Fabio Farinosi over 5 years ago
Hi Karin,
Thanks for your help.
Sorry, I'm not very familiar with the ksh and I'm having some troubles running this script.
I installed the mksh in the cygwin environment, put your script in a .ksh file and tried to run it but it gives me the following error indicating the line where the for loop starts:
file.ksh[15]: syntax error: unexpected 'do
In addition, I'd really appreciate if you could give me some guidance in how I should edit the script to go take the lon lat arrays from a file like the one attached.
Thank you very much
pts_test.txt (134 Bytes) pts_test.txt |
RE: -outputtab for multiple combinations of points - Added by Karin Meier-Fleischer over 5 years ago
The script is a Korn Shell script so you need to have ksh installed.
You can modify the script and run it, lets assume the script name is your_script.ksh:
ksh your_script.ksh
But you can rewrite it to use another shell scripting language as you like, it's on you.
-Karin
RE: -outputtab for multiple combinations of points - Added by Fabio Farinosi over 5 years ago
Hello Karin
Sorry, I've then realized that there was a problem of Windows/UNIX encoding in the line ending of the script, because I used a normal EditPad Lite 7 to write down a script inspired by yours instead of writing it down directly in a vi editor.
The editor I used was using the Windows encoding to end the lines in the code and that was causing the syntax error. The encoding is specifically:
^M
It took me a while to figure this out because the vi editor of CygWin was not highlighting this problem, while the UNIX one was.
Once this issue was fixed, the code was running smoothly. This methodology, however, is extremely costly in processing time. The domain in my .nc file has almost 4 million cells for more than 14 thousands time steps, I need to extract time series for about 400 thousands points. This script would do that sequentially for each of the lon/lat combinations. In each time step, the procedure would spend about 800/1000 seconds only in the remapnn step.
It would be nice to have a more efficient way to extract data for specific set of coordinates.
Thank you very much again.
Best,
F
RE: -outputtab for multiple combinations of points - Added by Max Ballhausen over 5 years ago
Hi there, thanks for this script!
I am trying to extract from negative longitudes or latitudes, and run into an error:
test110: set: -0: unknown option
test110: set: -.: unknown option
test110: set: -9: unknown option
test110: set: -6: unknown option
test110: set: -3: unknown option
test110: set: -2: unknown option
test110: set: -4: unknown option
test110: set: -6: unknown option
This is for example for longitude -0.963246
I tried to fix it with the conversion using -convertlon or -convertlat but this only gives another error.
In that case all my extracted values become 1e+20
Any help would be appreciated!
Thanks
Max
script and inputfile are attached!
RE: -outputtab for multiple combinations of points - Added by Karin Meier-Fleischer over 5 years ago
Hi Max,
what did you do exactly? -- Sorry didn't noticed that test1 is a script. It is always a good style to name a script including a suffix, e.g. test1.ksh.
-Karin
RE: -outputtab for multiple combinations of points - Added by Karin Meier-Fleischer over 5 years ago
if you want to extract only one single point than you don't have to use arrays.
cdo -O remapnn,lon=-0.963246/lat=38.308579 $infile tmp.nc cdo -outputtab,lon,lat,date,value tmp.nc >> outfile.txt
Or in a script
#!/bin/ksh #-- delete existing ASCII output file rm -rf outfile.txt #-- name of the input file infile=euin.nc #-- lon lat location lon=-0.963246 lat=38.308579 #-- retrieve the data of the selected points cdo -O remapnn,lon=${lon}/lat=${lat} $infile tmp.nc cdo -outputtab,lon,lat,date,value tmp.nc >> outfile.txt #-- remove temporary file rm tmp.nc exit
-Karin
RE: -outputtab for multiple combinations of points - Added by Max Ballhausen over 5 years ago
Thanks for the quick reply. I will have to extract more than one pair of points. I just put one pair in for the sake of testing it :-)
I think the problem that I have is related to korn shell and how it deals with negative values.. Could that be the case?
RE: -outputtab for multiple combinations of points - Added by Max Ballhausen over 5 years ago
I solved it!
Instead of
set -A lon -0.963246
set -A lat 38.308579
it has to be
set A lon - 0.963246 38.308579
set -A lat -
Got the hint from this thread:
https://www.unix.com/shell-programming-and-scripting/186579-solved-capturing-output-korn-variable.html
Best
Max
RE: -outputtab for multiple combinations of points - Added by Max Ballhausen over 5 years ago
I solved it!
Instead of
set -A lon -0.963246
set -A lat 38.308579
it has to be
set -A lon -- -0.963246
set -A lat -- 38.308579
Got the hint from this thread:
https://www.unix.com/shell-programming-and-scripting/186579-solved-capturing-output-korn-variable.html
Best
Max
RE: -outputtab for multiple combinations of points - Added by Karin Meier-Fleischer over 5 years ago
Yup, that's the way when you want to use arrays. I should have mentioned that, too.
-Karin