CDO

Try

cdo remapnn,lon=${lon}_lat=${lat} temperature.nc $out.nc

Usually the underline '_' is a valid character for variable name. Thats why the shell tries to evaluate a variable called 'lon_lat', which isn't defined. the braces make this clear - I use them all the time to avoid situations like this. The funny part starts, if you have defined lon_lat somewhere else for a different purpose and therefore don't get an error message, but results, that might look ok in the first place ...

hth
ralf

I am trying to extract values to a series of points (meteo stations) as table. I have adapted the shell script and now it looks like this:

#!/bin/bash
while read p ; do
  echo $p
   cdo -outputtab,date,lon,lat,value -remapnn,lon=${lon}_lat=${lat}, T_ave.nc > $pct.txt
   done <stations.txt

...remapnn (Abort): Open failed on lon=_lat=!

code,lon,lat
57,27.17,46.1
19,23.56,46.06
3,21.35,46.13
55,26.91,46.53
15,23.11,44.48

Check this

while read p ; do
  echo $p; lon=$(echo $p | cut -d ',' -f 2); lat=$(echo $p | cut -d ',' -f 3);
   cdo -outputtab,date,lon,lat,value -remapnn,lon=${lon}_lat=${lat} -topo > $pct.txt
   done <stations.txt

in your loop, there is no definition of lon and lat - if you just use them, they are empty by definition.

hth
ralf

Thanks! It worked like a charm.
Bogdan

Hi all,

I am also trying to write a loop to extract time series for a list of grid points using the resource here but I am having issues. When I manually type in the grid points I can obtain the time series without any issues. For example, I successfully used:

cdo -outputtab,date,value -remapnn,lon=80.625_lat=30.625 $ifile> ofile1.txt
cdo -outputtab,date,value -remapnn,lon=80.875_lat=30.625 $ifile> ofile2.txt


But when I try to loop through the same lat-lons, it does not work. I wrote a loop to read lat long from a text file formatted as below:
station,lat,lon
TRMMp80,30.625,80.625
TRMMp81,30.625,80.875

My code adapted from here is:

ifile="TRMM3b42v7_1998-2016_NPL.nc4" #.nc4 file to read
istnlst="TRMM_selpts.txt" #list of lat-lon for which to extract TS
ofile="tab.txt"
while read p; do
echo -e "Reading station..... "$p
stn=$(echo $p | cut -d ',' -f 1)
out="TSfiles/"$stn$ofile
lat=$(echo $p | cut -d ',' -f 2); lon=$(echo $p | cut -d ',' -f 3);
cdo -outputtab,date,value -remapnn,lon=${lon}_lat=${lat} $ifile>$out
sed -i '' -e "s/value/$stn/1" $out #replace value column name w/ stn name
echo
done <$istnlst


I get the following error for the first lat-lon combination. Attached text file shows the messages I get when I run the above mentioned codes.

_lat=30.625 TRMM3b42v7_1998-2016_NPLpcpSWAT.nc4 (pipe1.1)".
_lat=30.625!nn (Abort): Open failed on lon=80.625


The second loop runs without errors. But the problem gets worse when I add more stations to the list. I cannot extract time series for a single grid when I give a longer list of stations. Any idea why this is happening? Any help is greatly appreciated.

Hi! I tried Ralph Mueller's structure with a different command and I am getting the following error. Any thoughts?

The structure of my csv file is the same as in the example:
1,23,18,206,201
2,34,29,299,294
3,43,39,295,290
4,38,33,239,234

My code

while read n ; do
echo $n;
latmax=$(echo $n | cut -d ',' -f 2);
latmin=$(echo $n | cut -d ',' -f 3);
lonmax=$(echo $n | cut -d ',' -f 4);
lonmin=$(echo $n | cut -d ',' -f 5);
cdo sellonlatbox,${lonmin},${lonmax},${latmin},${latmax}, outfile_sst.nc outfile$n.nc ;
done < coordextended2.csv

result:

1,23,18,206,201

< contains invalid character at position 4!201
2,34,29,299,294

< contains invalid character at position 4!294
3,43,39,295,290

< contains invalid character at position 4!290
4,38,33,239,234

< contains invalid character at position 4!234
5,-59,-65,307,298

Hi Sara!

I used this

while read n ; do
  echo $n;
  latmax=$(echo $n | cut -d ',' -f 2);
  latmin=$(echo $n | cut -d ',' -f 3);
  lonmax=$(echo $n | cut -d ',' -f 4);
  lonmin=$(echo $n | cut -d ',' -f 5);
  cdo -f nc sellonlatbox,${lonmin},${lonmax},${latmin},${latmax} -topo outfile$(echo $n | tr ',' '_').nc ;
done < coordextended2.csv

hth
ralf

I don't like ',' in filenames - that's why I replaced them with '_'

Hi Ralph!

Thank you so much for your help! I'm using version 1.9.5. Unfortunately, the code you sent is not working for me, I think it might have to do with how it reads the csv file because if I give it some numbers it does seem to work, for example:
lonmin=(50 51 52 53);
latmin=(40 41 42 43);
latmax=(41 42 43 44);
lonmax=(51 52 53 54);

for n in {0..3};
do
cdo sellonlatbox,${lonmin[$n]},${lonmax[$n]},${latmin[$n]},${latmax[$n]}, outfile_sst.nc outfile$n.nc ;
done

But I when ask it to read the coordinates from coordextended2csv, the loop doesnt work. I am attaching the csv file that I use. The netcdf is too large to attach here.

hi Sara!

I think I got it. Your coordinates files is a DOS-formated text. you have to format it to UNIX with a tool like dos2unix. I transformed your upload and it worked out of the box. Normally this tool is available as a package for many linux distros.

hth
ralf

Thank you so much, Ralph!! that was the issue

you're welcome

Ralf Mueller wrote:

Try[...]

Usually the underline '_' is a valid character for variable name. Thats why the shell tries to evaluate a variable called 'lon_lat', which isn't defined. the braces make this clear - I use them all the time to avoid situations like this. The funny part starts, if you have defined lon_lat somewhere else for a different purpose and therefore don't get an error message, but results, that might look ok in the first place ...

hth
ralf

I used this solution and that of Bogdan (below this answer), but I am getting the same error.

remapp.bash: line 5: syntax error near unexpected token `done'
remapp.bash: line 5: ` done <stations.txt'

My stations.txt file looks like this.

Stations,lat,lon
Simulia,23.29,86.36
Tusuma,23.08,86.66
Mukutmanipur,22.95,86.77
Phulberia,22.92,86.62
Staten,22.62,87.01
Main_outlet,22.43,87.44
Mohanpur,22.40,87.34

What should I do? Please help me.

Akhi

Please upload your script or the command line. Every detail counts for a good answer ;-)

cheers
ralf

Maria Kar wrote:

I want to apply the "remapnn" command in order to extract time series for a set of points that are stored in the "Station.csv" file. The stucture of the Station.csv file is the following:

Station Lon Lat
Maputo 32.57 -25.92
XaiXai 33.63 -25.06
Inhambane 35.38 -23.87
Vilankulo 35.32 -22
Beira 34.9 -19.8
Chimoio 33.47 -19.12
Tete 33.64 -16.11

for -f "tokens=1,2,3 skip=1 rmims=," i in (Station_Table2.csv) do (
wgrib2 GSM_GPV_Rgl_I$yymmdd_grib2.bin -match ":APCP:" -s -lon j k > gpv_rain/$yymmdd_i_rain_gpv.csv
wgrib2 GSM_GPV_Rgl_I$yymmdd_grib2.bin -match ":TCDC:" -s -lon j k > gpv_cloud/$yymmdd_i_tcdc_gpv.csv
wgrib2 GSM_GPV_Rgl_I$yymmdd_grib2.bin -match ":UGRD:" -s -lon j k > gpv_wind/$yymmdd_i_U_gpv.csv
wgrib2 GSM_GPV_Rgl_I$yymmdd_grib2.bin -match ":VGRD:" -s -lon j k > gpv_wind/$yymmdd_i_V_gpv.csv

echo i j k
)
echo Now, completed retreaving GSM05 Rain_Cloud_Wind gpvdata Initial= $yymmdd$
the output have to be 2021061500_Maputo_rain_gpv.csv.

this for-loop does not work in bash - which shell is it then?

this is batch script. I need help to convert to bash script.

Hello Ralf,

I have netcdf data of daily precipitation for the period of 1 year (prec_2015.nc). I want to extract the value in each time series based on multiple coordinates (coordinates.txt).

I use the loop that you provided above. However, it failed to generate the output.

#!/usr/bin/bash
while read p ; do
  echo $p; 
  lon=$(echo $p | cut -d ',' -f 2); 
  lat=$(echo $p | cut -d ',' -f 3);
  cdo -outputtab,date,lon,lat,value -remapnn,lon=${lon}_lat=${lat} -topo > $extracted.txt
  done <coordinate.txt

$ ./script.sh
id,lon,lat
cdo(2) remapnn: Process started
cdo(3) topo: Process started
!do(2) remapnn (Abort): Open failed on lon=lon_lat=lat
1,27.95,-26.672
cdo(2) remapnn: Process started
cdo(3) topo: Process started
cdo(3) topo: Set default filetype to GRIB
cdo(2) remapnn: Nearest neighbor weights from lonlat (720x360) to lonlat (1x1) grid
cdo(3) topo:  [0.01s]
cdo(2) remapnn: Processed 259200 values from 1 variable over 1 timestep [0.01s]
cdo outputtab: Processed 1 value from 1 variable over 1 timestep [0.01s 10MB]
.
.
.
.
.

Could you help me to solve this error?
The files that I used to run the loop are attached.

I still do not understand why did you use "-topo" instead of the input file?
How is the input file linked into the loop?

I use CDO version 1.9.4rc1

Thank you
Christian

Hi Christ,

the coordinate.txt file contains a header line which has to be skipped. Instead of echo ${p} use tail -n +2 in your script to skip the first line.

Hi Karin,

Thank you for your prompt response.
I have tried, but it still gets an error.
Could you show me how exactly I use tail -n +2 in the script?

Best
Christ

I choose another way to create 20 files containing the data for all timesteps. First you have to add a newline to the last line of the coordinates.txt file. Then use the following script:

#!/usr/bin/env bash

coords=coordinate.txt
infile=prec_2015.nc

rm -f extracted*.txt

while IFS=, read -r id lon lat
do  
  if [[ $id == "id" ]]; then
      continue
  fi

  echo "$id $lon  $lat" 

  cdo -s -outputtab,date,lon,lat,value -remapnn,lon=${lon}_lat=${lat} \
       $infile >> extracted_${id}.txt

done < $coords

Run the script

bash script.sh

Thank you so much, Karin. It works.
I really appreciate your guidance.

Best
Christ

Hello Karin,

Following my comment above,

I am replacing the infile with the infile that contains multiple variables, such as: prec, tmax, tmin, rad, and wind (attached merge_5_variables_example.nc)

Is there a way in cdo to obtain output table with multiple variables where each variable is in a separate column?

This is the output that I expect

 date          lon    lat    pr tasmax tasmin rsds wind 
2015-01-01  27.95 -26.672    x     x    x    x   x
2015-01-02  27.95 -26.672
2015-01-03  27.95 -26.672
.
.
.
.
.    

In the extracted output (extracted_1.txt)

#      date     name    lon    lat    value 
 2015-01-01       pr  27.95 -26.672        0 
 2015-01-01   tasmax  27.95 -26.672  23.8035 
 2015-01-01   tasmin  27.95 -26.672  12.2056 
 2015-01-01     rsds  27.95 -26.672  23.9925 
 2015-01-01  sfcWind  27.95 -26.672  1.96399 
 2015-01-02       pr  27.95 -26.672 1.06814e-06 
 2015-01-02   tasmax  27.95 -26.672  27.0092 
 2015-01-02   tasmin  27.95 -26.672  12.0303 
 2015-01-02     rsds  27.95 -26.672  31.6022 
 2015-01-02  sfcWind  27.95 -26.672  1.65828 
 2015-01-03       pr  27.95 -26.672 1.52581e-06 
 2015-01-03   tasmax  27.95 -26.672  30.6931 
 2015-01-03   tasmin  27.95 -26.672  13.9199 
 2015-01-03     rsds  27.95 -26.672  29.5652 
 2015-01-03  sfcWind  27.95 -26.672  1.09478 

Adjusting the table manually is gonna hurt me because I have more than 200.000 coordinate points.

Looking forward for your response.