cdo outputtab
Added by Muhammad Ramzan almost 2 years ago
I have a NetCDF file I want to get the data into the CSV file. I am using the following command.
cdo outputtab,date,lon,lat,value era_temperature_2016.nc > test_2016.csv
I am getting the following output
- date lon lat value
2016-01-01 72.9313 33.6871 286.3865
2016-01-01 72.9313 33.6871 287.8337
2016-02-01 72.9313 33.6871 289.5742
2016-02-01 72.9313 33.6871 291.0781
2016-03-01 72.9313 33.6871 292.2905
2016-03-01 72.9313 33.6871 293.7372
2016-04-01 72.9313 33.6871 296.703
2016-04-01 72.9313 33.6871 298.181
2016-05-01 72.9313 33.6871 303.2804
2016-05-01 72.9313 33.6871 304.7249
2016-06-01 72.9313 33.6871 304.5008
2016-06-01 72.9313 33.6871 305.9426
2016-07-01 72.9313 33.6871 301.9163
2016-07-01 72.9313 33.6871 303.3689
2016-08-01 72.9313 33.6871 301.4589
2016-08-01 72.9313 33.6871 302.9072
2016-09-01 72.9313 33.6871 301.5118
2016-09-01 72.9313 33.6871 302.9666
2016-10-01 72.9313 33.6871 300.3135
2016-10-01 72.9313 33.6871 301.8115
2016-11-01 72.9313 33.6871 294.0796
2016-11-01 72.9313 33.6871 295.5635
2016-12-01 72.9313 33.6871 290.7698
2016-12-01 72.9313 33.6871 292.2829
cdo outputtab: Processed 24 values from 1 variable over 12 timesteps [0.01s 44MB].
when I run the following command
cdo infon era_temperature_2016.nc
I get the following output
-1 : Date Time Level Gridsize Miss : Minimum Mean Maximum : Parameter name
1 : 2016-01-01 00:00:00 975 3795 0 : 279.24 288.05 294.10 : t
2 : 2016-01-01 00:00:00 1000 3795 0 : 281.57 289.20 294.85 : t
3 : 2016-02-01 00:00:00 975 3795 0 : 280.10 289.55 296.12 : t
4 : 2016-02-01 00:00:00 1000 3795 0 : 281.64 290.64 296.63 : t
5 : 2016-03-01 00:00:00 975 3795 0 : 285.34 293.21 299.40 : t
6 : 2016-03-01 00:00:00 1000 3795 0 : 286.83 294.51 300.75 : t
7 : 2016-04-01 00:00:00 975 3795 0 : 287.85 297.03 303.41 : t
8 : 2016-04-01 00:00:00 1000 3795 0 : 289.42 298.57 304.99 : t
9 : 2016-05-01 00:00:00 975 3795 0 : 292.71 301.67 309.10 : t
10 : 2016-05-01 00:00:00 1000 3795 0 : 294.11 303.24 310.79 : t
11 : 2016-06-01 00:00:00 975 3795 0 : 295.53 302.51 310.17 : t
12 : 2016-06-01 00:00:00 1000 3795 0 : 296.83 304.01 311.66 : t
13 : 2016-07-01 00:00:00 975 3795 0 : 296.17 302.09 312.87 : t
14 : 2016-07-01 00:00:00 1000 3795 0 : 297.26 303.51 314.38 : t
15 : 2016-08-01 00:00:00 975 3795 0 : 296.01 300.90 309.52 : t
16 : 2016-08-01 00:00:00 1000 3795 0 : 297.21 302.34 311.00 : t
17 : 2016-09-01 00:00:00 975 3795 0 : 295.40 300.27 308.99 : t
18 : 2016-09-01 00:00:00 1000 3795 0 : 296.71 301.79 310.48 : t
19 : 2016-10-01 00:00:00 975 3795 0 : 285.79 297.38 303.29 : t
20 : 2016-10-01 00:00:00 1000 3795 0 : 288.00 298.78 304.88 : t
21 : 2016-11-01 00:00:00 975 3795 0 : 280.24 292.93 298.61 : t
22 : 2016-11-01 00:00:00 1000 3795 0 : 282.15 294.09 299.74 : t
23 : 2016-12-01 00:00:00 975 3795 0 : 278.99 291.18 296.56 : t
24 : 2016-12-01 00:00:00 1000 3795 0 : 281.15 292.30 298.00 : t
How I can get the level variable when using this command
cdo outputtab,date,lon,lat,value era_temperature_2016.nc > test_2016.csv
Replies (17)
RE: cdo outputtab - Added by Karin Meier-Fleischer almost 2 years ago
See https://code.mpimet.mpg.de/projects/cdo/embedded/cdo.pdf#subsection.2.14.6 there is a table of the outputtab keywords and as you can see there is lev that can be used to retrieve the levels too.
This ticket is linked to the previous https://code.mpimet.mpg.de/boards/2/topics/13777
RE: cdo outputtab - Added by Muhammad Ramzan almost 2 years ago
Thanks
Hi
I am trying to run this command on the netcdf file but getting the error please
cdo outputtab,date,lon,lat,lev,value precipitation_2017.nc > 2017.txt
the error message is please
cdo outputtab: Open failed on >precipitation_2017.nc<
Unsupported file type
RE: cdo outputtab - Added by Karin Meier-Fleischer almost 2 years ago
Have a look at your data file.
ncdump -h precipitation_2017.nc ncdump: precipitation_2017.nc: NetCDF: Unknown file format file precipitation_2017.nc precipitation_2017.nc: gzip compressed data, was "dataset-satellite-precipitation-ec25f67a-d1b5-43de-aead-454aa09d0700.tar", last modified: Mon Dec 12 11:20:01 2022, max compression, original size modulo 2^32 389120
RE: cdo outputtab - Added by Muhammad Ramzan almost 2 years ago
Hello
I am trying to run the following code. cdo outputtab command is triminng the values of lat and longitude. How I can keep the full length of the lat and longitude as given in the coords.txt please
Thanks
#!/bin/bash infile="X_2021.nc" coords="coords.txt" while read line do line=$(echo $line | sed -e 's/\r//g') station=$(echo $line | cut -d ' ' -f 1) #-- skip header line if [[ "$station" == "station" ]]; then continue; fi #-- select station coordinates lat=$(echo $line | cut -d ' ' -f 2) lon=$(echo $line | cut -d ' ' -f 3) station="${station}_${lat}_${lon}" #-- extract the station data cdo -remapnn,"lon=${lon}_lat=${lat}" ${infile} ${station}_out.nc cdo outputtab,date,time,year,month,day,x,y,lon,lat,lev,name,value ${station}_out.nc > ${station}_out.txt done < $coords
coords.txt (15.7 KB) coords.txt | |||
X_2021.nc (536 KB) X_2021.nc |
RE: cdo outputtab - Added by Karin Meier-Fleischer almost 2 years ago
You can set the number of digits for lon and lat in outputtab with lon:<num_digits> and lat:<num_digits>. In your case:
cdo outputtab,date,time,year,month,day,x,y,lon:10,lat:10,lev,name,value ${station}_out.nc > ${station}_out.txt
RE: cdo outputtab - Added by Muhammad Ramzan almost 2 years ago
Thanks
it is not working please I am getting the same output please
cdo outputtab,date,time,year,month,day,x,y,lon:10,lat:10,lev,name,value ${station}_out.nc > ${station}_out.txt
Please see the attached files.
Thanks
RE: cdo outputtab - Added by Karin Meier-Fleischer almost 2 years ago
Hm, that seems not to work anymore for version 2.1.0 and 2.0.5 too. You can use a workaround to replace the wrong lon and lat values by the correct ones:
cdo -outputtab,date,time,year,month,day,x,y,lon,lat,lev,name,value \ outputs/${station}_out.nc | \ awk '{$8="'${lon}'"; $9="'${lat}'";print}' > outputs/${station}_out.txt
E.g. Output:
# date time year month day x 72.57155787 36.44991823 lat lev name value 2021-01-01 00:00:00 2021 1 1 72.5716 36.4499 72.57155787 36.44991823 0 u10 -0.001083623 2021-01-01 00:00:00 2021 1 1 72.5716 36.4499 72.57155787 36.44991823 0 v10 -0.05580788 2021-01-01 00:00:00 2021 1 1 72.5716 36.4499 72.57155787 36.44991823 0 si10 1.121761 ...
RE: cdo outputtab - Added by Karin Meier-Fleischer almost 2 years ago
Oops, it also changes the header line $8 and $9
RE: cdo outputtab - Added by Karin Meier-Fleischer almost 2 years ago
Okay, to keep the header untouched and delete the last two cdo info messages from the output file do
cdo -s -outputtab,date,time,year,month,day,x,y,lon,lat,lev,name,value \ outputs/${station}_out.nc | \ awk 'NR>1 {$8="'${lon}'"; $9="'${lat}'"}1' > outputs/${station}_out.txt
RE: cdo outputtab - Added by Muhammad Ramzan almost 2 years ago
Thanks
I am getting the following error message please
do remapnn: Nearest neighbor weights from lonlat (69x55) to lonlat (1x1) grid
cdo remapnn: Processed 273240 values from 6 variables over 12 timesteps [0.02s 46MB].
./values2.sh: line 22: outputs/1_36.44991823_72.57155787_out.txt: No such file or directory
cdo outputtab: Open failed on >outputs/1_36.44991823_72.57155787_out.nc<
No such file or directory
cdo remapnn: Nearest neighbor weights from lonlat (69x55) to lonlat (1x1) grid
cdo remapnn: Processed 273240 values from 6 variables over 12 timesteps [0.02s 45MB].
./values2.sh: line 22: outputs/2_35.89191425_71.72687311_out.txt: No such file or directory
cdo outputtab: Open failed on >outputs/2_35.89191425_71.72687311_out.nc<
No such file or directory
cdo remapnn: Nearest neighbor weights from lonlat (69x55) to lonlat (1x1) grid
cdo remapnn: Processed 273240 values from 6 variables over 12 timesteps [0.02s 46MB].
./values2.sh: line 22: outputs/3_35.16956641_71.8344582_out.txt: No such file or directory
cdo outputtab: Open failed on >outputs/3_35.16956641_71.8344582_out.nc<
No such file or directory
cdo remapnn: Nearest neighbor weights from lonlat (69x55) to lonlat (1x1) grid
cdo remapnn: Processed 273240 values from 6 variables over 12 timesteps [0.02s 45MB].
./values2.sh: line 22: outputs/4_35.42472896_72.16393117_out.txt: No such file or directory
cdo outputtab: Open failed on >outputs/4_35.42472896_72.16393117_out.nc<
No such file or directory
cdo remapnn: Nearest neighbor weights from lonlat (69x55) to lonlat (1x1) grid
cdo remapnn: Processed 273240 values from 6 variables over 12 timesteps [0.02s 45MB].
./values2.sh: line 22: outputs/5_35.00569553_71.77647769_out.txt: No such file or directory
cdo outputtab: Open failed on >outputs/5_35.00569553_71.77647769_out.nc<
No such file or directory
cdo remapnn: Nearest neighbor weights from lonlat (69x55) to lonlat (1x1) grid
cdo remapnn: Processed 273240 values from 6 variables over 12 timesteps [0.02s 46MB].
./values2.sh: line 22: outputs/6_34.94443242_71.65510941_out.txt: No such file or directory
cdo outputtab: Open failed on >outputs/6_34.94443242_71.65510941_out.nc<
No such file or directory
values2.sh (702 Bytes) values2.sh |
RE: cdo outputtab - Added by Karin Meier-Fleischer almost 2 years ago
You have to delete the outputs/ in front of the file names. I didn't want to have all the files in the same directory as the script.
RE: cdo outputtab - Added by Karin Meier-Fleischer almost 2 years ago
You should read and understand what I have send you because it's on you to take further care on it.
RE: cdo outputtab - Added by Muhammad Ramzan almost 2 years ago
Thanks
it worked but new code is repeating the observations
- date time year month day x y lon lat lev name value
2021-01-01 00:00:00 2021 1 1 71.6551 34.9444 71.65510941 34.94443242 0 u10 0.2442991
2021-01-01 00:00:00 2021 1 1 71.6551 34.9444 71.65510941 34.94443242 0 u10 0.2442991
2021-01-01 00:00:00 2021 1 1 71.6551 34.9444 71.65510941 34.94443242 0 v10 -1.768886
2021-01-01 00:00:00 2021 1 1 71.6551 34.9444 71.65510941 34.94443242 0 v10 -1.768886
2021-01-01 00:00:00 2021 1 1 71.6551 34.9444 71.65510941 34.94443242 0 si10 1.956628
2021-01-01 00:00:00 2021 1 1 71.6551 34.9444 71.65510941 34.94443242 0 si10 1.956628
2021-01-01 00:00:00 2021 1 1 71.6551 34.9444 71.65510941 34.94443242 0 t2m 277.643
2021-01-01 00:00:00 2021 1 1 71.6551 34.9444 71.65510941 34.94443242 0 t2m 277.643
2021-01-01 00:00:00 2021 1 1 71.6551 34.9444 71.65510941 34.94443242 0 sp 86510.81
2021-01-01 00:00:00 2021 1 1 71.6551 34.9444 71.65510941 34.94443242 0 sp 86510.81
2021-01-01 00:00:00 2021 1 1 71.6551 34.9444 71.65510941 34.94443242 0 tp 0.0006449826
2021-01-01 00:00:00 2021 1 1 71.6551 34.9444 71.65510941 34.94443242 0 tp 0.0006449826
6_34.94443242_71.65510941_2_out.txt (12 KB) 6_34.94443242_71.65510941_2_out.txt | |||
6_34.94443242_71.65510941_out.txt (6.64 KB) 6_34.94443242_71.65510941_out.txt | |||
values.sh (609 Bytes) values.sh | |||
values2.sh (688 Bytes) values2.sh |
RE: cdo outputtab - Added by Karin Meier-Fleischer almost 2 years ago
See the corrected command without the print in the awk call.
RE: cdo outputtab - Added by Muhammad Ramzan over 1 year ago
Hello
can you please help to modify the code in values2.sh , I have to run this code on multiple files air.1948 ... air.2020. and I want to the name of the txt files be name of filein front of the txt files. please help me how I can achieve this please
values2.sh (680 Bytes) values2.sh |
RE: cdo outputtab - Added by Karin Meier-Fleischer over 1 year ago
This not really a CDO question. If you need to work with shell scripts you have to learn more about it. There are multiple tutorials about bash, zsh and other Shells available.
To strip off the path prefix and the .nc suffix to get only the base name of a file, e.g.:
infile="/home/mygroup/myid/folder1/file1.nc" infile_base=$(basename -s .nc $infile)
Result: file1
RE: cdo outputtab - Added by Muhammad Ramzan over 1 year ago
Hello
I am working on the similar thing , this time nc file have lev variable I only want the values when lev variable is 1000, 925 and 850 in the txt files, how to d this in the code. the nc files have 17 layers of temperature, while running the values2.sh code its taking a lot of time.
Please help me with this
Thanks
values2.sh (726 Bytes) values2.sh |