Project

General

Profile

cdo outputtab

Added by Muhammad Ramzan almost 2 years ago

I have a NetCDF file I want to get the data into the CSV file. I am using the following command.

cdo outputtab,date,lon,lat,value era_temperature_2016.nc > test_2016.csv

I am getting the following output

  1. date lon lat value
    2016-01-01 72.9313 33.6871 286.3865
    2016-01-01 72.9313 33.6871 287.8337
    2016-02-01 72.9313 33.6871 289.5742
    2016-02-01 72.9313 33.6871 291.0781
    2016-03-01 72.9313 33.6871 292.2905
    2016-03-01 72.9313 33.6871 293.7372
    2016-04-01 72.9313 33.6871 296.703
    2016-04-01 72.9313 33.6871 298.181
    2016-05-01 72.9313 33.6871 303.2804
    2016-05-01 72.9313 33.6871 304.7249
    2016-06-01 72.9313 33.6871 304.5008
    2016-06-01 72.9313 33.6871 305.9426
    2016-07-01 72.9313 33.6871 301.9163
    2016-07-01 72.9313 33.6871 303.3689
    2016-08-01 72.9313 33.6871 301.4589
    2016-08-01 72.9313 33.6871 302.9072
    2016-09-01 72.9313 33.6871 301.5118
    2016-09-01 72.9313 33.6871 302.9666
    2016-10-01 72.9313 33.6871 300.3135
    2016-10-01 72.9313 33.6871 301.8115
    2016-11-01 72.9313 33.6871 294.0796
    2016-11-01 72.9313 33.6871 295.5635
    2016-12-01 72.9313 33.6871 290.7698
    2016-12-01 72.9313 33.6871 292.2829
    cdo outputtab: Processed 24 values from 1 variable over 12 timesteps [0.01s 44MB].

when I run the following command

cdo infon era_temperature_2016.nc

I get the following output

-1 : Date Time Level Gridsize Miss : Minimum Mean Maximum : Parameter name
1 : 2016-01-01 00:00:00 975 3795 0 : 279.24 288.05 294.10 : t
2 : 2016-01-01 00:00:00 1000 3795 0 : 281.57 289.20 294.85 : t
3 : 2016-02-01 00:00:00 975 3795 0 : 280.10 289.55 296.12 : t
4 : 2016-02-01 00:00:00 1000 3795 0 : 281.64 290.64 296.63 : t
5 : 2016-03-01 00:00:00 975 3795 0 : 285.34 293.21 299.40 : t
6 : 2016-03-01 00:00:00 1000 3795 0 : 286.83 294.51 300.75 : t
7 : 2016-04-01 00:00:00 975 3795 0 : 287.85 297.03 303.41 : t
8 : 2016-04-01 00:00:00 1000 3795 0 : 289.42 298.57 304.99 : t
9 : 2016-05-01 00:00:00 975 3795 0 : 292.71 301.67 309.10 : t
10 : 2016-05-01 00:00:00 1000 3795 0 : 294.11 303.24 310.79 : t
11 : 2016-06-01 00:00:00 975 3795 0 : 295.53 302.51 310.17 : t
12 : 2016-06-01 00:00:00 1000 3795 0 : 296.83 304.01 311.66 : t
13 : 2016-07-01 00:00:00 975 3795 0 : 296.17 302.09 312.87 : t
14 : 2016-07-01 00:00:00 1000 3795 0 : 297.26 303.51 314.38 : t
15 : 2016-08-01 00:00:00 975 3795 0 : 296.01 300.90 309.52 : t
16 : 2016-08-01 00:00:00 1000 3795 0 : 297.21 302.34 311.00 : t
17 : 2016-09-01 00:00:00 975 3795 0 : 295.40 300.27 308.99 : t
18 : 2016-09-01 00:00:00 1000 3795 0 : 296.71 301.79 310.48 : t
19 : 2016-10-01 00:00:00 975 3795 0 : 285.79 297.38 303.29 : t
20 : 2016-10-01 00:00:00 1000 3795 0 : 288.00 298.78 304.88 : t
21 : 2016-11-01 00:00:00 975 3795 0 : 280.24 292.93 298.61 : t
22 : 2016-11-01 00:00:00 1000 3795 0 : 282.15 294.09 299.74 : t
23 : 2016-12-01 00:00:00 975 3795 0 : 278.99 291.18 296.56 : t
24 : 2016-12-01 00:00:00 1000 3795 0 : 281.15 292.30 298.00 : t

How I can get the level variable when using this command

cdo outputtab,date,lon,lat,value era_temperature_2016.nc > test_2016.csv


Replies (17)

RE: cdo outputtab - Added by Karin Meier-Fleischer almost 2 years ago

See https://code.mpimet.mpg.de/projects/cdo/embedded/cdo.pdf#subsection.2.14.6 there is a table of the outputtab keywords and as you can see there is lev that can be used to retrieve the levels too.

This ticket is linked to the previous https://code.mpimet.mpg.de/boards/2/topics/13777

RE: cdo outputtab - Added by Muhammad Ramzan almost 2 years ago

Thanks

Hi

I am trying to run this command on the netcdf file but getting the error please

cdo outputtab,date,lon,lat,lev,value precipitation_2017.nc > 2017.txt

the error message is please

cdo outputtab: Open failed on >precipitation_2017.nc<
Unsupported file type

RE: cdo outputtab - Added by Karin Meier-Fleischer almost 2 years ago

Have a look at your data file.

ncdump -h precipitation_2017.nc 
ncdump: precipitation_2017.nc: NetCDF: Unknown file format

file precipitation_2017.nc 
precipitation_2017.nc: gzip compressed data, was "dataset-satellite-precipitation-ec25f67a-d1b5-43de-aead-454aa09d0700.tar", last modified: Mon Dec 12 11:20:01 2022, max compression, original size modulo 2^32 389120

RE: cdo outputtab - Added by Muhammad Ramzan almost 2 years ago

Hello

I am trying to run the following code. cdo outputtab command is triminng the values of lat and longitude. How I can keep the full length of the lat and longitude as given in the coords.txt please

Thanks

#!/bin/bash

infile="X_2021.nc" 
coords="coords.txt" 

while read line
do
   line=$(echo $line | sed -e 's/\r//g')
   station=$(echo $line | cut -d ' ' -f 1)
   #-- skip header line
   if [[ "$station" == "station" ]]; then continue; fi
   #-- select station coordinates
   lat=$(echo $line | cut -d ' ' -f 2)
   lon=$(echo $line | cut -d ' ' -f 3)
   station="${station}_${lat}_${lon}" 
   #-- extract the station data
   cdo -remapnn,"lon=${lon}_lat=${lat}" ${infile} ${station}_out.nc
   cdo outputtab,date,time,year,month,day,x,y,lon,lat,lev,name,value ${station}_out.nc > ${station}_out.txt
done < $coords

RE: cdo outputtab - Added by Karin Meier-Fleischer almost 2 years ago

You can set the number of digits for lon and lat in outputtab with lon:<num_digits> and lat:<num_digits>. In your case:

cdo outputtab,date,time,year,month,day,x,y,lon:10,lat:10,lev,name,value ${station}_out.nc > ${station}_out.txt

RE: cdo outputtab - Added by Muhammad Ramzan almost 2 years ago

Thanks

it is not working please I am getting the same output please

cdo outputtab,date,time,year,month,day,x,y,lon:10,lat:10,lev,name,value ${station}_out.nc > ${station}_out.txt

Please see the attached files.

Thanks

RE: cdo outputtab - Added by Karin Meier-Fleischer almost 2 years ago

Hm, that seems not to work anymore for version 2.1.0 and 2.0.5 too. You can use a workaround to replace the wrong lon and lat values by the correct ones:

  cdo -outputtab,date,time,year,month,day,x,y,lon,lat,lev,name,value \
        outputs/${station}_out.nc | \
        awk '{$8="'${lon}'"; $9="'${lat}'";print}' > outputs/${station}_out.txt

E.g. Output:

# date time year month day x 72.57155787 36.44991823 lat lev name value
2021-01-01 00:00:00 2021 1 1 72.5716 36.4499 72.57155787 36.44991823 0 u10 -0.001083623
2021-01-01 00:00:00 2021 1 1 72.5716 36.4499 72.57155787 36.44991823 0 v10 -0.05580788
2021-01-01 00:00:00 2021 1 1 72.5716 36.4499 72.57155787 36.44991823 0 si10 1.121761
...

RE: cdo outputtab - Added by Karin Meier-Fleischer almost 2 years ago

Oops, it also changes the header line $8 and $9 :D

RE: cdo outputtab - Added by Karin Meier-Fleischer almost 2 years ago

Okay, to keep the header untouched and delete the last two cdo info messages from the output file do

  cdo -s -outputtab,date,time,year,month,day,x,y,lon,lat,lev,name,value \
        outputs/${station}_out.nc | \
        awk 'NR>1 {$8="'${lon}'"; $9="'${lat}'"}1' > outputs/${station}_out.txt

RE: cdo outputtab - Added by Muhammad Ramzan almost 2 years ago

Thanks

I am getting the following error message please

do remapnn: Nearest neighbor weights from lonlat (69x55) to lonlat (1x1) grid
cdo remapnn: Processed 273240 values from 6 variables over 12 timesteps [0.02s 46MB].
./values2.sh: line 22: outputs/1_36.44991823_72.57155787_out.txt: No such file or directory
cdo outputtab: Open failed on >outputs/1_36.44991823_72.57155787_out.nc<
No such file or directory
cdo remapnn: Nearest neighbor weights from lonlat (69x55) to lonlat (1x1) grid
cdo remapnn: Processed 273240 values from 6 variables over 12 timesteps [0.02s 45MB].
./values2.sh: line 22: outputs/2_35.89191425_71.72687311_out.txt: No such file or directory
cdo outputtab: Open failed on >outputs/2_35.89191425_71.72687311_out.nc<
No such file or directory
cdo remapnn: Nearest neighbor weights from lonlat (69x55) to lonlat (1x1) grid
cdo remapnn: Processed 273240 values from 6 variables over 12 timesteps [0.02s 46MB].
./values2.sh: line 22: outputs/3_35.16956641_71.8344582_out.txt: No such file or directory
cdo outputtab: Open failed on >outputs/3_35.16956641_71.8344582_out.nc<
No such file or directory
cdo remapnn: Nearest neighbor weights from lonlat (69x55) to lonlat (1x1) grid
cdo remapnn: Processed 273240 values from 6 variables over 12 timesteps [0.02s 45MB].
./values2.sh: line 22: outputs/4_35.42472896_72.16393117_out.txt: No such file or directory
cdo outputtab: Open failed on >outputs/4_35.42472896_72.16393117_out.nc<
No such file or directory
cdo remapnn: Nearest neighbor weights from lonlat (69x55) to lonlat (1x1) grid
cdo remapnn: Processed 273240 values from 6 variables over 12 timesteps [0.02s 45MB].
./values2.sh: line 22: outputs/5_35.00569553_71.77647769_out.txt: No such file or directory
cdo outputtab: Open failed on >outputs/5_35.00569553_71.77647769_out.nc<
No such file or directory
cdo remapnn: Nearest neighbor weights from lonlat (69x55) to lonlat (1x1) grid
cdo remapnn: Processed 273240 values from 6 variables over 12 timesteps [0.02s 46MB].
./values2.sh: line 22: outputs/6_34.94443242_71.65510941_out.txt: No such file or directory
cdo outputtab: Open failed on >outputs/6_34.94443242_71.65510941_out.nc<
No such file or directory

values2.sh (702 Bytes) values2.sh

RE: cdo outputtab - Added by Karin Meier-Fleischer almost 2 years ago

You have to delete the outputs/ in front of the file names. I didn't want to have all the files in the same directory as the script.

RE: cdo outputtab - Added by Karin Meier-Fleischer almost 2 years ago

You should read and understand what I have send you because it's on you to take further care on it.

RE: cdo outputtab - Added by Muhammad Ramzan almost 2 years ago

Thanks

it worked but new code is repeating the observations

  1. date time year month day x y lon lat lev name value
    2021-01-01 00:00:00 2021 1 1 71.6551 34.9444 71.65510941 34.94443242 0 u10 0.2442991
    2021-01-01 00:00:00 2021 1 1 71.6551 34.9444 71.65510941 34.94443242 0 u10 0.2442991
    2021-01-01 00:00:00 2021 1 1 71.6551 34.9444 71.65510941 34.94443242 0 v10 -1.768886
    2021-01-01 00:00:00 2021 1 1 71.6551 34.9444 71.65510941 34.94443242 0 v10 -1.768886
    2021-01-01 00:00:00 2021 1 1 71.6551 34.9444 71.65510941 34.94443242 0 si10 1.956628
    2021-01-01 00:00:00 2021 1 1 71.6551 34.9444 71.65510941 34.94443242 0 si10 1.956628
    2021-01-01 00:00:00 2021 1 1 71.6551 34.9444 71.65510941 34.94443242 0 t2m 277.643
    2021-01-01 00:00:00 2021 1 1 71.6551 34.9444 71.65510941 34.94443242 0 t2m 277.643
    2021-01-01 00:00:00 2021 1 1 71.6551 34.9444 71.65510941 34.94443242 0 sp 86510.81
    2021-01-01 00:00:00 2021 1 1 71.6551 34.9444 71.65510941 34.94443242 0 sp 86510.81
    2021-01-01 00:00:00 2021 1 1 71.6551 34.9444 71.65510941 34.94443242 0 tp 0.0006449826
    2021-01-01 00:00:00 2021 1 1 71.6551 34.9444 71.65510941 34.94443242 0 tp 0.0006449826

RE: cdo outputtab - Added by Karin Meier-Fleischer almost 2 years ago

See the corrected command without the print in the awk call.

RE: cdo outputtab - Added by Muhammad Ramzan over 1 year ago

Hello

can you please help to modify the code in values2.sh , I have to run this code on multiple files air.1948 ... air.2020. and I want to the name of the txt files be name of filein front of the txt files. please help me how I can achieve this please

values2.sh (680 Bytes) values2.sh

RE: cdo outputtab - Added by Karin Meier-Fleischer over 1 year ago

This not really a CDO question. If you need to work with shell scripts you have to learn more about it. There are multiple tutorials about bash, zsh and other Shells available.

To strip off the path prefix and the .nc suffix to get only the base name of a file, e.g.:

infile="/home/mygroup/myid/folder1/file1.nc" 

infile_base=$(basename -s .nc $infile)

Result: file1

RE: cdo outputtab - Added by Muhammad Ramzan over 1 year ago

Hello

I am working on the similar thing , this time nc file have lev variable I only want the values when lev variable is 1000, 925 and 850 in the txt files, how to d this in the code. the nc files have 17 layers of temperature, while running the values2.sh code its taking a lot of time.

Please help me with this

Thanks

values2.sh (726 Bytes) values2.sh
    (1-17/17)