Text to netCDF - for multiple years of daily data
Added by Sarah Roffe about 3 years ago
Hi,
I have 43 years of data stored in a text file with the columns following as time, lat, lon, tmax. I have been trying to convert this data to a netCDF file using R, but I have been struggling with no luck.
Is there any way to undertake this task using CDO? I would really appreciate any advice, and have attached a sample text file for the first year of my data for help in advising me. Further information, is that my data has 100 longitudes by 80 latitudes, with the first longitude at 5.25 and the first latitude at -0.25 with a grid resolution of 0.5.
Kind regards,
Sarah
| tester_1979.txt (121 MB) tester_1979.txt |
Replies (3)
RE: Text to netCDF - for multiple years of daily data - Added by Karin Meier-Fleischer about 3 years ago
Hi Sara,
you need to do some work before you can convert the tmax data to netCDF file. The best is to write a script which reads the data, creates the grid description file and calls cdo to write the netCDF file. There are some examples in this forum, but here is the script which works fine for your test data.
#!/bin/ksh
#
# 100 longitudes 5.25 incr. 0.5
# 80 latitudes -0.25 incr. 0.5
#
# "X","time","lat","lon","tmax"
# "1",1979-01-01,-0.25,5.25,NA
# "2",1979-01-01,-0.25,5.75,NA
# "3",1979-01-01,-0.25,6.25,28.1435432434082
# "4",1979-01-01,-0.25,6.75,28.1435451507568
# "5",1979-01-01,-0.25,7.25,NA
input="$HOME/Downloads/tester_1979.txt"
varname="tmax"
timeaxis="1979-01-01,00:12:00,1days"
missing=-999.99
#-- read tmax data and set 'NA' to -999.99 (missing value)
tail -n +2 $input | cut -d ',' -f 5 | sed -e 's|NA|-999.99|g'> tmax.txt
#-- generate the grid description file for $input
cat << EOF > gridfile.txt
gridtype = lonlat
gridsize = 8000
xname = lon
xlongname = longitude
xunits = degrees_east
yname = lat
ylongname = latitude
yunits = degrees_north
xsize = 100
xfirst = 5.25
xinc = 0.5
ysize = 80
yfirst = -0.25
yinc = -0.5
EOF
#-- convert to netCDF
cdo -r -f nc -settaxis,${timeaxis} \
-setname,${varname} \
-setctomiss,${missing} \
-input,gridfile.txt \
tmax.nc < tmax.txt
-Karin
RE: Text to netCDF - for multiple years of daily data - Added by Ralf Mueller about 3 years ago
here is a python version of the final call of Karin's script
cdo = Cdo()
cdo.debug = True
gridfile = 'gridfile.txt'
inputfile = 'tmax.txt'
cdo.settaxis('1979-01-01,00:12:00,1days',
options = ' -r -f nc',
input = "-setname,tmax -setctomiss,-999.99 -input,{} tmax.nc < ".format(gridfile),
output = inputfile)
the rest should be straight forward python, but this last call is tricky, because for making it work you have to use the input file as an output key.
RE: Text to netCDF - for multiple years of daily data - Added by Sarah Roffe about 3 years ago
Hi Karin and Ralph,
Thanks for kindly helping me out with this. It is truly appreciated.
I managed to convert my data to netCDF with no issues.
Regards,
Sarah