How to delete a list of dates from one netcdf file and append it

Added by Lyndon Mark Olaguera almost 4 years ago

Dear CDO experts,

I have one file vwnd_1979-2020.nc and I want to delete a list of dates from this file.

Here' a portion of the text file containing the dates (YYYY-MM-DD). The filename is "break_first_day.txt"

I want to delete +10 days after these dates. Here's a script for reading the above file.


#!/bin/bash 

file='break_first_day.txt'

for lead in 10
do
i=1
while IFS= read -r line; do

#Reading each line   
test=$(date -d "$line +$one days" +%Y-%m-%d)
echo $test

year=$(echo $test | cut -c1-4)
echo $year
month=$(echo $test | cut -c6-7)
echo $month
day=$(echo $test | cut -c9-10)
echo $day

cdo -O delete,day=$day,month=$month,year=$year vwnd_1979-2020.nc vwnd_1979-2020.nc

i=$((i+1))
done < $file
done

I got the following errors and the dates are not actually deleted.
Any suggestions on how to do this correctly?
I think my problem is how to append the file not overwrite. I cant find any function in the documentation that does this.

1979-07-20
1979
07
20
Warning: Month >7< not found!
Warning: Day >20< not found!
cdo    delete: Processed 1 variable over 15341 timesteps [61.05s 77MB].
1979-07-26
1979
07
26
Warning: Month >7< not found!
Warning: Day >26< not found!
cdo    delete: Processed 1 variable over 15341 timesteps [61.02s 77MB].

I'll appreciate any help.

Lyndz

Replies (3)

RE: How to delete a list of dates from one netcdf file and append it - Added by Brendan DeTracey almost 4 years ago

I don't see it defined but $one is 10? Also

I am uncertain, but I suspect multiple items in a delete command are not logically ANDed. I think your delete command will delete all data in 1979, all months 7 and all days 26.
You can not, in general with cdo, read and write to the same file. I am surprised cdo allowed this without an I/O stream error.

What is the time step of your data? Is it daily?

RE: How to delete a list of dates from one netcdf file and append it - Added by Lyndon Mark Olaguera almost 4 years ago

Hi Brendan,

Thank you for this.

I forgot to copy it but the $one=$lead (yes it is 10 for this case).

The time step is daily.

RE: How to delete a list of dates from one netcdf file and append it - Added by Brendan DeTracey almost 4 years ago

Being daily means you can use the date key for delete:

date  
STRING Comma-separated list of dates (format YYYY-MM-DDThh:mm:ss).

It can take a comma-separated list of dates. So take your date list, and preprocess it instead into a comma-separated list. Do this before calling cdo. I do not know if the following will work without a sample data file to test it.

break_file='break_first_day.txt'
in_file='vwnd_1979-2020.nc'
comma_delimited_dates=''
delta_days=10
out_file=${infile%.nc}_broken.nc # bash parameter substitution makes output file names easy
echo "$break_file" "$delta_days" "$in_file" "$out_file" 
# read whole file into bash array
date_list=( $(cat "$break_file") )
# loop and process
for date_entry in ${date_list[@]} ; do
    date_break=$(date -d "$date_entry + $delta_days days" +%Y-%m-%dT%H:%M:%S)
    comma_delimited_dates+="$date_break",
done
# strip trailing comma
comma_delimited_dates=${comma_delimited_dates::-1}
echo "$comma_delimited_dates" 
cdo -delete,date="$comma_delimited_dates" "$in_file" "$out_file"

Keep working on your shell scripting!

(1-3/3)

Project

General

Profile

CDO