Project

General

Profile

Forums » Open discussion »

Calculating daily mean fields from multiple input files

Added by Fernando San Deigo about 1 month ago

Hi all. I used to use the command below to calculate daily mean fields from multiple hourly input files
cdo -daymean -selvar,TG -cat input*.nc output.nc where input*.nc covers 1 month worth of hourly data (720 files).

ncdumd would return this

ncdump -h output.nc
netcdf tg {
dimensions:
        Times = UNLIMITED ; // (97 currently)
        west_east = 349 ;
        south_north = 349 ;
variables:

I tried same thing today and got an output with only 1 timestep

netcdf tg {
dimensions:
        Times = UNLIMITED ; // (1 currently)
        west_east = 349 ;
        south_north = 349 ;
variables:

Has anything changed with cdo -cat?
Thanks


Replies (7)

RE: Calculating daily mean fields from multiple input files - Added by Estanislao Gavilan about 1 month ago

Hi Fernando,

Did you try?

cdo -daymean -selvar,TG -cat 'input*.nc' output.nc

RE: Calculating daily mean fields from multiple input files - Added by Fernando San Deigo 26 days ago

Thanks for your reply. Yes I have tried that I am still getting a one-timestep file even though I have multiple days as input.
I checked the time stamp on the input files (see below) and noticed that all of them have 0000-00-00T00:00:00. Could that be causing the problem?

cdo -showtimestamp 2018123115.LDASOUT_DOMAIN1
  0000-00-00T00:00:00
cdo    showtimestamp: Processed 53 variables over 1 timestep [0.04s 50MB].

RE: Calculating daily mean fields from multiple input files - Added by Estanislao Gavilan 26 days ago

Hi Fernando,

it seems something is happening with your timeaxis. Can you upload a time step? Also, what cdo version are you using?

RE: Calculating daily mean fields from multiple input files - Added by Fernando San Deigo 26 days ago

Hi there. I have attached one of my file. I am using CDO version 1.9.9rc1 . My IT dept has not been able to update it to a newer version due to installation problems. In the meantime, I will work on a bash script to fix the files time stamps. Thanks for your help

2020090500.LDASOUT_DOMAIN1 (49.3 MB) 2020090500.LDASOUT_DOMAIN1

RE: Calculating daily mean fields from multiple input files - Added by Estanislao Gavilan 25 days ago

Hi Fernando,

I looked into the file. cdo was able to output the date, but that file format does not follow the cf convection. Thats probably why you are having problems with the cat command. For instance you time dimensions are two instead of one. Also, the attributes are not correct.

Best regards,

Estanislao

RE: Calculating daily mean fields from multiple input files - Added by Karin Meier-Fleischer 25 days ago

The variable Time is of type character which is not allowed in CF. But you can write a shell script to add a correct time variable to the files.

This example creates 12 hourly data from the given example file to check if it will work. For your multiple input files you have to use a loop.

#!/usr/bin/env bash

infile=2020090500.LDASOUT_DOMAIN1

# get the Times variable string
t_string=$(ncdump -v Times $infile | \
              sed -e '1,/data:/d' -e '$d' -e 's/[ ";]//g' | \
              tail -n +3)
# split the Times variable string into date string and time string
date_string=$(echo $t_string | cut -d '_' -f 1)
time_string=$(echo $t_string | cut -d '_' -f 2)

# add new time variable (12 hourly data)
cdo -r -settaxis,${date_string},${time_string},1day $infile o0.nc

# generate 12 hourly data file
cdo -r -shifttime,12hour -settaxis,${date_string},${time_string},1day $infile o1.nc
cdo -r -shifttime,1day -settaxis,${date_string},${time_string},1day $infile o2.nc
cdo -r -shifttime,36hour -settaxis,${date_string},${time_string},1day $infile o3.nc
cdo -r -shifttime,2day -settaxis,${date_string},${time_string},1day $infile o4.nc

# 
cdo -daymean -selvar,TG -cat 'o*.nc' outfile.nc
    (1-7/7)