Hourly to Daily averages with daymean
Added by Nicholas Depsky about 5 years ago
Hello,
I've just attempted to convert an ERA5 hourly total precipitation dataset from 1979-1984 to daily averages using daymean as follows:
cdo daymean era5_hourly_79_84.nc era5_daily_79_84.nc
However, the output file era5_daily_79_84.nc is MUCH bigger (4.5 GB) than the input hourly file (750 MB). Why is this? I was expecting it to be ~24 times smaller since it is supposed to be aggregating hourly to daily. Am I missing something?
Thanks,
Nick
Replies (2)
RE: Hourly to Daily averages with daymean - Added by Ralf Mueller about 5 years ago
hi Nicholas!
Could you run the showformat operator on both files? or provide a download link to both files. for netcdf4 there are internal chunksize parameters that can lead to big filesize differences. but without the files I can't say anything specific.
cheers
ralf
RE: Hourly to Daily averages with daymean - Added by Kira Moriguchi about 4 years ago
This is late, but since this comes out front page Google Search. The showformat output for ERA5 hourly SST is NetCDF2. But more importantly from ncdump is:
short sst(time, latitude, longitude) ;
sst:scale_factor = 0.000605748755636473 ;
sst:add_offset = 289.734194195935 ;
The original file is packed. For example, the file downloaded from ECMWF for SST hourlies for January 2019 is 1.43 GB (1,544,921,088 bytes)(Windows 10), the size is After unpacking the files size is 5.75 GB (6,179,618,816 bytes). Since CDO doesn't do repacking, that is why the output file size is bigger. You need to repack the file yourself if you want the size to small. nco tools can do this, but it is RAM taxing. Alternatively you can request CDO developers to include repacking in their package.