file size of outfile combining infiles with mergetime or cat operator
Added by Alain Dezetter almost 5 years ago
Hi,
I'm an absolut beginner with cdo and I'm trying to use it with some basic operators first.
I have combined several files of pr variable from africa cordex experiment :
pr_AFR-44_MOHC-HadGEM2-ES_historical_r1i1p1_SMHI-RCA4_v1_day_19510101-19551230.nc
pr_AFR-44_MOHC-HadGEM2-ES_historical_r1i1p1_SMHI-RCA4_v1_day_19560101-19601230.nc
pr_AFR-44_MOHC-HadGEM2-ES_historical_r1i1p1_SMHI-RCA4_v1_day_19610101-19651230.nc
pr_AFR-44_MOHC-HadGEM2-ES_historical_r1i1p1_SMHI-RCA4_v1_day_19660101-19701230.nc
pr_AFR-44_MOHC-HadGEM2-ES_historical_r1i1p1_SMHI-RCA4_v1_day_19710101-19751230.nc
pr_AFR-44_MOHC-HadGEM2-ES_historical_r1i1p1_SMHI-RCA4_v1_day_19760101-19801230.nc
pr_AFR-44_MOHC-HadGEM2-ES_historical_r1i1p1_SMHI-RCA4_v1_day_19810101-19851230.nc
pr_AFR-44_MOHC-HadGEM2-ES_historical_r1i1p1_SMHI-RCA4_v1_day_19860101-19901230.nc
pr_AFR-44_MOHC-HadGEM2-ES_historical_r1i1p1_SMHI-RCA4_v1_day_19910101-19951230.nc
pr_AFR-44_MOHC-HadGEM2-ES_historical_r1i1p1_SMHI-RCA4_v1_day_19960101-20001230.nc
pr_AFR-44_MOHC-HadGEM2-ES_historical_r1i1p1_SMHI-RCA4_v1_day_20010101-20051230.nc
each file is approximately 40 Mo of size, but when I combine theses file in one file for the entire time period with cat or mergetime operator, I obtain a file that is 3Go large !! ??
Why ? Have I missed something ?
Thanks in advance for any response
Alain
Replies (5)
RE: file size of outfile combining infiles with mergetime or cat operator - Added by Karin Meier-Fleischer almost 5 years ago
Hi Alain,
CORDEX data files are usually netcdf4 files with compression which is not automatically used by CDO.
You can retrieve the type of the file using the sinfo operator with one of the files:
cdo sinfo infiles.nc
To use the netcdf4 compression and do the mergetime
cdo -O -z zip_1 -mergetime pr_AFR-44_MOHC-HadGEM2-ES_historical_r1i1p1_SMHI-RCA4_v1_day_*.nc outfile.nc
-Karin
RE: file size of outfile combining infiles with mergetime or cat operator - Added by Alain Dezetter almost 5 years ago
Many thank's for quick answer to a new user !
Best regards
Alain
RE: file size of outfile combining infiles with mergetime or cat operator - Added by Ralf Mueller almost 5 years ago
the compression can use different chunksizes, so the access to one if your input files would be helpful. can you upload one of them?
ralf
RE: file size of outfile combining infiles with mergetime or cat operator - Added by Alain Dezetter almost 5 years ago
Thank's for help Ralf
the command line given by Karin gave me the expected result:
the file size is now 452Mo and not 3Go
Best regards
Alain
(if you always want a file to explore it, I can upload one of then (40 Mo))
RE: file size of outfile combining infiles with mergetime or cat operator - Added by Ralf Mueller almost 5 years ago
Alain Dezetter wrote:
(if you always want a file to explore it, I can upload one of then (40 Mo))
naw - if it's solved it's solved ;-)
cheers
ralf