how to read gzipped files with cdo?
Added by Ian Scott-Fleming about 11 years ago
Is there a way to get cdo operators to either read from stdin or read a gzip-compressed file?
I was looking for something along the lines of
zcat myfile.nc.gz | cdo showyear -
or
cdo showyear -z myfile.nc.gz
Many of our raw data files are compressed with gzip to save space, and rather than unzip them to an actual file, I would like to be able to pipe the output of zcat to cdo.
Many cmdline unix programs can read from stdin if the filename is given as "-", but this doesn't seem to work.
CDO documentation shows how to pipe output from one cdo operator to another, but I can't figure out how to get input into cdo from an external pipe.
Anybody got a good solution for this?
thanks.
Replies (2)
RE: how to read gzipped files with cdo? - Added by Ralf Mueller about 11 years ago
CDO does not support UNIX pipes. Especially netcdf files have a definition and a data area. Applications have to jump from one to the other, which cannot be done with a unix pipe (implemented as FIFO).
CDO can work with Netcdf4's builtin data compression (done with zlib/szip). I guess, you'll have to decompress you input files. A ramdisk might be useful for speedup.
hth
ralf
RE: how to read gzipped files with cdo? - Added by Ian Scott-Fleming about 11 years ago
bummer. ok. thanks for the explanation.
i can live with that.
zcat to temp file, delete temp file when done.
slower, but doesn't the cause the world fall apart...