Project

General

Profile

Forums » Open discussion »

Does CDO have to read input from on-disk file, and write output into disk?

Added by QINQIN KONG almost 4 years ago

According to the user guide, it seems that CDO only support reading input for on-disk file and also has to write output into disk. Is that so? Can CDO read input from in-memory xarray for example, and produce output as in-memory array as well?

Thanks!


Replies (4)

RE: Does CDO have to read input from on-disk file, and write output into disk? - Added by Ralf Mueller almost 4 years ago

hi!

The CDO binary supports input/output for files on disk in netCDF or GRIB format. In you want to interact with an xarray data structure, you have to write it to disk with to_netcdf and use the filename. Please note, that using a tempfile has low IO cost because the system-wide tmp directory is usually mapped into RAM.

cheers
ralf

RE: Does CDO have to read input from on-disk file, and write output into disk? - Added by QINQIN KONG almost 4 years ago

Hi Ralf. Thanks for the quick answer! Could you please elaborate more on your last sentence: "Please note, that using a tempfile has low IO cost because the system-wide tmp directory is usually mapped into RAM." For example, what kind of tempfile has low or high IO cost, and why system-wide tmp directory being mapped into RAM is a consideration?

Thanks!

-Qin

RE: Does CDO have to read input from on-disk file, and write output into disk? - Added by Ralf Mueller almost 4 years ago

Unix-like operating systems have a special folder (usually /tmp) for temporary files. This is mapped into the main memory - hence it does not suffer from slow IO like a HDD. Same is true for /dev/shm.

So in case you want to work with xarray and CDO I recommend the python bindings of CDO. It does this xarray->netcdf file conversion in /tmp under the hood.

cheers
ralf

    (1-4/4)