Project

General

Profile

Forums » Support »

Usage of the tee operator.

Added by Diego Jimenez almost 5 years ago

I used the tee operator to store in a file an intermediate result but fails to do that, and I get an error. Then my question can be technical or about how to use the operator.

The calculation is storing a time average and use it to get anomalies. I have a file X.nc with only one variable. Xmean.nc is the time average and Xanom.nc is the anomaly. Thus,

cdo -O -t echam6 -f nc4 sub X.nc -tee -timmean X.nc Xmean.nc Xanom.nc

Is this correct? Or how should I use tee operator in this case? Is there a limitation on the netCDF files? Or is the restriction in the binary that I am using (obtained via conda-forge)?

The error message reads:

cdo(2) tee: Process started
cdo(3) timmean: Process started
Error in: process_int.cc:234 outstream -1 of 1 not found. Was called with streamIDX = 1
HDF5-DIAG: Error detected in HDF5 (1.10.4) thread 139665869522688:
#000: H5D.c line 372 in H5Dget_space(): not a dataset
major: Invalid arguments to routine
minor: Inappropriate type
cdf_get_var1_double: ncid = 131072 varid = 0

Error (cdf_get_var1_double): NetCDF: HDF error
HDF5-DIAG: Error detected in HDF5 (1.10.4) thread 139665880012544:
#000: H5T.c line 1754 in H5Tclose(): not a datatype
major: Invalid arguments to routine
minor: Inappropriate type

Error (cdf_close): NetCDF: HDF error
  • glibc detected * cdo: double free or corruption (!prev): 0x00007f06858803a0 *

Replies (3)

RE: Usage of the tee operator. - Added by Diego Jimenez almost 5 years ago

I gave it another try. This time the X file is not netCDF but GRIB. Then the command is

cdo -O -t echam6 sub X.grb -tee -timmean X.grb Xmean.grb Xanom.grb

I have a segmentation fault, with the following messages that seem very ominous. Then in the meantime, I will stop using tee.

cdo(2) tee: Process started
cdo(3) timmean: Process started
Error in: process_int.cc:234 outstream -1 of 1 not found. Was called with streamIDX = 1
ERROR, namespaceSetActive, namespace.c, line 228
errorString: "assertion `(unsigned)nId < namespacesSize && namespaces[nId].resStage != NAMESPACE_STATUS_UNUSED` failed"
  • glibc detected * cdo: corrupted double-linked list: 0x00007f241154f1a0 * ======= Backtrace: =========
    /lib64/libc.so.6(+0x75dee)[0x7f240d7dcdee]
    /lib64/libc.so.6(+0x78ff6)[0x7f240d7dfff6]
    /lib64/libc.so.6(exit+0x115)[0x7f240d79c9c5]
    cdo(+0x26bf08)[0x7f240f1c4f08]
    cdo(+0x26be88)[0x7f240f1c4e88]
    cdo(+0x27c4fd)[0x7f240f1d54fd]
    cdo(+0x27ce8f)[0x7f240f1d5e8f]
    cdo(+0x27d088)[0x7f240f1d6088]
    cdo(+0x27d6c9)[0x7f240f1d66c9]
    cdo(+0x2aaa11)[0x7f240f203a11]
    cdo(+0x204e4a)[0x7f240f15de4a]
    cdo(+0x20acfb)[0x7f240f163cfb]
    cdo(+0x1f6369)[0x7f240f14f369]
    cdo(+0x14c83e)[0x7f240f0a583e]
    /lib64/libpthread.so.0(+0x7aa1)[0x7f240db02aa1]
    /lib64/libc.so.6(clone+0x6d)[0x7f240d84fbdd] ======= Memory map: ========
    7f23fc000000-7f23fd07d000 rw-p 00000000 00:00 0
    7f23fd07d000-7f2400000000 ---p 00000000 00:00 0
    7f2404000000-7f2404021000 rw-p 00000000 00:00 0
    7f2404021000-7f2408000000 ---p 00000000 00:00 0
    7f24099db000-7f24099dc000 ---p 00000000 00:00 0
    7f24099dc000-7f240a3dc000 rw-p 00000000 00:00 0
    7f240a3dc000-7f240a3dd000 ---p 00000000 00:00 0
    7f240a3dd000-7f240addd000 rw-p 00000000 00:00 0
    7f240ae0e000-7f240ae15000 rw-p 00000000 00:00 0
    7f240ae15000-7f240ae1a000 r--p 00000000 3a1:4f52a 2251806588106976861 /mnt/lustre01/work/mh0066/m300556/miniconda3/envs/glamdring/lib/libjpeg.so.9.3.0
    7f240ae1a000-7f240ae47000 r-xp 00005000 3a1:4f52a 2251806588106976861 /mnt/lustre01/work/mh0066/m300556/miniconda3/envs/glamdring/lib/libjpeg.so.9.3.0
    7f240ae47000-7f240ae50000 r--p 00032000 3a1:4f52a 2251806588106976861 /mnt/lustre01/work/mh0066/m300556/miniconda3/envs/glamdring/lib/libjpeg.so.9.3.0
    7f240ae50000-7f240ae51000 ---p 0003b000 3a1:4f52a 2251806588106976861 /mnt/lustre01/work/mh0066/m300556/miniconda3/envs/glamdring/lib/libjpeg.so.9.3.0
    7f240ae51000-7f240ae52000 r--p 0003b000 3a1:4f52a 2251806588106976861 /mnt/lustre01/work/mh0066/m300556/miniconda3/envs/glamdring/lib/libjpeg.so.9.3.0
    7f240ae52000-7f240ae53000 rw-p 0003c000 3a1:4f52a 2251806588106976861 /mnt/lustre01/work/mh0066/m300556/miniconda3/envs/glamdring/lib/libjpeg.so.9.3.0
    7f240ae53000-7f240ae54000 rw-p 00000000 00:00 0
    7f240ae54000-7f240ae6a000 r-xp 00000000 fd:00 4718703 /lib64/libresolv-2.12.so
    7f240ae6a000-7f240b06a000 ---p 00016000 fd:00 4718703 /lib64/libresolv-2.12.so
    7f240b06a000-7f240b06b000 r--p 00016000 fd:00 4718703 /lib64/libresolv-2.12.so
    7f240b06b000-7f240b06c000 rw-p 00017000 fd:00 4718703 /lib64/libresolv-2.12.so
    7f240b06c000-7f240b06e000 rw-p 00000000 00:00 0
    7f240b06e000-7f240b072000 r--p 00000000 3a1:4f52a 2251806609799986650 /mnt/lustre01/work/mh0066/m300556/miniconda3/envs/glamdring/lib/libkrb5support.so.0.1
    7f240b072000-7f240b078000 r-xp 00004000 3a1:4f52a 2251806609799986650 /mnt/lustre01/work/mh0066/m300556/miniconda3/envs/glamdring/lib/libkrb5support.so.0.1
    7f240b078000-7f240b07a000 r--p 0000a000 3a1:4f52a 2251806609799986650 /mnt/lustre01/work/mh0066/m300556/miniconda3/envs/glamdring/lib/libkrb5support.so.0.1
    7f240b07a000-7f240b07b000 ---p 0000c000 3a1:4f52a 2251806609799986650 /mnt/lustre01/work/mh0066/m300556/miniconda3/envs/glamdring/lib/libkrb5support.so.0.1
    7f240b07b000-7f240b07c000 r--p 0000c000 3a1:4f52a 2251806609799986650 /mnt/lustre01/work/mh0066/m300556/miniconda3/envs/glamdring/lib/libkrb5support.so.0.1
    7f240b07c000-7f240b07d000 rw-p 0000d000 3a1:4f52a 2251806609799986650 /mnt/lustre01/work/mh0066/m300556/miniconda3/envs/glamdring/lib/libkrb5support.so.0.1
    7f240b07d000-7f240b07e000 rw-p 00000000 00:00 0
    7f240b07e000-7f240c97f000 r--p 00000000 3a1:4f52a 2251806587637265042 /mnt/lustre01/work/mh0066/m300556/miniconda3/envs/glamdring/lib/libicudata.so.58.2
    7f240c97f000-7f240c980000 r--p 01900000 3a1:4f52a 2251806587637265042 /mnt/lustre01/work/mh0066/m300556/miniconda3/envs/glamdring/lib/libicudata.so.58.2
    7f240c980000-7f240c987000 r--p 00000000 3a1:4f52a 2251806587637244487 /mnt/lustre01/work/mh0066/m300556/miniconda3/envs/glamdring/lib/libhdf5_hl.so.100.1.1
    7f240c987000-7f240c99c000 r-xp 00007000 3a1:4f52a 2251806587637244487 /mnt/lustre01/work/mh0066/m300556/miniconda3/envs/glamdring/lib/libhdf5_hl.so.100.1.1
    7f240c99c000-7f240c9a2000 r--p 0001c000 3a1:4f52a 2251806587637244487 /mnt/lustre01/work/mh0066/m300556/miniconda3/envs/glamdring/lib/libhdf5_hl.so.100.1.1
    7f240c9a2000-7f240c9a3000 ---p 00022000 3a1:4f52a 2251806587637244487 /mnt/lustre01/work/mh0066/m300556/miniconda3/envs/glamdring/lib/libhdf5_hl.so.100.1.1
    7f240c9a3000-7f240c9a4000 r--p 00022000 3a1:4f52a 2251806587637244487 /mnt/lustre01/work/mh0066/m300556/miniconda3/envs/glamdring/lib/libhdf5_hl.so.100.1.1
    7f240c9a4000-7f240c9a5000 rw-p 00023000 3a1:4f52a 2251806587637244487 /mnt/lustre01/work/mh0066/m300556/miniconda3/envs/glamdring/lib/libhdf5_hl.so.100.1.1
    7f240c9a5000-7f240c9a6000 rw-p 00000000 00:00 0
    7f240c9a6000-7f240c9b8000 r--p 00000000 3a1:4f52a 2251806587637292872 /mnt/lustre01/work/mh0066/m300556/miniconda3/envs/glamdring/lib/libdf.so.0.0.0
    7f240c9b8000-7f240ca16000 r-xp 00012000 3a1:4f52a 2251806587637292872 /mnt/lustre01/work/mh0066/m300556/miniconda3/envs/glamdring/lib/libdf.so.0.0.0
    7f240ca16000-7f240ca28000 r--p 00070000 3a1:4f52a 2251806587637292872 /mnt/lustre01/work/mh0066/m300556/miniconda3/envs/glamdring/lib/libdf.so.0.0.0Abortado

RE: Usage of the tee operator. - Added by Uwe Schulzweida almost 5 years ago

The syntax of your CDO command is correct. Unfortunately the function tee seems not to work correctly since CDO relase 1.9.4.
We have done some modifications in CDO but unfortunately missed to check this function.

RE: Usage of the tee operator. - Added by Diego Jimenez almost 5 years ago

Thanks, Uwe, for the information. I look forward to see a working tee in version to follow :) This is a very handy operator in the concept.

    (1-3/3)