CDO

Hi Chuang,

this is a shell problem limiting the number of files to be listed.

You have to split your files into groups and merge it with cdo group by group.

Assume your input files are daily data files named like 20150101.nc, 20150102.nc, 20150103.nc, ...

cdo mergetime 201501*.nc out_2015_01.nc
cdo mergetime 201502*.nc out_2015_02.nc
cdo mergetime 201503*.nc out_2015_03.nc
...

Then merge the output files

cdo mergetime out_*.nc out_complete.nc

-Karin

Hi

Can you tell us what the maximum number of arguments is?

Thx,
Chris

this limit is usually set by maximal command line length of your shell. AFAIK cdo does not impose an extra limit.

hth
ralf

Hi

Thanks for the answer. I checked a long cdo command and found that this criterion means the number of characters of the argument, and not the number of arguments provided to cdo.

In my example, I start counting the number of characters of the cdo argument after the `cdo`, i.e., if the command is

$ cdo a b
     1234
# or
$ /path/to/bin/cdo a b
                  1234

I count 4 as he number of characters of the cdo argument.

Counting like this, the number of characters of the argument of the longest successful `cdo -select,name=var <long list> out.nc` command is 2612710. As soon as if I add one letter to the output file name, e.g. `out2.nc`, the error "Argument list too long" arises.

Note that multiple spaces in the cdo command seem to be counted as single spaces. I.e. `cdo -select,name=var <space><space><space> <long list> out.nc` also works despite the increased number of spaces.

Also, this result is independent of the actual number of files provided by <long list>. If I increased the number of input files by 1, and reduced the number of characters of the output name `out.nc` by the same number of characters of the addtional filename, the command worked successfully.

Surprisingly, this result, 2612710, seems to be independent of

MAX_ARG_STRLEN = $(getconf PAGE_SIZE)*32 = 4096*32 = 131072
# and
getconf ARG_MAX = 2097152

which were the solutions I found in the net to this issue (the command has 29362 spaces, i.e. 2612710 - 29362 = 2583348). So probably my "counting" is wrong?

Kind regards,
Chris

hi Christopher!

I think your problem has nothing to do with CDO itself. but if you upload the script that you use for getting the error, I could at least try to reproduce it.