CDO

Hi Zachary!

For some operations you just need the proper amount of memory. CDO tries to use as minimal as possible, but if the grids are too large, interpolation can exceed the physical memory of your machine. the randomness worries me a bit, but there might be competing processes running.

There is one thing you could do (in case your input or output grid is too large): splitting into regions

Hard to say more without knowledge about input/output you use.

best
ra;lf

Hey ra;lf,

Thanks for your response. I had a feeling that the grids were possibly just too large. Since I am running cdo in a kubernetes environment I am able to play around with the memory a bit but even at 9Gb of memory I still get failure. There is nothing else running on the pod except a small api that should never use that much memory (and will only run 1 cdo action at a time). I think your recommendation of splitting the input file might do the trick but I do have 1 question about it. Since I'm running an interpolation over the data couldn't splitting the grid cause some distortion along the edges of the split?

Thanks,
Zack

It depends if you the problem is caused by the input or output grid. Splitting only the output grid should not harm, but with the input you can get into trouble (missing overlapping cells in the splited input e.g.)

Okay I'll give it a shot with the output grids. Thanks for your help!

another idea: if your grids are constant, you can pre-compute the interpolation weights on another machine and just use them in the virtual environment.