Using CDO for EOFs

Added by Nicola Maher almost 7 years ago

For basic EOF use:
where n is the number of EOFs you want as output

export CDO_WEIGHT_MODE=off
export MAX_JACOBI_ITER=100

cdo eof,n input.nc eigval.nc pattern.nc
cdo eofcoeff pattern.nc input.nc pc.nc
cdo -div eigval.nc -timsum eigval.nc /explvar.nc

Notes:
1. It is important to note that the magnitude (even the sign) of the EOF patterns is ambiguous.
2. Best to never change CDO_WEIGHT_MODE within a workflow unless you really want to find out something about the weights or the grid.
3. In principle it is always better to use the non-weighted version because it is cleaner and consistent with the output from other programs. If, however, you have a dataset where the size of the cells varies, you will get a more accurate estimate from the weighted version.
4. There is no need to normalize the variance as the EOF partitions the variance into space and modes (EOFs) and time (principal coefficients).
5. Patterns are directly comparable.
6. If the time series have a really different variance, you may not have the same order of the EOFs as the difference in variance may be due to some additional mode which in an extreme case might even dominate. So you might also consider comparing tha patterns of different order.

Replies (3)

RE: Using CDO for EOFs - Added by Christopher Danek almost 4 years ago

Thanks for this. Should the result depend on the choice of the EOF operator (i.e. cdo eof or cdo eoftime) and the choice of --P? Because I get some differences in the explained variance:

eof -P1
   eof1: 21.77277%
   eof2: 9.355396%
   eof3: 5.557447%

eof -P2
   eof1: 21.40523%
   eof2: 9.197991%
   eof3: 5.46247%

eof -P4
   eof1: 20.26945%
   eof2: 8.709419%
   eof3: 5.17318%

eof -P8
   eof1: 19.82159%
   eof2: 8.516847%
   eof3: 5.057224%

eoftime -P1
   eoftime1: 21.77277%
   eoftime2: 9.355396%
   eoftime3: 5.557447%

eoftime -P2
   eoftime1: 21.06187%
   eoftime2: 9.047274%
   eoftime3: 5.374549%

eoftime -P4
   eoftime1: 20.8404%
   eoftime2: 8.951262%
   eoftime3: 5.318119%

eoftime -P8
   eoftime1: 19.30656%
   eoftime2: 8.291214%
   eoftime3: 4.92649%

The more parallel tasks, the lower the explained variance becomes. Does that make sense? In Appendix B of the cdo documentation it is written that the cdo operators are suited for parallelization.

Thanks for answer,
Chris

ps: the verbose output of all these calls was the same:

...
 OpenMP:  num_procs=8  max_threads=8  num_devices=0
# or num_procs=4, 2, ...
cdo eoftime: Using CDO_SVD_MODE 'jacobi' from  default
cdo eoftime: Using CDO_WEIGHT_MODE 'off' from Environment
cdo eoftime: Counting timesteps in ifile
cdo eoftime: Found 701 timeSteps
cdo eoftime: Calculating 3 eigenvectors and 701 eigenvalues in time_space
cdo eoftime: Allocated eigenvalue/eigenvector structures with nts=701 gridsize=4608
cdo eoftime: Calculating covar matrices for 1 levels of var0 (temp2)
cdo eoftime: processing level 0
cdo eoftime: allocating covar with 701 x 701 elements | npack=4608
cdo eoftime: finished calculation of covar-matrix for var temp2
cdo eoftime: Using MAX_JACOBI_ITER 100 from Environment
cdo eoftime: Using FNORM_PRECISION 1e-12 from default
cdo eoftime: Finished one-sided jacobi scheme for eigenvalue computation after 20 iterations
cdo eoftime: Processed 3230208 values from 1 variable over 701 timesteps [15.52s 85MB]
# or cdo eof: ...

RE: Using CDO for EOFs - Added by Uwe Schulzweida almost 4 years ago

This is definitely a bug in the eof operator. I found a data race error in a parallel region. This bug will be fixed in the next CDO release.

Thanks for the report!

Uwe

RE: Using CDO for EOFs - Added by Christopher Danek almost 4 years ago

Thanks Uwe

One more question: can you give the definition of the spatial weights W used by cdo eof (section 2.11 in the cdo docu)?

Thanks

(1-3/3)

Project

General

Profile

CDO