Using the MiKlip Evaluation System

... in the web

To log in to the miklip system you may use a browser:

MiKlip Central Evaluation System

use your DKRZ login! Here you find some frequently asked questions Q_and_A.

... in the shell

To log in to the miklip system you may use ssh from any linux/unix system:

ssh -X

The -X will allow you to connect to the remote X server (e.g. to display images). b123456 should be your DKRZ account, if you still don't have one please read: [[miklip:Access to MiKlip Server]].

Start setting up the environment by loading the proper module (You may copy the following line as is into your shell):

module load miklip-ces

This activates the system for your current session. You might notice some other modules have been loaded.
WARNING: The evaluation_system works in bash, to avoid incompabilities choose bash as the default shell:

Developers note: The module just loads other required modules and extends PATH to point to to evaluation_system/bin and PYTHONPATH to evaluation_system/src

Overview of implemented Plugins - always shell & web

The Plugin Section
including plugin documentation, if available...

Working with Freva

Freva is the all in one framework with the main features: freva --help
Available commands:
  --plugin       : Applies some analysis to the given data.
  --history      : provides access to the configuration history (use --help for more help)
  --databrowser  : Find data in the system
  --crawl_my_data: Use this command to update your projectdata.
  --esgf         : Browse ESGF data and create wget script

This is the main tool for the evaluation system.
Usage: freva --COMMAND [OPTIONS]
To get help for the individual commands use
  freva --COMMAND --help
freva Option Description Example
--plugin apply some analysis tool freva --plugin pca outputdir=/tmp variable=tas
--history browser your history freva --history or freva --history --plugin movieplotter
--databrowser Search CMIP5, MIKLIP or your data with the fast auto-completion browser. If you are in bash, try hitting tab twice whenever you want to input a new attribute or value for the search, you'll see all possible values listed. freva --databrowser project=baseline1 variable=tas time_frequency=mon ensemble=r[1-5]i1p1 experiment=*196[0-5]
--crawl_my_data when you have your own little database, how2put your data into database freva --crawl_my_data --path=/miklip/integration/data4miklip/projectdata/bxxxxx
--esgf Contact the esgf to query for datasets files and retrieve the wget script used for download them freva --esgf --download-script /tmp/download.wget variable=tas time_frequency=mon ensemble=r1i1p1 experiment=decadal1965

All commands have a -h or --help flag to display the commands help.

Basic commands for freva:
To get the main help freva --help
To list the tools freva --plugin
To get the tool help freva --plugin sometool --help
To see the history freva --history
To see the history help freva --history --help

The first news with respect to usability:

Using&Devoloping (with the old commands!)

Use cases

Compute PCA for every single ensemble separately

Iterate over all ensemble members of the experiment started in 1960 and compute the PCA of them for the variable tas.

In bash this would be:

for file in $(freva --databrowser project=baseline1 variable=tas time_frequency=mon experiment=*1960); do
  freva --plugin pca input=$file outputdir=/tmp variable=tas pcafile=pca_$(basename $file)