Using the MiKlip Evaluation System¶

... in the web¶

To log in to the miklip system you may use a browser:

MiKlip Central Evaluation System

use your DKRZ login! Here you find some frequently asked questions Q_and_A.

... in the shell¶

To log in to the miklip system you may use ssh from any linux/unix system:

ssh -X b123456@miklip.dkrz.de

The -X will allow you to connect to the remote X server (e.g. to display images). b123456 should be your DKRZ account, if you still don't have one please read: [[miklip:Access to MiKlip Server]].

Start setting up the environment by loading the proper module (You may copy the following line as is into your shell):

module load miklip-ces

This activates the system for your current session. You might notice some other modules have been loaded.
WARNING: The evaluation_system works in bash, to avoid incompabilities choose bash as the default shell: luv.dkrz.de

Developers note: The module just loads other required modules and extends PATH to point to to evaluation_system/bin and PYTHONPATH to evaluation_system/src

Overview of implemented Plugins - always shell & web¶

The Plugin Section
including plugin documentation, if available...

Working with Freva¶

Freva is the all in one framework with the main features: freva --help

Freva
Available commands:
  --plugin       : Applies some analysis to the given data.
  --history      : provides access to the configuration history (use --help for more help)
  --databrowser  : Find data in the system
  --crawl_my_data: Use this command to update your projectdata.
  --esgf         : Browse ESGF data and create wget script

This is the main tool for the evaluation system.
Usage: freva --COMMAND [OPTIONS]
To get help for the individual commands use
  freva --COMMAND --help

freva Option	Description	Example
--plugin	apply some analysis tool	freva --plugin pca input=myfile.nc outputdir=/tmp variable=tas
--history	browser your history	freva --history or freva --history --plugin movieplotter
--databrowser	Search CMIP5, MIKLIP or your data with the fast auto-completion browser. If you are in bash, try hitting tab twice whenever you want to input a new attribute or value for the search, you'll see all possible values listed.	freva --databrowser project=baseline1 variable=tas time_frequency=mon ensemble=r[1-5]i1p1 experiment=*196[0-5]
--crawl_my_data	when you have your own little database, how2put your data into database	freva --crawl_my_data --path=/miklip/integration/data4miklip/projectdata/bxxxxx
--esgf	Contact the esgf to query for datasets files and retrieve the wget script used for download them	freva --esgf --download-script /tmp/download.wget variable=tas time_frequency=mon ensemble=r1i1p1 experiment=decadal1965

All commands have a -h or --help flag to display the commands help.

Basic commands for freva:

To get the main help	freva --help
To list the tools	freva --plugin
To get the tool help	freva --plugin sometool --help
To see the history	freva --history
To see the history help	freva --history --help

The first news with respect to usability:¶

Using&Devoloping (with the old commands!)

Use cases¶

Compute PCA for every single ensemble separately¶

Iterate over all ensemble members of the experiment started in 1960 and compute the PCA of them for the variable tas.

In bash this would be:

for file in $(freva --databrowser project=baseline1 variable=tas time_frequency=mon experiment=*1960); do
  freva --plugin pca input=$file outputdir=/tmp variable=tas pcafile=pca_$(basename $file)
done