Project

General

Profile

Make Experiments!

New features

Setting list values from the command line

Since r102, when setting variables on the mkexp command line, the value is split at each comma, thus forming a value list. Thus

mkexp config_file VAR=1,2,3

is now equivalent to setting

VAR = 1, 2, 3 # used to be "1, 2, 3"

in config_file. In former versions, the value was always treated as a single string.

Changing section variables (namelists, jobs) on the command line

With r102, the command line handling of mkexp has been extended to also allow setting of variables in an arbitrary section of the configuration. The syntax is

mkexp config_file section.[subsection. ...].variable=value

i.e. section and variable names are separated by periods. An actual period in the names must be replaced by a double period (..), e.g. for changing a namelist file 'namelist.echam'

mkexp config_file namelists.namelist..echam.runctl.nproma=48

In a config file, this reads

[namelists]
[[namelist.echam]]
[[[runctl]]]
nproma = 48

Support for prepackaged option sets

Since r103, mkexp supports a new special configuration variable, EXP_OPTIONS. It takes a list of config names that are provided in the new 'standard_options' subdirectory. Settings from these config files are added to the experiment type's standard settings, allowing to build named 'option sets' that may be switched on and off by simply adding names to or removing names from the EXP_OPTIONS list. For e.g. switching a model to write monthly mean output directly, one may gather all namelist and variable settings needed into 'standard_options/monmean.config'. When defining a new experiment, simply set

EXP_OPTIONS = monmean

to load all these settings with a single change.

Using native variables in job scripts

With r105 and r106, mkexp has full support for using job script variables in config values. If a value contains $${identifier}, identifier is recognized as a job script variable. Thus, for a config and template as in

# config file
FILENAME = $${HOME}/.profile
#! /bin/sh
#%# script template
PROFILE=%{FILENAME}

the resulting script will look like

#! /bin/sh
PROFILE=${HOME}/.profile

Native variables for non-shell scripts

To allow adjusting the syntax for different target languages, the job section allows to define a special string format template, .var_format, where each occurrence of %s is replaced by identifier. For use in a Python string, this format would be {%s}:

FILENAME = $${HOME}/.profile
[jobs]
  [[python_job]]
    .var_format = {%s}
#! /usr/bin/env python
import os
profile = "%{FILENAME}".format(**os.environ)

resulting in

#! /usr/bin/env python
import os
profile = "{HOME}/.profile".format(**os.environ)

Note that this allows using the same config variable in both shell and non-shell scripts.

Initializing native variables

The previous examples rely on environment variables that are initialized by the system. To use custom variables and initialize them within the config file, the template must provide a definition for each native variable, setting it to the config variable's value:

DIRNAME = /some/where
FILENAME = over_the_rainbow
FILEPATH = $${DIRNAME}/$${FILENAME}
#! /bin/sh

DIRNAME=%{DIRNAME}
FILENAME=%{FILENAME}

INFO=%{FILEPATH}

When running mkexp, the result will be

#! /bin/sh

DIRNAME=/some/where
FILENAME=over_the_rainbow

INFO=${DIRNAME}/${FILENAME}

so all variables used are defined using the corresponding config variables.

Advanced initialization of native variables

In the previous example the list of variables in the script template must be kept consistent with the variables used in the config file. As this is easy to forget, mkexp provides a special variable VARIABLES_ containing the names of all native variables, and a template function context returning the corresponding value for a given variable name. This allows automatic updating of the variable list in the script, based on the config file, e.g. using the config file above with a template like this

#! /bin/sh

#% for var in VARIABLES_:
%{var}=%{context(var)}
#% endfor

INFO=%{FILEPATH}

As VARIABLES_ will be filled by mkexp to contain the list (DIRNAME, FILENAME), this will result in the same script as above, without the need to re-iterate the variable list within the script template.