Installation

QA-DKRZ may be installed either via the conda package manager or from a GitHub repository.

Recommended is the conda package manager installing a ready-to-use package. At present, a 64-bit processor is required.

If you want to work with sources or use a different machine architecture, then the installation from source should be the first choice.

For CMIP6, QA-DKRZ binds the CMOR validator PrePARE.py (D. Nadeau, LLNL, https://cmor.llnl.gov). This is conducted only by conda, e.g.

$ conda create -n cmor -c conda-forge -c pcmdi -c uvcdat cmor

Details in section Work-flow below.

No matter what the preference is conda or from GitHub, you are asked for a location of tables and programs during the installation.

By Conda Package Manager

Make sure that you have conda installed. The quick steps to install miniconda on Linux 64-bit are:

$ wget https://repo.continuum.io/miniconda/Miniconda-latest-Linux-x86_64.sh
$ bash Miniconda-latest-Linux-x86_64.sh

Note

The installation is tested on various 64-bit machines.

Please check that the conda package manager is in your PATH. For example you may set the PATH environment variable as following:

$ export PATH=$PATH:your-path/your-miniconda/bin
$ conda -h

conda provides the QA-DKRZ package with all dependencies included. You may install it to the root environment or to any other with a name of your choice, e.g.

$ conda create -n qa-dkrz -c conda-forge -c h-dh qa-dkrz

From GitHUB repository

Requirements

The tool requires the BASH shell and a C/C++ compiler (works for AIX, too).

Building the QA tool

The sources are downloaded from GitHub by

$ git clone  https://github.com/IS-ENES-Data/QA-DKRZ

Any installation is done with the script install ( a prefix ‘./’ could be helpful in some cases) within the QA-DKRZ root directory. Please, have also a look at the Work-flow section.

  • A file install_configure in QA-DKRZ binds necessary lib and include directories. A template .install_configure is provided. If not copied to install_configure and edited before the first launch, then it is created and the user is asked to edit the file.
  • Environmental variables CC, CXX, CFLAGS, and CXXFLAGS are accepted.
  • install establishes access to libraries, which may be linked (recommended) or built (triggered by option --build).
  • Proceedings are logged in file install.log.
  • The executables are compiled for projects named on the command-line.
  • When install is started the first time for a given project, then corresponding external tables and programs are downloaded to a QA_HOME directory; the user is asked for the path.
  • The user’s home directory contains a config.txt file in directory .qa-dkrz created and updated by user defined control statements.

The full set of options is described by:

$ ./install --help

Building Libraries

If you decide to use your own set of libraries (accessing provided ones is preferred by respective settings in the install_configure file), then this is accomplished by

$ ./install --build [opts]

Sources of the following libraries are downloaded and installed:

The libraries are built in sub-directory local/source. If libraries had been built previously, then the sources are updated and the libraries are rebuilt.

Work-flow

Please note that the work-flow has changed. By now, operational mode is generally distinct from installation or updating.

Operation is conducted (with some reasonable options) by e.g.

$ qa-dkrz -f task-file

with task-file containing frequently or task-specific options like PROJECT_DATA, QA_RESULTS, SELECT, EMAIL_SUMMARY, NUM_EXEC_THREADS, CHECK_MODE and PROJECT or QA_CONF. The full set of options used by default is given in file QA_HOME/tables/projects/PROJECT/PROJECT_qa.conf with QA_HOME chosen during the installation. Additionally, every valid option may be provided on the command-line by prefixing -e.

Note that option RUN_CMOR3_LLNL must be given for running PrePARE.py.

Installation and update tasks are indicated by option --install=..., which is additionally provided to the qa-dkrz call. Then, all options of install have to be given by a comma-separated list; the usual -- prefix of options may be omitted, e.g.

$ qa-dkrz --install=help

would display all install options. The --install option may be merged with operational options conducted afterwards.

The separation of operation and install/updating is broken by option --auto-up for backward-compatibility. Additionally, when working with the GitHub based package, the --install option may be replaced by the QA-DKRZ/install command (then options with -- prefix). Note that conda does not know the path to install.

Operation/installation/update with conda requires activating and deactivating the corresponding environment. Provided that conda is within the path or aliased:

$ source activate your-chosen-environment-name
$ qa-dkrz options
$ source deactivate.

Command which qa-dkrz while the environment is activated reveals the path to qa-dkrz. Given to PATH or as alias would make activate/deactivate redundant.

There are three stages for updating:

install [PROJECT] without any additional option.
Compilation of executables for PROJECT. Only for GitHub based installation; no effect for a conda installed qa-dkrz.
install –up
Updating once every other number of days, while num has been given by option --uf=num or one day be default.
install –force
Unconditional update

Information is kept in a configuration file located in ~/.qa-dkrz/config.txt by default. User may not edit it, but could. If missing, it is restored during the next qa-dkrz call.

A corrupt or incomplete installation becoming apparent during an operational run would issue a message for resolving the conflict.

For those who want to have full access to the sources, but do not like to grapple with library dependencies, a hybrid operation of conda and GitHub based sources QA_DKRZ is possible. Just bind the conda installed libs to the file QA-DKRZ/install_configure and operate/update from the GitHub based QA-DKRZ.