Installation¶
QA-DKRZ may be installed either via the conda package manager or from a GitHub repository.
Recommended is the conda package manager installing a ready-to-use package. At present, a 64-bit processor is required.
If you want to work with sources or use a different machine architecture, then the installation from source should be the first choice.
For CMIP6, QA-DKRZ binds the CMOR validator PrePARE.py (D. Nadeau, LLNL, https://cmor.llnl.gov). This is conducted only by conda, e.g.
$ conda create -n cmor -c conda-forge -c pcmdi -c uvcdat cmor
Details in section Work-flow below.
No matter what the preference is conda or from GitHub, you are asked for a location of tables and programs during the installation.
By Conda Package Manager¶
Make sure that you have conda installed. The quick steps to install miniconda on Linux 64-bit are:
$ wget https://repo.continuum.io/miniconda/Miniconda-latest-Linux-x86_64.sh
$ bash Miniconda-latest-Linux-x86_64.sh
Note
The installation is tested on various 64-bit machines.
Please check that the conda
package manager is in your PATH
.
For example you may set the PATH
environment variable as following:
$ export PATH=$PATH:your-path/your-miniconda/bin
$ conda -h
conda provides the QA-DKRZ package with all dependencies included. You may install it to the root environment or to any other with a name of your choice, e.g.
$ conda create -n qa-dkrz -c conda-forge -c h-dh qa-dkrz
From GitHUB repository¶
Requirements¶
The tool requires the BASH shell and a C/C++ compiler (works for AIX, too).
Building the QA tool¶
The sources are downloaded from GitHub by
$ git clone https://github.com/IS-ENES-Data/QA-DKRZ
Any installation is done with the script install
( a prefix ‘./’ could
be helpful in some cases) within the QA-DKRZ
root directory. Please, have also a look
at the Work-flow section.
- A file
install_configure
inQA-DKRZ
binds necessarylib
andinclude
directories. A template.install_configure
is provided. If not copied toinstall_configure
and edited before the first launch, then it is created and the user is asked to edit the file. - Environmental variables CC, CXX, CFLAGS, and CXXFLAGS are accepted.
install
establishes access to libraries, which may be linked (recommended) or built (triggered by option--build
).- Proceedings are logged in file
install.log
. - The executables are compiled for projects named on the command-line.
- When
install
is started the first time for a given project, then corresponding external tables and programs are downloaded to aQA_HOME
directory; the user is asked for the path. - The user’s home directory contains a config.txt file in directory .qa-dkrz created and updated by user defined control statements.
The full set of options is described by:
$ ./install --help
Building Libraries¶
If you decide to use your own set of libraries (accessing provided ones is preferred by respective settings in the install_configure file), then this is accomplished by
$ ./install --build [opts]
Sources of the following libraries are downloaded and installed:
- zlib-1.2.8 from www.zlib.net,
- hdf5-1.8.9 from www.hdfgroup.org,
- netcdf-4.3.0 from www.unidata.ucar.edu (shared, no FORTRAN, non-parallel),
- udunits package from http://www.unidata.ucar.edu/packages/udunits.
The libraries are built in sub-directory local/source
.
If libraries had been built previously, then the sources are updated and
the libraries are rebuilt.
Work-flow¶
Please note that the work-flow has changed. By now, operational mode is generally distinct from installation or updating.
Operation is conducted (with some reasonable options) by e.g.
$ qa-dkrz -f task-file
with task-file containing frequently or task-specific options like
PROJECT_DATA
, QA_RESULTS
, SELECT
, EMAIL_SUMMARY
, NUM_EXEC_THREADS
,
CHECK_MODE
and PROJECT
or QA_CONF
.
The full set of options used by default is given in file
QA_HOME/tables/projects/PROJECT/PROJECT_qa.conf
with QA_HOME
chosen during the installation. Additionally, every valid option
may be provided on the command-line by prefixing -e
.
Note that option RUN_CMOR3_LLNL
must be given for running PrePARE.py.
Installation and update tasks are indicated by option --install=...
, which
is additionally provided to the qa-dkrz call. Then, all options of install
have to be given by a comma-separated list; the usual --
prefix of options
may be omitted, e.g.
$ qa-dkrz --install=help
would display all install options.
The --install
option may be merged with operational options conducted afterwards.
The separation of operation and install/updating is broken by option --auto-up
for
backward-compatibility. Additionally, when working with the GitHub based package,
the --install
option may be replaced by the QA-DKRZ/install command
(then options with --
prefix).
Note that conda does not know the path to install.
Operation/installation/update with conda requires activating and deactivating the corresponding environment. Provided that conda is within the path or aliased:
$ source activate your-chosen-environment-name
$ qa-dkrz options
$ source deactivate.
Command which qa-dkrz while the environment is activated reveals the path to qa-dkrz. Given to PATH or as alias would make activate/deactivate redundant.
There are three stages for updating:
- install [PROJECT] without any additional option.
- Compilation of executables for PROJECT. Only for GitHub based installation; no effect for a conda installed qa-dkrz.
- install –up
- Updating once every other number of days, while num has been given by option
--uf=num
or one day be default. - install –force
- Unconditional update
Information is kept in a configuration file located in ~/.qa-dkrz/config.txt by default. User may not edit it, but could. If missing, it is restored during the next qa-dkrz call.
A corrupt or incomplete installation becoming apparent during an operational run would issue a message for resolving the conflict.
For those who want to have full access to the sources, but do not like to grapple with library dependencies, a hybrid operation of conda and GitHub based sources QA_DKRZ is possible. Just bind the conda installed libs to the file QA-DKRZ/install_configure and operate/update from the GitHub based QA-DKRZ.