Best Practice
Installation
- Quick and easy:
conda create -n qa-dkrz -c conda-forge -c h-dh qa-dkrz
- Full sources:
git clone https://github.com/IS-ENES-Data/QA-DKRZ
- Use access to locally provided libraries via the
install_configure
file
- CMIP6 with PrePARE.py:
conda create -n cmor -c conda-forge -c pcmdi -c uvcdat cmor
- Verify the success of the installation by running
qa-dkrz --example[=path]
,
see Results.
- Automatic updating is enabled by option
--auto-up
until --auto-up=d[isable]
is provided;
works for both qa-dkrz
and install
.
Before a Run
- There is a default for almost every option in the project tables
- Gather frequently changed options in a file and provided on the command-line (-f);
this file could also contain QA_CONF=with-user-defined-name.
- Options on the command-line (but -f) only for testing or rechecking small data sets.
- Use the SELECT option to partition the entire data set, e.g. for CORDEX,
SELECT AFR-44=
(note that in this case ‘=’ is required;
may-be with prefixed ‘.*/’ depending on PROJECT_DATA)
would check the data of every model available for the specified domain.
On the other hand, SELECT .*/historical=orog
would find any orography
file in all historical in the given PROJECT_DATA=sub-dir-tree
.
- Option
CT_PATH_INDEX
defines the name of
a consistency table, which is created and utilised to check consistency across
sub-atomic files and across parent experiments, if available,
of a given variable. This kind of table is consulted for
experiments and versions matching the same name.
- Similarly for logfile names by
LOG_PATH_INDEX
Operational Mode
- Before starting to check data, please make sure that the configuration was set
as desired by
qa-dkrz [opts] -e_show_conf
.
- Check the selected experiments:
qa_DKRZ [opts] -e_show_exp
.
- Try for a single file first and inspect the log-file, i.e. run
qa-dkrz [opts] -e_next
, then resume without -e_next.
- Use
nohup
for long-term execution in the background.
- If the script is run in the foreground, then command-line option ‘-m’ may
be helpful by showing the current file name and the number
of variables (done/selected) on a status line .
- Have a look at the QA results in directory
QA_RESULTS/check_logs/Annotationi/experiment-name.json
.
- Manual termination of a session: if an immediate break is required,
please inquire the process-id (pid), e.g. by
ps -ef
, and execute the
command ‘kill -TERM pid’. This will close the current session neatly
leaving no remnants and being ready for resumption.