Active learning step

Experimental data is useful to constrain the allowed parameter ranges in theoretical models. In practise, we are looking for exclusion contours in the parameter space. When the number of free parameter is larger, it is crucial to sketch the contour in the high dimensional parameter space using minimum amount of points. This can be achieved by a smarter decision-making process on values of parameters for next loops based on the historical upper limit results. This process is the last step of a single loop and can be mapped to the junction/checkpoint step of pchain.

A popular algorithm for solving such tasks is Gaussian Process, although other algorithms are possible. We provide a container (gitlab-registry.cern.ch/zhangruihpc/steeringcontainer:0.0.7) with many of the algorithms available and a script active_learning_zz.py as an example to demonstrate how to call the algorithm (the script can be either inside the container or submit as a local file through pchain). The script essentially used APIs from a library called hpogrid (repo) but you are free to directly access the available algorithms hyperopt, nevergrad, scikit-optimize, or Ray[tune].

With the following command, the script will output two JSON files results.json and history.json. The first file contains updated values for the local parameter and is needed by the junction executable as discussed in Section Workflow description in CWL. The second is a persistent file through all loops to record all historical results. These will be used when construct the junction block in pchain discussed shortly.

# '/ATLASMLHPO/module/active_learning_zz.py' is the full path inside this container
python /ATLASMLHPO/module/active_learning_zz.py -pv myparamMZD 30 myparamMH 60

Warning

We are extending the junction step to allow parallelising more jobs and the command and script may update.

Integrate to pchain

Below is the junction step in CWL.

Junction step in pchain

  checkpoint:
    run: junction
    in:
      opt_inDS: inner_work_reana/outDS
      opt_containerImage:
        default: docker://gitlab-registry.cern.ch/zhangruihpc/steeringcontainer:0.0.7
      opt_exec:
        default: "bash script_junction.sh %{i} %{DS0} %{myparamMZD} %{myparamMHD}"
      opt_args:
        default: "--site CERN --outputs results.json --persistentFile history.json --forceStaged"
    out: [outDS]

Again, all the magic is in the script_junction.sh script. Here is how it look like:

script_junction.sh

#!/bin/bash

tar xvf *tar
nloop=4

if [[ $1 -ge $nloop ]]; then
    to_continue='false'
else
    to_continue='true'
fi

python /ATLASMLHPO/module/active_learning_zz.py -s parameter_space.json -l limit_stage/output.json -pv myparamMZD $3 myparamMHD $4 -c $to_continue;

In this example, it runs four loops to the end as indicated in L4.