Matrix

Overview

Teaching: 5 min
Exercises: 10 min
Questions
  • How can we make job templates?

Objectives
  • Don’t Repeat Yourself (DRY)

  • Use a single job for multiple jobs

Matrix

From the previous lesson, we tried to build the code against two different ROOT images by adding an extra job:

jobs:
  greeting:
    runs-on: ubuntu-latest
    steps:
      - run: echo hello world

  build_skim:
    runs-on: ubuntu-latest
    container: rootproject/root-conda:6.18.04
    steps:
      - name: checkout repository
        uses: actions/checkout@v2
      - name: build
        run: |
          COMPILER=$(root-config --cxx)
          FLAGS=$(root-config --cflags --libs)
          $COMPILER -g -O3 -Wall -Wextra -Wpedantic -o skim skim.cxx $FLAGS

  build_skim_latest:
    runs-on: ubuntu-latest
    container: rootproject/root-conda:latest
    steps:
      - name: checkout repository
        uses: actions/checkout@v2
      - name: latest
        run: |
          COMPILER=$(root-config --cxx)
          FLAGS=$(root-config --cflags --libs)
          $COMPILER -g -O3 -Wall -Wextra -Wpedantic -o skim skim.cxx $FLAGS

Building a matrix across different versions

We could do better using matrix. The latter allows us to test the code against a combination of versions in a single job.

jobs:
  greeting:
   runs-on: ubuntu-latest
   steps:
     - run: echo hello world

 build_skim:
   runs-on: ubuntu-latest
   container: rootproject/root-conda:${{ matrix.version }}
   strategy:
     matrix:
       version: [6.18.04, latest]
   steps:
     - name: checkout repository
       uses: actions/checkout@v2

     - name: build
        run: |
          COMPILER=$(root-config --cxx)
          FLAGS=$(root-config --cflags --libs)
          $COMPILER -g -O3 -Wall -Wextra -Wpedantic -o skim skim.cxx $FLAGS

More details on matrix: https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions.

Let’s update our .github/workflow/main.yml and use act to run the job with matrix.

act -j build_skim
[example/build_skim-1] 🧪  Matrix: map[version:6.18.04]
[example/build_skim-1] 🚀  Start image=rootproject/root-conda:6.18.04
[example/build_skim-2] 🧪  Matrix: map[version:latest]
[example/build_skim-2] 🚀  Start image=rootproject/root-conda:latest
[example/build_skim-1]   🐳  docker run image=rootproject/root-conda:6.18.04 entrypoint=["/usr/bin/tail" "-f" "/dev/null"] cmd=[]
[example/build_skim-1]   🐳  docker cp src=/tmp/eventselection/. dst=/github/workspace
[example/build_skim-1] ⭐  Run checkout repository
[example/build_skim-1]   ✅  Success - checkout repository
[example/build_skim-1] ⭐  Run COMPILER=$(root-config --cxx)
FLAGS=$(root-config --cflags --libs)
$COMPILER -g -O3 -Wall -Wextra -Wpedantic -o skim skim.cxx $FLAGS
[example/build_skim-1]   ✅  Success - COMPILER=$(root-config --cxx)
FLAGS=$(root-config --cflags --libs)
$COMPILER -g -O3 -Wall -Wextra -Wpedantic -o skim skim.cxx $FLAGS
[example/build_skim-2]   🐳  docker run image=rootproject/root-conda:latest entrypoint=["/usr/bin/tail" "-f" "/dev/null"] cmd=[]
[example/build_skim-2]   🐳  docker cp src=/tmp/eventselection/. dst=/github/workspace
[example/build_skim-2] ⭐  Run checkout repository
[example/build_skim-2]   ✅  Success - checkout repository
[example/build_skim-2] ⭐  Run COMPILER=$(root-config --cxx)
FLAGS=$(root-config --cflags --libs)
$COMPILER -g -O3 -Wall -Wextra -Wpedantic -o skim skim.cxx $FLAGS
[example/build_skim-2]   ✅  Success - COMPILER=$(root-config --cxx)
FLAGS=$(root-config --cflags --libs)
$COMPILER -g -O3 -Wall -Wextra -Wpedantic -o skim skim.cxx $FLAGS

We can push the changes to GitHub and see how it will look like.

git add .github/workflows/main.yml
git commit -m "add multi jobs"
git push -u origin feature/add-ci

While the jobs are running, let’s imagine we don’t want our CI/CD to crash if that happens. We have to add continue-on-error: true to a job

runs-on: ubuntu-latest
continue-on-error: true

For the matrix case, Github Actions fails the entire workflow and stops all the running jobs if any of the jobs in the matrix fails. This can be prevented by using fail-fast: false key:value.

strategy:
  fail-fast: false

More details: https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions.

Allow a specific matrix job to fail

But what if we want to only allow the job with version set to latest to fail without failing the workflow run?

Solution

runs-on: ubuntu-latest
container: rootproject/root-conda:${{ matrix.version }}
continue-on-error: ${{ matrix.allow_failure }}
strategy:
  matrix:
    version: [6.18.04]
    allow_failure: [false]
    include:
      - version: latest
        allow_failure: true

Look how much cleaner you’ve made the code. You should now see that it’s pretty easy to start adding more build jobs for other images in a relatively clean way, as you’ve now abstracted the actual building from the definitions.

Key Points

  • Using matrix allows to test the code against a combination of versions.