Adding CI to Your Existing Code

Overview

Teaching: 5 min
Exercises: 10 min
Questions
  • I have code already in GitHub, how can I add CI to it?

Objectives
  • Learn how to get your CI/CD Runners to build your code

  • Try and see if the CI/CD can catch problems with our code.

Time To Skim

The Naive Attempt

As of right now, your .github/workflows/main.yml should look like

name: example
on: push
jobs:
  greeting:
    runs-on: ubuntu-latest
    steps:
      - run: echo hello world

Let’s go ahead and teach our CI to build our code. Let’s add another job (named build_skim) that runs in parallel for right now, and runs the compiler ROOT uses.
Note: ROOT is a open source framework uses in High Energy Physics.

Let’s give a try.

COMPILER=$(root-config --cxx)
$COMPILER -g -O3 -Wall -Wextra -Wpedantic -o skim skim.cxx

The compilation will result in an output binary called skim.

Adding a new job

How do we change the CI in order to add a new job that compiles our code?

Solution

jobs:
  greeting:
    runs-on: ubuntu-latest
    steps:
      - run: echo hello world

  build_skim:
    runs-on: ubuntu-latest
    steps:
      - name: build
        run: |
          COMPILER=$(root-config --cxx)
          $COMPILER -g -O3 -Wall -Wextra -Wpedantic -o skim skim.cxx

Check jobs

act -l
ID          Stage  Name        
build_skim  0      build_skim  
greeting    0      greeting

We can see 2 parallel jobs. Let’s commit the changes we made.

git add .github/workflows/main.yml
git commit -m "add build skim job"
git push -u origin feature/add-actions

Job failure root-config Job failure root-config details

No root-config?

Ok, so maybe we were a little naive here. Let’s start debugging locally.

We can reproduce this error:

act -j build_skim

You got this error when you tried to build

[example/build_skim] 🚀  Start image=catthehacker/ubuntu:act-latest
[example/build_skim]   🐳  docker run image=catthehacker/ubuntu:act-latest entrypoint=["/usr/bin/tail" "-f" "/dev/null"] cmd=[]
[example/build_skim] ⭐  Run COMPILER=$(root-config --cxx)
$COMPILER -g -O3 -Wall -Wextra -Wpedantic -o skim skim.cxx
| /github/workflow/0: line 1: root-config: command not found
[example/build_skim]   ❌  Failure - COMPILER=$(root-config --cxx)

Broken Build

What happened?

Answer

It turns out we had the wrong docker image for our build. If you look at the log, you’ll see

🚀  Start image=catthehacker/ubuntu:act-latest
  🐳  docker run image=catthehacker/ubuntu:act-latest entrypoint=["/usr/bin/tail" "-f" "/dev/null"] cmd=[]

How do we fix it? We need to define the image with ROOT installed.

Docker???

Don’t panic. You do not need to understand docker to be able to use it.

Let’s go ahead and update our .github/workflow/main.yml and fix it to use a versioned docker image that has root: rootproject/root-conda:6.18.04 from the rootproject/root-conda docker hub page.

runs-on: ubuntu-latest
container: rootproject/root-conda:6.18.04

If you run again act -j build_skim, you see

[example/build_skim] 🚀  Start image=rootproject/root-conda:6.18.04

Failed again???

What’s that?
error: skim.cxx: No such file or directory

Answer

It seems the job cannot access the repository. We need to instruct GitHub actions to checkout the repository.

steps:
   - name: checkout repository
     uses: actions/checkout@v2

Let’s go ahead and tell our CI to checkout the repository.

Still failed??? What the hell.

What happened?

Answer

It turns out we just forgot the include flags needed for compilation. If you look at the log, you’ll see

 | skim.cxx:11:10: fatal error: ROOT/RDataFrame.hxx: No such file or directory
 |  #include "ROOT/RDataFrame.hxx"
 |           ^~~~~~~~~~~~~~~~~~~~~
  #include "ROOT/RDataFrame.hxx"
           ^~~~~~~~~~~~~~~~~~~~~
 | compilation terminated.
 Error: exit with `FAILURE`: 1

How do we fix it? We just need to add another variable to add the flags at the end via $FLAGS defined as FLAGS=$(root-config --cflags --libs).

Ok, let’s go ahead and update our .github/workflow/main.yml again, and it better be fixed or so help me…

Building multiple versions

Great, so we finally got it working… CI/CD isn’t obviously powerful when you’re only building one thing. Let’s build both the version of the code we’re testing and also test that the latest ROOT image (rootproject/root-conda:latest) works with our code. Call this new job build_skim_latest.

Adding the build_skim_latest job

What does the .github/workflow/main.yml look like now?

Solution

jobs:
  greeting:
    runs-on: ubuntu-latest
    steps:
      - run: echo hello world

  build_skim:
    runs-on: ubuntu-latest
    container: rootproject/root-conda:6.18.04
    steps:
      - name: checkout repository
        uses: actions/checkout@v2

      - name: build
        run: |
          COMPILER=$(root-config --cxx)
          FLAGS=$(root-config --cflags --libs)
          $COMPILER -g -O3 -Wall -Wextra -Wpedantic -o skim skim.cxx $FLAGS

  build_skim_latest:
    runs-on: ubuntu-latest
    container: rootproject/root-conda:latest
    steps:
      - name: checkout repository
        uses: actions/checkout@v2

      - name: latest
        run: |
          COMPILER=$(root-config --cxx)
          FLAGS=$(root-config --cflags --libs)
          $COMPILER -g -O3 -Wall -Wextra -Wpedantic -o skim skim.cxx $FLAGS

We’re ready for a coffee break.

Key Points

  • Setting up CI/CD shouldn’t be mind-numbing

  • All defined jobs run in parallel by default