Continuous Integration / Continuous Deployment (CI/CD) - Github/Gitea Edition

GitHub is a distributed git platform used for code hosting and collaboration. It can also be used to automatically run the hosted code on Github’s servers via GitHub Actions. Actions are workflow automation scripts. We’ll learn how to develop on to make our code robust to errors, preserved, and reproducible.

Gitea is an open source replacement for GitHub.

The aim of this module is to help build and explore the meaning of a CI/CD workflow in both platforms.

Start Lesson

Prerequisites

This assumes that you’ll have some basic background with your command line, for example:

How to execute custom shell scripts (if you are not familiar with the shell, click here)
How to run python scripts (if you are not familiar with python, click here)
How to interact with remotes in git (if you are not familiar with git, click here; for simplified authentication with the gh command line interface, see this version of the git training)

The skills we’ll focus on:

Making scripts exit correctly
Building a CI/CD workflow of unlimited potential
Understanding how job runners work (and get access to your clones)
Protecting secret information while allowing jobs to run

Learning Objectives

Episode	Learning Objectives
Introduction	Understand why CI/CD is important Learn what can be possible with CI/CD Find resources to explore in more depth
Exit Codes	Understand exit codes How to print exit codes How to set exit codes in a script How to ignore exit codes Create a script that terminates in success/error
Being Assertive	Understand how assertions in Python correspond to exit codes Figure out how pytest fits in
Understanding YAML	Learn about YAML
YAML and GitHub Actions	Learn where to find more details about everything for the GitHub Actions. Understand the components of GitHub Actions YAML file.
Coffee break!	Refresh your mind.
Hello CI World	Add CI/CD to your project.
Adding CI to Your Existing Code	Learn how to get your CI/CD Runners to build your code Try and see if the CI/CD can catch problems with our code.
Matrix	Don’t Repeat Yourself (DRY) Use a single job for multiple jobs
Coffee break!	Refresh your mind.
Dependent Jobs	Run some jobs in serial.
A Skimmer Higgs	Learn how to skim code and set up artifacts.
Getting into the Spy Game	Add custom environment variables Learn how to give your CI/CD Runners access to private information
Making Plots to Take Over The World	Use everything you learned to make plots!
Let's Actually Make A Test (For Real)	Actually add a test on the output of running physics
Homework	Add more testing, perhaps to statistics.
Bonus Episode: Building and deploying a Docker container to Github Packages	To be able to build a Docker container and share it via GitHub Packages

Schedule

Time	Episode	Questions
	Setup	Before you start
00:00	Introduction 5 min	What is continuous integration / continuous deployment?
00:05	Exit Codes 20 min	What is an exit code?
00:25	Being Assertive 10 min	What happens with assertions in Python?
00:35	Understanding YAML 5 min	What is YAML?
00:40	YAML and GitHub Actions 5 min	What is the GitHub Actions specification?
00:45	Coffee break! 15 min	Get up, stretch out, take a short break.
01:00	Hello CI World 15 min	How do I run a simple GitHub Actions job?
01:15	Adding CI to Your Existing Code 15 min	I have code already in GitHub, how can I add CI to it?
01:30	Matrix 15 min	How can we make job templates?
01:45	Coffee break! 15 min	Get up, stretch out, take a short break.
02:00	Dependent Jobs 10 min	How do you make some jobs run after other jobs?
02:10	A Skimmer Higgs 15 min	How can I run my skimming code in the GitHub Actions?
02:25	Getting into the Spy Game 15 min	How can I give my GitHub Actions private information?
02:40	Making Plots to Take Over The World 15 min	How do we make plots?
02:55	Let's Actually Make A Test (For Real) 25 min	How does a realistic workflow look for a physics analysis?
03:20	Homework 30 min	Where to go next? Want some homework?
03:50	Bonus Episode: Building and deploying a Docker container to Github Packages 40 min	How to build a Docker container for Python packages? How to share Docker images?

Authors and Contributors

Contributor	ORCID
Giordon Stark	0000-0001-6616-3433
Saransh Chopra	0000-0003-3046-7675
Aman Desai	0000-0003-2631-9696
Max Fatouros	0000-0002-7438-652X
Guillermo A. Fidalgo-Rodríguez	0000-0001-8605-9772
Clemens Lange	0000-0002-3632-3157
Kilian Lieret	0000-0003-2792-7511
Laura Marqués	0000-0002-3593-5557
Emery Nibigira	0000-0001-5821-291X