05.13.21

Github Actions on Self-Hosted Runners for Kubernetes

By Edgar Acosta
Github-Actions-on-Self-Hosted-Runners-for-Kubernetes-scaled

Currently, 56 million developers are using GitHub to develop their applications today, and that statistic is predicted to grow to 100 million by 2025. Developers optimize GitHub for multiple tasks, from writing a piece of code for their application to deploying it in an environment. This requires a solution that can automate all the mundane tasks such as code reviews, issue tracking, branch management, etc., so that instead developers can focus on developing applications. GitHub Actions and runners provide the solutions for automating all the internal workflows from idea through to production.

What Are GitHub Actions?

As GitHub Action supports automated developer workflows, the feature means you can implement a CI/CD pipeline. By leveraging GitHub actions, every time something happens in your repository, you can configure automatic actions to execute in response. An engineer creating a pull request, creating an issue, joining as a contributor, and merging pull requests into the master branch are examples of GitHub action events.

GitHub Actions allow you to define pipeline style templates for workloads such as jobs, builds, and other various tasks in your GitHub repository. Start with a YAML file where you define a trigger which can be a pull request. Once your trigger kicks off, you can configure requests to compile code, package code, run docker containers, run arbitrary code, run a testing cycle, build or deploy the application.

To do all these tasks, you require GitHub runners.

GitHub Runners and Self-Hosted Runners

GitHub Runners are the applications responsible for running the jobs inside a GitHub action workflow. Host this runner in a virtual environment on any cloud provider (AWS, Azure, GCP), or it can be a self-hosted runner in your own environment. Self-Hosted Runners allow you to save a lot of time and effort by executing your pipelines with pre-installed and pre-configured tools needed for your pipeline jobs. These self-hosted runners can also run inside containers on a Kubernetes cluster.

In this article, we will learn how to optimize GitHub Actions with self-hosted runners deploying them on a Kubernetes cluster step by step.

Setting Up GitHub Actions

To follow the steps in this article, you need a GitHub account and a repository that is storing files. Go to the “Actions” tab inside your GitHub repository and click on “Set up this workflow”.

Getting started with GitHub Actions

This will create a YAML file automatically in the workflow which will build any new commit in the GitHub repository. Since a new file has been added, we can go ahead and commit the changes in the repository by clicking on start commit.

#github workflows

Once you commit the changes, a workflow will start, which will run the build for this commit. Click on “Create main.yaml” workflow to get the details of the build.

all workflows

Click on the build.

click on build

Now, you can see all the details of the build steps defined in main.yaml. 

build screen

The GitHub Action is ready. Let us set up a self-hosted GitHub runner now.

Setting Up Self-Hosted GitHub Action Runners

To set up a self-hosted GitHub runner, go to Settings > Actions in your GitHub repository, and scroll to the bottom. Here you will find “Self-hosted runners”, click on add runner to create a new self-hosted runner.

self-hosted runners

This will provide the steps to install a GitHub action runner on your machine, depending on the operating system you have. These are the steps to set up an action runner on an Ubuntu machine.

# Create a folder
osboxes@osboxes:~$ mkdir actions-runner && cd actions-runner
# Download the latest runner package
osboxes@osboxes:~$ curl -o actions-runner-linux-x64-2.278.0.tar.gz -L https://github.com/actions/runner/releases/download/v2.278.0/actions-runner-linux-x64-2.278.0.tar.gz
# Extract the installer
osboxes@osboxes:~$ tar xzf ./actions-runner-linux-x64-2.278.0.tar.gz
# Create the runner and start the configuration experience
osboxes@osboxes:~/actions-runner$ ./config.sh --url https://github.com/<GITHUB_USER_ALIAS>/<REPOSITORY_NAME> --token ANDM2G24R4H4WTD3GCPW58LARVT2Q

This command will start the GitHub Actions self-hosted runner registration. It will ask the name, label, and workspace of the runner you want to have and finally save all the settings.

self-hosted runner registration

Finally, start the self-hosted runner to listen to the jobs running.

osboxes@osboxes:~$  ./run.sh
connected to github

Now go back to Settings > Actions in your GitHub repository and refresh the page, you will see the runner is up and running and in the idle state.

linux runners

You need to use the self-hosted label for all the self-hosted runners. Now to run the GitHub actions in your self-hosted runner, edit the “runs-on” parameter in the main.yaml file as self-hosted and commit this change.

workflows and triggers

The commit will create another workflow run with a build, running inside your self-hosted Github runner.

new workflow

Self-hosted GitHub Actions Runners on Kubernetes 

Now that the self-hosted runner is running on a linux machine, let’s look at how to deploy it on Kubernetes. To follow the next steps, you’ll need a Kubernetes cluster running.

Firstly, I will need a dockerfile to build a docker image. In this dockerfile, I will install GitHub runner binary and start the container using GitHub Personal Access Token.

osboxes@osboxes:~$ gedit dockerfile

FROM debian:jessie
ARG RUNNER_VERSION="2.278.0"

ENV GITHUB_PERSONAL_TOKEN ""
ENV GITHUB_OWNER ""
ENV GITHUB_REPOSITORY ""

RUN apt-get update \
    && apt-get install -y \
        curl \
        sudo \
        git \
        jq \
        tar \
        gnupg2 \
        apt-transport-https \
        ca-certificates  \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*
    
RUN useradd -m github && \
    usermod -aG sudo github && \
    echo "%sudo ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers

USER github
WORKDIR /home/github

RUN curl -O -L https://github.com/actions/runner/releases/download/v$RUNNER_VERSION/actions-runner-linux-x64-$RUNNER_VERSION.tar.gz
RUN tar xzf ./actions-runner-linux-x64-$RUNNER_VERSION.tar.gz
RUN sudo ./bin/installdependencies.sh

COPY --chown=github:github entrypoint.sh ./entrypoint.sh
RUN sudo chmod u+x ./entrypoint.sh

ENTRYPOINT ["/home/github/entrypoint.sh"]



The entrypoint will register the GitHub token and install the runner’s software.

osboxes@osboxes:~$ gedit entrypoint.sh

#!/bin/sh
registration_url="https://api.github.com/repos/${GITHUB_OWNER}/${GITHUB_REPOSITORY}/actions/runners/registration-token"
echo "Requesting registration URL at '${registration_url}'"
payload=$(curl -sX POST -H "Authorization: token ${GITHUB_PAT}" ${registration_url})
export RUNNER_TOKEN=$(echo $payload | jq .token --raw-output)
./config.sh \
    --name $(hostname) \
    --token ${RUNNER_TOKEN} \
    --url https://github.com/${GITHUB_OWNER}/${GITHUB_REPOSITORY} \
    --work ${RUNNER_WORKDIR} \
    --unattended \
    --replace
remove() {
    ./config.sh remove --unattended --token "${RUNNER_TOKEN}"
}
trap 'remove; exit 130' INT
trap 'remove; exit 143' TERM
./run.sh "$*" &
wait $!

Now, build the docker image. It will run the dockerfile and execute all the steps one by one to create the runner docker image.

osboxes@osboxes:~$ docker build . -t runner
Sending build context to Docker daemon  2.107GB
Step 1/14 : FROM debian: jessie
jessie: Pulling from library/debian
---> 48e774d3c4f5
Step 2/14 : ARG GITHUB_RUNNER_VERSION="2.278.0"
Step 3/14 : ENV RUNNER_NAME "runner"
Step 4/14 : ENV GITHUB_PAT ""
Step 5/14 : ENV GITHUB_OWNER ""
Step 6/14 : ENV GITHUB_REPOSITORY ""
Step 7/14 : ENV RUNNER_WORKDIR "_work"
Step 8/14 : RUN apt-get update     && apt-get install -y         curl         sudo         git         jq     && apt-get clean     && rm -rf /var/lib/apt/lists/*     && useradd -m github     && usermod -aG sudo github     && echo "%sudo ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers
Step 9/14 : USER github
Step 10/14 : WORKDIR /home/github
Step 11/14 : RUN curl -Ls https://github.com/actions/runner/releases/download/v${GITHUB_RUNNER_VERSION}/actions-runner-linux-x64-${GITHUB_RUNNER_VERSION}.tar.gz | tar xz     && sudo ./bin/installdependencies.sh
Step 12/14 : COPY --chown=github:github entrypoint.sh ./entrypoint.sh
Step 13/14 : RUN sudo chmod u+x ./entrypoint.sh
Step 14/14 : ENTRYPOINT ["/home/github/entrypoint.sh"]
Successfully built d10b1931c41e
Successfully tagged runner:latest

When you check now, you find a new docker image “runner”. 

osboxes@osboxes:~$ docker images
REPOSITORY                          TAG           IMAGE ID       CREATED         SIZE
runner                              latest        d10b1931c41e   4 minutes ago   422MB

Push this runner docker image on DockerHub. This docker image will be used for the Kubernetes deployment in the coming steps.

Next, you need to create a namespace on the Kubernetes cluster where the runner deployment will happen.

osboxes@osboxes:~$ kubectl create ns gitaction

Go to GitHub Account Settings > Developer settings > Personal access tokens and create a new token to authenticate runner to the deployment.

Create a secret file to pass the GitHub personal access token to the deployment.

osboxes@osboxes:~$ gedit secret.yaml

apiVersion: v1
kind: Secret
metadata:
  name: github-secret
  namespace: gitaction
type: Opaque
stringData:
  GITHUB_PERSONAL_TOKEN: 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

Apply the secret using kubectl.

osboxes@osboxes:~$ kubectl apply -f ./secret.yaml
secret/github-secret created

Now the most crucial part, create the deployment to run the containers of runner docker image inside the Kubernetes pods. This deployment will use the secret stored in secret.yaml and spin one runner on the Kubernetes cluster inside the gitaction namespace.

osboxes@osboxes:~$ gedit Kubernetes.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: runner
  namespace: gitaction
  labels:
    app: runner
spec:
  replicas: 1
  selector:
    matchLabels:
      app: runner
  template:
    metadata:
      labels:
        app: runner
    spec:
      containers:
      - name: runner
        image: geekman/runner:latest
        env:
        - name: GITHUB_OWNER
          value: geekman
        - name: GITHUB_REPOSITORY
          value: jenkins-pipeline
        - name: GITHUB_PERSONAL_TOKEN 
          valueFrom:
            secretKeyRef:
              name: github-secret
              key: GITHUB_PERSONAL_TOKEN

Apply the deployment using kubectl.

osboxes@osboxes:~$ kubectl apply -f kubernetes.yaml 
deployment.apps/linux-runner configured

Check the status of the deployment.

osboxes@osboxes:~$ kubectl -n gitaction get deployment
NAME           READY   UP-TO-DATE   AVAILABLE   AGE
runner         1/1     1            0           2m

This command will show the single runner pod running inside the gitaction namespace.

osboxes@osboxes:~$ kubectl -n gitaction get pods
NAME                            READY   STATUS             RESTARTS   AGE
runner-74c97495cb-8s7m7         1/1     Running             1         15s

On Kubernetes cluster, you can easily scale the runner container by increasing the replicas. The command below starts 2 more runner pods on the cluster.

osboxes@osboxes:~$ kubectl -n gitaction scale deployment runner --replicas=3
deployment.extensions/runner scaled

Now if you check the number of runner pods, it will be 3. Two new pods are starting.

NAME                                   READY             STATUS             RESTARTS   AGE
runner-74c97495cb-6r5f7                 1/1              ContainerCreating    0          15s
runner-74c97495cb-7y5r6                 1/1              ContainerCreating    0          15s
runner-74c97495cb-8s7m7                 1/1              Running              0          2m

Congrats, you have successfully deployed self-hosted GitHub action runners on a Kubernetes cluster.

Conclusion

Optimize GitHub Action as a CI/CD solution for your pipelines to build, test, deploy using YAML configurations. Self-hosted runners can help you save a lot of costs that you would incur using a cloud provider. Go ahead and leverage your own GitHub Actions and self-hosted runners before deploying them on a Kubernetes cluster.


Caylent provides a critical DevOps-as-a-Service function to high growth companies looking for expert support with Kubernetes, cloud security, cloud infrastructure, and CI/CD pipelines. Our managed and consulting services are a more cost-effective option than hiring in-house, and we scale as your team and company grow. Check out some of the use cases, learn how we work with clients, and read more about our DevOps-as-a-Service offering.