Currently, 56 million developers are using GitHub to develop their applications today, and that statistic is predicted to grow to 100 million by 2025....
Explore how we helped our customer in the financial sector migrate from High-Performance Computing (HPC) workloads on an on-premise Slurm cluster to AWS ParallelCluster, detailing the process, challenges, and benefits.
Migrating to the cloud is deeply desirable due to ease of the management, scalability and many other factors, however poor choices in the migration process can lead to increased costs, poor performance and tech debt. Learn about the top 7 cloud migration mistakes and how to avoid them.
Learn how AWS Control Tower can accelerate your cloud foundation by simplifying and automating AWS account creation, governance & management.
Currently, 56 million developers are using GitHub to develop their applications today, and that statistic is predicted to grow to 100 million by 2025. Developers optimize GitHub for multiple tasks, from writing a piece of code for their application to deploying it in an environment. This requires a solution that can automate all the mundane tasks such as code reviews, issue tracking, branch management, etc., so that instead developers can focus on developing applications. GitHub Actions and runners provide the solutions for automating all the internal workflows from idea through to production.
As GitHub Action supports automated developer workflows, the feature means you can implement a CI/CD pipeline. By leveraging GitHub actions, every time something happens in your repository, you can configure automatic actions to execute in response. An engineer creating a pull request, creating an issue, joining as a contributor, and merging pull requests into the master branch are examples of GitHub action events.
GitHub Actions allow you to define pipeline style templates for workloads such as jobs, builds, and other various tasks in your GitHub repository. Start with a YAML file where you define a trigger which can be a pull request. Once your trigger kicks off, you can configure requests to compile code, package code, run docker containers, run arbitrary code, run a testing cycle, build or deploy the application.
To do all these tasks, you require GitHub runners.
GitHub Runners are the applications responsible for running the jobs inside a GitHub action workflow. Host this runner in a virtual environment on any cloud provider (AWS, Azure, GCP), or it can be a self-hosted runner in your own environment. Self-Hosted Runners allow you to save a lot of time and effort by executing your pipelines with pre-installed and pre-configured tools needed for your pipeline jobs. These self-hosted runners can also run inside containers on a Kubernetes cluster.
In this article, we will learn how to optimize GitHub Actions with self-hosted runners deploying them on a Kubernetes cluster step by step.
To follow the steps in this article, you need a GitHub account and a repository that is storing files. Go to the “Actions” tab inside your GitHub repository and click on “Set up this workflow”.
This will create a YAML file automatically in the workflow which will build any new commit in the GitHub repository. Since a new file has been added, we can go ahead and commit the changes in the repository by clicking on start commit.
Once you commit the changes, a workflow will start, which will run the build for this commit. Click on “Create main.yaml” workflow to get the details of the build.
Click on the build.
Now, you can see all the details of the build steps defined in main.yaml.
The GitHub Action is ready. Let us set up a self-hosted GitHub runner now.
To set up a self-hosted GitHub runner, go to Settings > Actions in your GitHub repository, and scroll to the bottom. Here you will find “Self-hosted runners”, click on add runner to create a new self-hosted runner.
This will provide the steps to install a GitHub action runner on your machine, depending on the operating system you have. These are the steps to set up an action runner on an Ubuntu machine.
This command will start the GitHub Actions self-hosted runner registration. It will ask the name, label, and workspace of the runner you want to have and finally save all the settings.
Finally, start the self-hosted runner to listen to the jobs running.
Now go back to Settings > Actions in your GitHub repository and refresh the page, you will see the runner is up and running and in the idle state.
You need to use the self-hosted label for all the self-hosted runners. Now to run the GitHub actions in your self-hosted runner, edit the “runs-on” parameter in the main.yaml file as self-hosted and commit this change.
The commit will create another workflow run with a build, running inside your self-hosted Github runner.
Now that the self-hosted runner is running on a linux machine, let’s look at how to deploy it on Kubernetes. To follow the next steps, you’ll need a Kubernetes cluster running.
Firstly, I will need a dockerfile to build a docker image. In this dockerfile, I will install GitHub runner binary and start the container using GitHub Personal Access Token.
The entrypoint will register the GitHub token and install the runner’s software.
Now, build the docker image. It will run the dockerfile and execute all the steps one by one to create the runner docker image.
When you check now, you find a new docker image “runner”.
Push this runner docker image on DockerHub. This docker image will be used for the Kubernetes deployment in the coming steps.
Next, you need to create a namespace on the Kubernetes cluster where the runner deployment will happen.
Go to GitHub Account Settings > Developer settings > Personal access tokens and create a new token to authenticate runner to the deployment.
Create a secret file to pass the GitHub personal access token to the deployment.
Apply the secret using kubectl.
Now the most crucial part, create the deployment to run the containers of runner docker image inside the Kubernetes pods. This deployment will use the secret stored in secret.yaml and spin one runner on the Kubernetes cluster inside the gitaction namespace.
Apply the deployment using kubectl.
Check the status of the deployment.
This command will show the single runner pod running inside the gitaction namespace.
On Kubernetes cluster, you can easily scale the runner container by increasing the replicas. The command below starts 2 more runner pods on the cluster.
Now if you check the number of runner pods, it will be 3. Two new pods are starting.
Congrats, you have successfully deployed self-hosted GitHub action runners on a Kubernetes cluster.
Optimize GitHub Action as a CI/CD solution for your pipelines to build, test, deploy using YAML configurations. Self-hosted runners can help you save a lot of costs that you would incur using a cloud provider. Go ahead and leverage your own GitHub Actions and self-hosted runners before deploying them on a Kubernetes cluster.
Caylent provides a critical DevOps-as-a-Service function to high growth companies looking for expert support with Kubernetes, cloud security, cloud infrastructure, and CI/CD pipelines. Our managed and consulting services are a more cost-effective option than hiring in-house, and we scale as your team and company grow. Check out some of the use cases, learn how we work with clients, and read more about our Infrastructure and DevOps Modernization services.
Caylent Team