CircleCI Self-hosted Runner Installation on Kubernetes

1 month ago • 3 min read

Cloud Server v3.x

Introduction
- Prerequisites
Getting Started
Chart Values
CircleCI server installation
- Upgrading self-hosted runner deployment for server
Limitations and pending work

Introduction

This installation guide is to help set up CircleCI sel-hosted runners on your Kubernetes cluster. We provide a Helm chart to simplify the installation process.

The Helm chart will spin up one or more pods of the same self-hosted runner resource class. This is useful for when you want all of these self-hosted runners to execute jobs requesting the same execution environment. Each runner will pull jobs off the queue on an as-available basis.

If you want to have different self-hosted runners specialized for different workloads, it is recommended to create different self-hosted runner resource classes and rerun these instructions and have separate charts for each self-hosted runner class you create.

If you are using Server, please make sure you read CircleCI server installation.

Prerequisites

Have a Kubernetes cluster up and running where you’d like to deploy your self-hosted runner(s).
Go through the steps to create and authenticate a CircleCI self-hosted runner resource class.

Getting Started

Clone the repository at https://github.com/CircleCI-Public/circleci-runner-k8s.
Modify the chart’s values in values.yaml per your needs. See documentation on the list of values.
Apply your chart to your cluster, setting the required parameters you should have obtained from the authentication step:
```
$ helm install "circleci-runner" ./ \
  --set runnerToken=$CIRCLECI_RUNNER_TOKEN \
  --set resourceClass=$CIRCLECI_RUNNER_RESOURCE_CLASS \
  --namespace your-namespace
```
For further information about configuring the helm chart see the readme

Verify your pods are up and running by checking their status and following the logs. You should see output like the following:

$ kubectl get pods --all-namespaces
NAMESPACE     NAME                                                             READY   STATUS    RESTARTS   AGE
default       circleci-runner-test-7d6b8fc6f-4z5wl                             1/1     Running   0          28h
default       circleci-runner-test-7d6b8fc6f-h97jz                             1/1     Running   0          28h
default       circleci-runner-test-7d6b8fc6f-pksc6                             1/1     Running   0          28h
default       circleci-runner-test-7d6b8fc6f-q74p4                             1/1     Running   0          28h
default       circleci-runner-test-7d6b8fc6f-wh6m2                             1/1     Running   0          28h

$ kubectl logs -f circleci-runner-test-7d6b8fc6f-4z5wl

... (output truncated)

time="2021-03-25T20:55:40Z" level=info msg="CircleCI launcher starting" config=/opt/circleci/launch-agent-config.yaml
time="2021-03-25T20:55:40Z" level=info msg="loaded config" name=circleci-runner-test-7d6b8fc6f-4z5wl url="https://runner.circleci.com"
time="2021-03-25T20:55:40Z" level=info msg="no task found"

Chart Values

Customizable parameters are left inside the values.yaml file. See the following chart for information about each of the values:

Value Default Required? Description

Value	Default	Required?	Description
`image.repository` `image.tag`	`circleci/runner` `launch-agent`	Y	You can extend a custom Docker image from the CircleCI default self-hosted runner and use that instead. For CircleCI Enterprise installations, you can use the `image.tag` to set a pinned version of launch agent. See the Self-hosted Runner Installation page for the compatible version tags.
`replicaCount`	1	Y	The number of replicas of this self-hosted runner you want in your cluster. Must currently be set and updated manually. See Pending Work.
`resourceClass`	-	Y	The resource class you created for your self-hosted runner. You can choose to fill it in the chart here or to pass it directly when applying the chart, as shown above.
`runnerToken`	-	Y	The token you created for your self-hosted runner resource class. You can choose to fill it in the chart here or to pass it directly when applying the chart, as shown above.
`env`	-	N	Environment variables to set in the `launch-agent` pod. Including values for runner configuration
all other values	-	N	Modify at your own discretion and risk.

image.repository image.tag

circleci/runner launch-agent

You can extend a custom Docker image from the CircleCI default self-hosted runner and use that instead.

For CircleCI Enterprise installations, you can use the image.tag to set a pinned version of launch agent. See the Self-hosted Runner Installation page for the compatible version tags.

replicaCount

The number of replicas of this self-hosted runner you want in your cluster. Must currently be set and updated manually. See Pending Work.

resourceClass

The resource class you created for your self-hosted runner. You can choose to fill it in the chart here or to pass it directly when applying the chart, as shown above.

runnerToken

The token you created for your self-hosted runner resource class. You can choose to fill it in the chart here or to pass it directly when applying the chart, as shown above.

env

Environment variables to set in the launch-agent pod. Including values for runner configuration

all other values

Modify at your own discretion and risk.

CircleCI server installation

When installing the Helm chart for use with a CircleCI server installation, the image.tag will need to be set to the pinned launch agent version specified in the Self-hosted Runner Installation instructions. The LAUNCH_AGENT_API_URL will also need to be set as an environment variable. This can be done with the --set flag, or in the env section of the values.yaml file, specifying the hostname or address of the server installation.

Upgrading self-hosted runner deployment for server

Modify the values.yaml file to specify the new image.tag to update to. Refer to the Chart Values section of this document for more details about the values.yaml file.

Deploy the changes to the cluster

$ helm upgrade -f values.yaml "circleci-runner" ./ \
  --set runnerToken=$CIRCLECI_RUNNER_TOKEN \
  --set resourceClass=$CIRCLECI_RUNNER_RESOURCE_CLASS \
  --set env.LAUNCH_AGENT_API_URL=<server_host> \
  --namespace your-namespace

Further information about the $ helm upgrade command and its usage can be found in the helm documentation

Limitations and pending work

The Helm chart currently does not configure permissions, therefore, the containers will not have elevated privileges. If you need elevated permissions (e.g., for Docker in Docker work, etc.), you will need to modify the chart yourself.
The Helm chart currently does not support auto-scaling - you will need to modify the replicaCount parameter inside values.yaml yourself and re-apply the chart to your cluster.
The Helm chart currently expects only one self-hosted runner resource class type and one token. If you want to configure clusters for multiple self-hosted runner resource classes, you will need to set up separate charts for each one.

Help make this document better

This guide, as well as the rest of our docs, are open source and available on GitHub. We welcome your contributions.

Suggest an edit to this page (please read the contributing guide first).
To report a problem in the documentation, or to submit feedback and comments, please open an issue on GitHub.
CircleCI is always seeking ways to improve your experience with our platform. If you would like to share feedback, please join our research community.

Need support?

Our support engineers are available to help with service issues, billing, or account related questions, and can help troubleshoot build configurations. Contact our support engineers by opening a ticket.

You can also visit our support site to find support articles, community forums, and training resources.

CircleCI Documentation by CircleCI is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.