TutorialsLast Updated Nov 3, 20266 min read

Testing locally with CircleCI runners

Zan Markan

Developer Advocate

In this article you will learn how to set up the CircleCI runner agent on your local machine to run tests. You will also learn how to configure your CircleCI pipeline so that the runner is invoked correctly.

Many development teams start their CI/CD journey with a local build box (or six) that runs their tests. In several mobile teams I worked on, for example, we had a few Mac Mini boxes with physical devices plugged in that we used for running local UI and unit tests. Eventually we migrated to a cloud-based solution, which brought us much greater stability and many new features. But moving to the cloud also meant our local hardware was obsolete. We had to rely on emulation, or opt for another cloud-based device farm for that.

With CircleCI’s runners, you can keep using your own devices to run tests on, while benefiting from everything else available in the CircleCI cloud. You can also use runners for tests that need to run on your own infrastructure, for security-critical scenarios, or when you are developing your own hardware.

With the CircleCI Free plan, features like runners are available to everyone.

Prerequisites

This tutorial assumes some experience with CircleCI and an understanding of how pipelines are configured to build and test software projects.

I have created a sample project that shows the runner configuration.

Note: This example shows an Android application and tests, using the runner on macOS. The principles behind it can be universally applied, though: to iOS apps, custom hardware, or to runners on different infrastructure, like Docker containers, Windows, Linux machines, and more.

How runners work

A runner is a process installed on your infrastructure that communicates with CircleCI. Runners function as their own resource class. Your config.yml still defines the pipeline, jobs, and workflows, but instead of using CircleCI’s cloud-based resources like containers and virtual machines, you execute it on your own.

You can manage your runners from the Self-Hosted Runners page of the CircleCI web app, which you open by selecting Runners in the sidebar. The screenshots in this article come from a different organization, so they show the tutorial-gwp namespace:

The CircleCI Self-Hosted Runners page lists the organization's resource classes, including one with a connected machine runner, its version, and an idle state, along with the organization's concurrency usage.

First, install the runner agent locally. This is a daemon process that runs in the background, checking for prompts from CircleCI. When CircleCI signals that a job must be triggered on the runner, the agent executes the workload. It also has access to the same credentials as the cloud version of CircleCI, so it runs seamlessly and can fetch the code from your repositories.

Once the job is complete, the results and any artifacts are sent back to the cloud service. Only the actual job execution is done on your infrastructure.

Implementation

Implementation consists of these processes:

  • Setting up the runner agent locally
  • Invoking the runner from your CircleCI config.yml

Setting up the runner agent

Installing the runner agent depends on which platform you are using. The current version of the agent is machine runner 3. Refer to the setup instructions for your platform in the CircleCI docs.

For my sample application, I installed the runner agent on a Mac computer.

Note: These steps are just a brief overview and not a complete guide.

The main steps to set up the runner are:

  1. Register a new runner in your organization
  2. Install the runner agent, following the instructions for your platform
  3. Start the runner daemon, and make sure it runs on system startup

Use the circleci CLI tool to register a new runner in your organization. If it is your first runner, you may need to create a namespace in that organization. Each organization has only one namespace, shared with any orbs it publishes, so if your organization already publishes orbs, use that namespace. You will be referring to that specific runner from the CircleCI config using that fully qualified name. In my case, that was zmarkan-demos-namespace/local-mac-runner; local-mac-runner is the name of the runner, and zmarkan-demos-namespace is the namespace I used.

Here is a sample command:

$ circleci runner resource-class create zmarkan-demos-namespace/local-mac-runner --description "Local Mac OS Runner" --generate-token

This command will also set up a new API token for this runner to use with the CircleCI API in the organization. That is made possible because of the --generate-token flag. Be sure to store that token for later; CircleCI shows it only once. This command uses version 1 of the CircleCI CLI, which takes the description as a --description flag.

If you prefer the web app, you can create the namespace, resource class, and token from the Self-Hosted Runners page instead. The first time, an organization admin has to accept the CircleCI Runner Terms before Runners appears in the sidebar. Then select Create resource class, fill in the form, and select Save and continue to get the token:

The Create a custom resource class form in the CircleCI web app has a namespace, a resource class label, and a description filled in, and it warns that a namespace can only be created once for an organization.

After registering the runner with CircleCI, install the runner agent following the macOS instructions in the docs. On macOS, you install it with Homebrew by running brew tap circleci-public/circleci and then brew install circleci-runner. On Linux, you install the circleci-runner package. The setup also expects tools like curl, sha256sum, and tar to be available; on macOS, sha256sum comes from brew install coreutils.

Next, configure the runner by editing its config.yaml file and adding these values:

  • The API token created in the previous step (api.auth_token)
  • A name for this runner instance (runner.name)

The token already ties the runner to your namespace and resource class.

The particulars vary based on the platform you are using. For example, on Mac the Homebrew package installs a launchd agent that you start with launchctl, after you approve the notarized binary. On Linux it will be a new service that starts with systemctl.

Invoking the runner from a pipeline

Once the runner agent is in motion, you can invoke it from a job in the .circleci/config.yml file. Make sure to specify the machine executor type and the resource class with your runner’s namespace and runner name. Here is an example:

version: 2.1

jobs:
  runner-test:
    machine: true
    resource_class: zmarkan-demos-namespace/local-mac-runner
    environment:
      JAVA_HOME: /Library/Java/JavaVirtualMachines/temurin-17.jdk/Contents/Home
      ANDROID_HOME: /Users/<your-username>/Library/Android/sdk
    steps:
      - checkout
      - run:
          name: Run connected check on local runner
          command: |
            ./gradlew connectedDebugAndroidTest

workflows:
  test-on-runner:
    jobs:
      - runner-test

In my example, I had some Android devices connected to my Mac, so the runner could access that local hardware to run tests on them.

I also needed to pass in some specific values that come out of the box when using CircleCI’s own infrastructure, like JAVA_HOME and ANDROID_HOME. This is because they were not readily accessible in the specific shell that CircleCI runner uses. On most modern Macs the shell used is Zsh; CircleCI runs Bash by default. Replace these paths with the locations of the JDK and the Android SDK on your own machine. The sample project needs JDK 17 or later.

The rest of the steps are straightforward and will depend on your application and what you want to execute. In my case that was checking out the code from the repo and running the tests. You could also use store_artifacts and store_test_results to process the tests and save outputs. The outputs would then be available in your CircleCI cloud dashboard.

The job page also shows which runner ran the job. Expand the Task lifecycle step to see the runner’s name, version, and platform. The job in this screenshot ran a simplified version of the config on a runner in a Docker container on Linux, so a Mac runner shows different details:

A successful CircleCI job page shows a self-hosted resource class as the Executor, and its expanded Task lifecycle step lists the runner version, runner name, and a Linux platform.

Conclusion

In this article you have learned how to set up a local CircleCI runner for executing CircleCI cloud workloads on your local infrastructure. Runners have many use cases, from security to execution on bespoke hardware, all with the same CI/CD experience that cloud-based CircleCI offers.