How to use the CircleCI local CLI

today • 5 min read

Cloud Server v2.x Server v3.x

This page describes how to use some features of the CircleCI CLI.

Orb development kit

The orb development kit refers to a suite of tools that work together to simplify the orb development process, with automatic testing and deployment on CircleCI. There are two commands in the CLI that are a part of the orb development kit:

The following command initializes a new orb project:

circleci orb init

The following command packs an orb with local scripts:

circleci orb pack

For more information on orb packing, see the Orbs Concepts page.

Validate an orb in your configuration file

You can validate your orb with the following:

circleci orb validate /tmp/my_orb.yml

The above command will look for an orb called my_orb.yml in the /tmp folder of the directory in which you ran the command.

Packing a config

circleci config pack

This CLI pack command (separate to circleci orb pack described above) allows you to create a single YAML file from several separate files (based on directory structure and file contents). The pack command implements FYAML, a scheme for breaking YAML documents across files in a directory tree. This is particularly useful for breaking up source code for large orbs and allows custom organization of your orbs' YAML configuration.

How you name and organize your files when using the pack command will determine the final orb.yml output. Consider the following folder structure example:

$ tree
.
└── your-orb-source
    ├── @orb.yml
    ├── commands
    │   └── foo.yml
    └── jobs
        └── bar.yml

3 directories, 3 files

The unix tree command is great for printing out folder structures. In the example tree structure above, the pack command will map the folder names and file names to YAML keys, and map the file contents as the values to those keys.

The following command will pack up the example folder from above:

$ circleci config pack your-orb-source

And the output will be in your .yml file:

# Contents of @orb.yml appear here
commands:
  foo:
    # contents of foo.yml appear here
jobs:
  bar:
    # contents of bar.yml appear here

Other configuration packing capabilities

A file beginning with @ will have its contents merged into its parent folder level. This can be useful at the top level of an orb, when one might want generic orb.yml to contain metadata, but not to map into an orb key-value pair.

Thus:

$ cat foo/bar/@baz.yml
{baz: qux}

Is mapped to:

bar:
  baz: qux

Processing a config

Running the following command validates your config, but will also display expanded source configuration alongside your original configuration (useful if you are using orbs):

circleci config process

Consider the following example configuration that uses the node orb:

version: 2.1

orbs:
  node: circleci/node@4.7.0

workflows:
  version: 2
  example-workflow:
      jobs:
        - node/test

Running the following command will output a YAML file like the example below (which is a mix of the expanded source and the original configuration commented out):

circleci config process .circleci/config.yml

# Orb 'circleci/node@4.7.0' resolved to 'circleci/node@4.7.0'
version: 2
jobs:
  node/test:
    docker:
    - image: cimg/node:13.11.0
    steps:
    - checkout
    - run:
        command: |
          if [ ! -f "package.json" ]; then
            echo
            echo "---"
            echo "Unable to find your package.json file. Did you forget to set the app-dir parameter?"
            echo "---"
            echo
            echo "Current directory: $(pwd)"
            echo
            echo
            echo "List directory: "
            echo
            ls
            exit 1
          fi
        name: Checking for package.json
        working_directory: ~/project
    - run:
        command: |
          if [ -f "package-lock.json" ]; then
            echo "Found package-lock.json file, assuming lockfile"
            ln package-lock.json /tmp/node-project-lockfile
          elif [ -f "npm-shrinkwrap.json" ]; then
            echo "Found npm-shrinkwrap.json file, assuming lockfile"
            ln npm-shrinkwrap.json /tmp/node-project-lockfile
          elif [ -f "yarn.lock" ]; then
            echo "Found yarn.lock file, assuming lockfile"
            ln yarn.lock /tmp/node-project-lockfile
          fi
          ln package.json /tmp/node-project-package.json
        name: Determine lockfile
        working_directory: ~/project
    - restore_cache:
        keys:
        - node-deps-{{ arch }}-v1-{{ .Branch }}-{{ checksum "/tmp/node-project-package.json" }}-{{ checksum "/tmp/node-project-lockfile" }}
        - node-deps-{{ arch }}-v1-{{ .Branch }}-{{ checksum "/tmp/node-project-package.json" }}-
        - node-deps-{{ arch }}-v1-{{ .Branch }}-
    - run:
        command: "if [[ ! -z \"\" ]]; then\n  echo \"Running override package installation command:\"\n  \nelse\n  npm ci\nfi\n"
        name: Installing NPM packages
        working_directory: ~/project
    - save_cache:
        key: node-deps-{{ arch }}-v1-{{ .Branch }}-{{ checksum "/tmp/node-project-package.json" }}-{{ checksum "/tmp/node-project-lockfile" }}
        paths:
        - ~/.npm
    - run:
        command: npm run test
        name: Run NPM Tests
        working_directory: ~/project
workflows:
  version: 2
  example-workflow:
    jobs:
    - node/test

# Original config.yml file:
# version: 2.1
#
# orbs:
#   node: circleci/node@4.7.0
#
# workflows:
#   version: 2
#   example-workflow:
#       jobs:
#         - node/test

Run a job in a container on your machine

The CLI enables you to run jobs in your configuration with Docker. This can be useful to run tests before pushing configuration changes, or debugging your build process without impacting your build queue.

Prerequisites

You will need to have Docker installed on your system, as well as the most recent version of the CLI. You will also need to have a project with a valid .circleci/config.yml file in it.

Running a job

The CLI allows you to run a single job from CircleCI on your desktop using Docker with the following command:

$ circleci local execute --job JOB_NAME

If your CircleCI configuration is set to version 2.1 or greater, you must first export your configuration to process.yml, and specify it when executing with the following commands:

circleci config process .circleci/config.yml > process.yml
circleci local execute -c process.yml --job JOB_NAME

The following commands will run an example build on your local machine on one of CircleCI’s demo applications:

git clone https://github.com/CircleCI-Public/circleci-demo-go.git
cd circleci-demo-go
circleci local execute --job build

The commands above will run the entire build job (only jobs, not workflows, can be run locally). The CLI will use Docker to pull down the requirements for the build and then execute your CI steps locally. In this case, Golang and Postgres Docker images are pulled down, allowing the build to install dependencies, run the unit tests, test the service is running, and so on.

Limitations of running jobs locally

Although running jobs locally with circleci is very helpful, there are some limitations.

Machine executor

You cannot use the machine executor in local jobs. This is because the machine executor requires an extra VM to run its jobs.

Add SSH keys

It is currently not possible to add SSH keys using the add_ssh_keys CLI command.

Workflows

The CLI tool does not provide support for running workflows. By nature, workflows leverage running jobs concurrently on multiple machines allowing you to achieve faster, more complex builds. Because the CLI is only running on your machine, it can only run single jobs (which make up parts of a workflow).

Caching and online-only Commands

Caching is not currently supported in local jobs. When you have either a save_cache or restore_cache step in your config, circleci will skip them and display a warning.

Further, not all commands may work on your local machine as they do online. For example, the Golang build reference above runs a store_artifacts step, however, local builds will not upload artifacts. If a step is not available on a local build you will see an error in the console.

Environment variables

For security reasons, encrypted environment variables configured in the web application will not be imported into local builds. As an alternative, you can specify environment variables to the CLI with the -e flag. See the output of the following command for more information.

circleci help build

If you have multiple environment variables, you must use the flag for each variable, for example:

circleci build -e VAR1=FOO -e VAR2=BAR

Test splitting

The CircleCI CLI is also used for some advanced features during job runs, for example test splitting for build time optimization.

Context management

Contexts provide a mechanism for securing and sharing environment variables across projects. While contexts have been traditionally managed on the CircleCI web application, the CircleCI CLI provides an alternative method for managing the usage of contexts in your projects. With the CLI, you can execute several context-oriented commands:

create - Create a new context
delete - Delete the named context
list - List all contexts
remove-secret - Remove an environment variable from the named context
show - Show a context
store-secret - Store a new environment variable in the named context

The above list are "sub-commands" in the CLI, which would be executed like so:

circleci context create

# Returns the following:
List all contexts

Usage:
  circleci context list <vcs-type> <org-name> [flags]

Many commands will require that you include additional information as indicated by the parameters delimited by < >.

As with most of the CLI’s commands, you will need to have properly authenticated your version of the CLI with a token to enable performing context related actions.

Next steps

Introduction to Execution Environments

Help make this document better

This guide, as well as the rest of our docs, are open source and available on GitHub. We welcome your contributions.

Suggest an edit to this page (please read the contributing guide first).
To report a problem in the documentation, or to submit feedback and comments, please open an issue on GitHub.
CircleCI is always seeking ways to improve your experience with our platform. If you would like to share feedback, please join our research community.

Need support?

Our support engineers are available to help with service issues, billing, or account related questions, and can help troubleshoot build configurations. Contact our support engineers by opening a ticket.

You can also visit our support site to find support articles, community forums, and training resources.

CircleCI Documentation by CircleCI is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.