Orb Authoring Process

today • 9 min read

Cloud Server v3.x

Introduction
Orb Development Kit
Create, test and publish an orb

Introduction

This orb authoring guide assumes you have read the Introduction to orbs document, the Introduction to authoring an orb document, and claimed your namespace. Now you are ready to develop an orb.

Whether you are writing your first orb or getting ready for production level, we recommend using our Orb Development Kit to get started. Alternatively, as orbs are packages of reusable configuration, they can be written manually, as singular yaml files, and published using our circleci orb cli.

Orb Development Kit

The Orb Development Kit refers to a suite of tools that work together to simplify the orb development process, with automatic testing and deployment on CircleCI. The orb init command is the key to using the Orb Development Kit. This command initiates a new orb project based on a template, and that template uses the other tools in the kit to automatically test and deploy your orb.

The Orb Development Kit is made up of the following components:

The orb template is a repository with CircleCi’s orb project template, which is automatically ingested and modified by the orb init command.

The CircleCI CLI contains two commands which are designed to work with the kit. The orb init command initializes a new orb project, and the orb pack command packs the orb source into a single orb.yml file.

The orb tools orb is an orb for creating orbs.

Create, test and publish an orb

Follow the steps below to create, test and publish your own orb, using the Orb Development Kit.

Getting started

To begin creating your new orb with the Orb Development Kit, follow the steps below. The starting point is creating a new repository on GitHub.com.

Ensure the organization on GitHub is the owner for the CircleCI namespace for which you are developing your orb. You can also view the video below learn more on getting started. If this is your own personal organization and namespace, you need not worry.

Create a new empty GitHub repository.

The name of your repository is not critical, but we recommend something similar to “myProject-orb”.

Note: Ensure that the repository is completely empty. Uncheck any options such as “Add a README.md” or “Choose a license”.

When complete, you will be brought to a page confirming your new repository and you should see the generated git URL. Note down the git URL, you will need it in step 4. You can select SSH or HTTPS, whichever you can authenticate with.

Note: While you must create a local directory for your orb before initializing, it is not necessary to pull down the orb repository. This process will be completed in the orb init process and pulling the repository beforehand will cause issues.
Update the CircleCI CLI

Ensure you are using the latest version of the CircleCI CLI. You must be using version v0.1.17087 of the CLI or later.
```
 $ circleci update

 $ circleci version
```
Initialize your orb

Open a terminal and initialize your new orb project using the orb init CLI command.

If you are using CircleCI server, you should ensure the --private flag is used here to keep your orbs private within your installation.

To initialize a public orb:
```
 circleci orb init /path/to/myProject-orb
```
To initialize a private orb:
```
 circleci orb init /path/to/myProject-orb --private
```
Once an orb is initialized, it cannot be switched from public to private or vice versa. Please make sure to add the --private flag if you intend to create a private orb.

The circleci orb init command is called, followed by a path that will be created and initialized for our orb project. It is best practice to use the same name for this directory and the git project repo.
Choose the fully automated orb setup option.
```
   ? Would you like to perform an automated setup of this orb?:
       ▸ Yes, walk me through the process.
   No, I'll handle everything myself.
```
When choosing the manual option, see Manual Orb Authoring Process for instructions on how to publish your orb.

When choosing the fully automated option, the Orb-Template will be downloaded and automatically modified with your customized settings. The project will be followed on CircleCI with an automated CI/CD pipeline included.

For more information on the included CI/CD pipeline, see the Orb Publishing Process documentation.

Alternatively, if you would simply like a convenient way of downloading the Orb-Template you can opt to handle everything yourself.
Follow the prompts to configure and set up your orb.

In the background, the orb init command will be copying and customizing the Orb Template based on your inputs. There are detailed README.md files within each directory that contain helpful information specific to the contents of each directory. You will also be asked for the remote git repository URL that you obtained back in step 1.

The Orb Template contains a full CI/CD pipeline (described in Orb Publishing Process) which automatically packs, tests, and publishes your orb.

In the setup process you will be asked if you would like to save your Personal API Token into an orb-publishing context. Saving this token is necessary for publishing development and production versions of your orb. If you have already made an orb in the past, you can skip this step, as the context will already exist.
Ensure the context is restricted

Restrict a context by navigating to Organization Settings > Contexts.

After completing your orb, you should see a new context called orb-publishing. Click into orb-publishing and add a Security Group. Use Security Groups to limit access to users that are allowed to trigger jobs. Only these users will have access to the private Personal API Token.

For more information, see the Contexts guide.
Push the changes up to Github.

During the setup process, the orb init command takes steps to prepare your automated orb development pipeline. The modified template code produced by the CLI must be pushed to the repository before the CLI can continue and automatically follow your project on circleci.com. Run the following command from a separate terminal when prompted to do so, substituting the name of your default branch:
```
 git push origin <default-branch>
```
Once complete, return to your terminal and confirm the changes have been pushed.
Complete the setup

Once the changes have been pushed, return to your terminal and continue the setup process. The CLI will now automatically follow the project on CircleCI, and attempt to trigger a pipeline to build and test your orb with sample code.

You will be provided with a link to the project building on CircleCI where you can view the full pipeline. You should also see the CLI has automatically migrated you into a new development branch, named alpha. You can use any branch naming you would like, you do not need to exclusively develop on alpha.
Enable Dynamic Configuration

Because we are making use of dynamic configuration, we must first enable this feature. You will receive an error on your first pipeline that will state that this feature is not yet enabled.

Following the Getting started with dynamic config in CircleCI, open the Project Settings page for your orb on CircleCI, navigate to the Advanced tab, and click on the Enable dynamic config using setup workflows button.

Once enabled, all future commits to your project will run through the full pipeline and test your orb. We could manually re-run the pipeline at this point, but since we are only working with sample code at this moment, we don’t need to.
Develop your orb

From a non-default branch (you will be moved to the alpha branch automatically at setup), begin modifying the sample orb code to your liking. On each push, your orb will be automatically built and tested.

Be sure to view the .circleci/test-deploy file to view how your orb components are being tested, and modify your tests as you change your orb. Learn more about testing your orb in the Orb Testing documentation.

When you are ready to deploy the first production version of your orb, find information on deploying changes in the Orb Publishing Process guide.

Writing your orb

Before you begin working on your orb, ensure you are on a non-default branch. We typically recommend starting your orb on the alpha branch.

$ git branch

* alpha
  main

If you have run the circleci orb init command, you will automatically be in the alpha branch and have a repository with .circleci and src directories.

Example: Orb Project Structure

type	name
	.circleci
	.github
	src
	.gitignore
	LICENSE
	README.md

Orb source

Navigate to the src directory to look at the included sections.

Example: Orb Project “src” Directory

type	name
	commands
	examples
	executors
	jobs
	scripts
	@orb.yml

The directories listed above represent orb components that can be included with your orb. @orb.yml acts as the root of your orb. In addition to the directories representing your orb’s yaml components, you will also see a ‘scripts/’ directory where we can store code we want to inject into our components.

Each directory within src corresponds with a reusable configuration component type, which can be added or removed from the orb. If, for example, your orb does not require any executors or jobs, these directories can be deleted.

@orb.yml

@orb.yml acts as the “root” to your orb project and contains the config version, the orb description, the display key, and imports any additional orbs if needed.

Use the display key to add clickable links to the orb registry for both your home_url (the home of the product or service), and source_url (the git repository URL).

version: 2.1

description: >
  Sample orb description

display:
  home_url: "https://www.website.com/docs"
  source_url: "https://www.github.com/EXAMPLE_ORG/EXAMPLE_PROJECT"

Commands

Author and add Reusable Commands to the src/commands directory. Each YAML file within this directory will be treated as an orb command, with a name which matches its filename.

This example shows a simple command which contains a single run step, which will echo “hello” and the value passed in the target parameter.

description: >
  # What will this command do?
  # Descriptions should be short, simple, and clear.
parameters:
  target:
    type: string
    default: "Hello"
    description: "To whom to greet?"
steps:
  - run:
      name: Hello World
      environment:
        ORB_PARAM_TARGET: << parameters.target >>
      command: echo "Hello ${ORB_PARAM_TARGET}"

Examples

Author and add Usage Examples to the src/examples directory. Usage Examples are not for use directly by end users in their project configs, but they provide a way for you, the orb developer, to share use-case specific examples on the Orb Registry for users to reference.

Each YAML file within this directory will be treated as an orb usage example, with a name which matches its filename.

View a full example from the Orb Template.

Executors

Author and add Parameterized Executors to the src/executors directory.

Each YAML file within this directory will be treated as an orb executor, with a name that matches its filename.

View a full example from the Orb Template.

Jobs

Author and add Parameterized Jobs to the src/jobs directory.

Each YAML file within this directory will be treated as an orb job, with a name that matches its filename.

Jobs can include orb commands and other steps to fully automate tasks with minimal user configuration.

View the hello.yml job example from the Orb Template.

description: >
  # What will this job do?
  # Descriptions should be short, simple, and clear.

docker:
  - image: cimg/base:current
parameters:
  greeting:
    type: string
    default: "Hello"
    description: "Select a proper greeting"
steps:
  - greet:
      greeting: "<< parameters.greeting >>"

Scripts

One of the major benefits of the Orb Development Kit is a script inclusion feature. When using the circleci orb pack command in the CLI (automated when using the Orb Development Kit), you can use the value <<include(file)>> within your orb config code, for any key, to include the file contents directly in the orb.

This is especially useful when writing complex orb commands, which might contain a lot of bash code, (although you could use python too!).

parameters:
  to:
    type: string
    default: "World"
    description: "Hello to whom?"
steps:
  - run:
      environment:
        PARAM_TO: <<parameters.to>>
      name: Hello Greeting
      command: <<include(scripts/greet.sh)>>

parameters:
  to:
    type: string
    default: "World"
    description: "Hello to whom?"
steps:
  - run:
      name: Hello Greeting
      command: echo "Hello <<parameters.to>>"

Why include scripts?

CircleCI configuration is written in YAML. Logical code such as bash can be encapsulated and executed on CircleCI through YAML, but, for developers, it is not convenient to write and test programmatic code within a non-executable format. Also, parameters can become cumbersome in more complex scripts as the <<parameter>> syntax is a CircleCI native YAML enhancement, and not something that can be interpreted and executed locally.

Using the Orb Development Kit and the <<include(file)>> syntax, you can import existing scripts into your orb, locally execute and test your orb scripts, and even utilize true testing frameworks for your code.

Using parameters with scripts

To keep your scripts portable and locally executable, it is best practice to expect a set of environment variables within your scripts and set them at the config level. The greet.sh file, which was included with the special <<include(file)>> syntax above in our greet.yml command file, looks like this:

echo Hello "${PARAM_TO}"

This way, you can both mock and test your scripts locally.

Testing orbs

Much like any software, to ensure quality updates, we must test our changes. There are a variety of methods and tools available for testing your orb from simple validation, to unit and integration testing.

In the .circleci/ directory created by the Orb Development Kit, you will find a config.yml file and a test-deploy.yml file. You will find in the config.yml file, the different static testing methods we apply to orbs, such as linting, shellchecking, reviewing, validating, and in some cases, unit testing. While, the test-deploy.yml config file is used to test a development version of the orb for integration testing.

Read our full Orb Testing Methodologies documentation.

Publishing your orb

With the Orb Development Kit, a fully automated CI and CD pipeline is automatically configured within .circleci/config.yml. This configuration makes it simple to automatically deploy semantically versioned releases of your orbs.

For more information, see the Orb Publishing Process guide.

Listing your orbs

List your available orbs using the CLI:

To list public orbs:

circleci orb list <my-namespace>

To list private orbs:

circleci orb list <my-namespace> --private

For more information on how to use the circleci orb command, see the CLI documentation.

Categorizing your orb

Orb categorization is not available on installations of CircleCI server.

You can categorize your orb for better discoverability in the Orb Registry. Categorized orbs are searchable by category in the Orb Registry. CircleCI may, from time to time, create or edit orb categorizations to improve orb discoverability.

Listing categories

Example of showing listing categories using the CLI

You can select up to two categories for your orb. These are the available categories:

Artifacts/Registry
Build
Cloud Platform
Code Analysis
Collaboration
Containers
Deployment
Infra Automation
Kubernetes
Language/Framework
Monitoring
Notifications
Reporting
Security
Testing

The list of categories can also be obtained by running the circleci orb list-categories CLI command. You can view the detailed docs for this command here.

Add an orb to a category

Adding an orb category

Add your orb to your chosen category by running circleci orb add-to-category <namespace>/<orb> "<category-name>". You can view the detailed docs for this command here.

Remove an orb from a category

Removing an orb from a category

Remove an orb from a category by running circleci orb remove-from-category <namespace>/<orb> "<category-name>". You can view the detailed docs for this command here.

Viewing an orb’s categorizations

Show which categorizations have been added to an orb

To see which categorizations have been applied an orb, check the output of circleci orb info <namespace>/<orb> for a list. You can view the detailed docs for this command here.

Help make this document better

This guide, as well as the rest of our docs, are open source and available on GitHub. We welcome your contributions.

Suggest an edit to this page (please read the contributing guide first).
To report a problem in the documentation, or to submit feedback and comments, please open an issue on GitHub.
CircleCI is always seeking ways to improve your experience with our platform. If you would like to share feedback, please join our research community.

Need support?

Our support engineers are available to help with service issues, billing, or account related questions, and can help troubleshoot build configurations. Contact our support engineers by opening a ticket.

You can also visit our support site to find support articles, community forums, and training resources.

CircleCI Documentation by CircleCI is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.