Using Shell Scripts
Cloud Server v3.x Server v2.x This document describes best practices for using shell scripts in your CircleCI configuration in the following sections:
Overview
Configuring CircleCI often requires writing shell scripts. While shell scripting can grant finer control over your build, it is a subtle art that can produce equally subtle errors. You can avoid many of these errors by reviewing the best practices explained below.
Shell script best practices
Use ShellCheck
ShellCheck is a shell script static analysis tool that gives warnings and suggestions for bash/sh shell scripts.
Use the Shellcheck orb for the simplest way to add shellcheck to your version: 2.1 configuration (remember to replace x.y.z with a valid version):
version: 2.1
orbs:
shellcheck: circleci/shellcheck@x.y.z
workflows:
check-build:
jobs:
- shellcheck/check # job defined within the orb so no further config necessary
- build-job:
requires:
- shellcheck/check # only run build-job once shellcheck has run
filters:
branches:
only: main # only run build-job on main branch
jobs:
build-job:
...
Alternatively, shell check can be configured without using the orb if you are using version 2 configuration:
version: 2
jobs:
shellcheck:
docker:
- image: koalaman/shellcheck-alpine:stable
auth:
username: mydockerhub-user
password: $DOCKERHUB_PASSWORD # context / project UI env-var reference
steps:
- checkout
- run:
name: Check Scripts
command: |
find . -type f -name '*.sh' | wc -l
find . -type f -name '*.sh' | xargs shellcheck --external-sources
build-job:
...
workflows:
version: 2
check-build:
jobs:
- shellcheck
- build-job:
requires:
- shellcheck # only run build-job once shellcheck has run
filters:
branches:
only: main # only run build-job on main branch
Note: Be careful when using set -o xtrace / set -x with ShellCheck. When the shell expands secret environment variables, they will be exposed in a not-so-secret way. In the example below, observe how the tmp.sh script file reveals too much.
> cat tmp.sh
#!/bin/sh
set -o nounset
set -o errexit
set -o xtrace
if [ -z "${SECRET_ENV_VAR:-}" ]; then
echo "You must set SECRET_ENV_VAR!"
fi
> sh tmp.sh
+ '[' -z '' ']'
+ echo 'You must set SECRET_ENV_VAR!'
You must set SECRET_ENV_VAR!
> SECRET_ENV_VAR='s3cr3t!' sh tmp.sh
+ '[' -z 's3cr3t!' ']'
Set Error Flags
There are several error flags you can set to automatically exit scripts when unfavorable conditions occur. As a best practice, add the following flags at the beginning of each script to protect yourself from tricky errors.
#!/usr/bin/env bash
# Exit script if you try to use an uninitialized variable.
set -o nounset
# Exit script if a statement returns a non-true return value.
set -o errexit
# Use the error status of the first failure, rather than that of the last item in a pipeline.
set -o pipefail
See also
For more detailed explanations and additional techniques, see this blog post on writing robust shell scripts.
Help make this document better
This guide, as well as the rest of our docs, are open source and available on GitHub. We welcome your contributions.
- Suggest an edit to this page (please read the contributing guide first).
- To report a problem in the documentation, or to submit feedback and comments, please open an issue on GitHub.
- CircleCI is always seeking ways to improve your experience with our platform. If you would like to share feedback, please join our research community.
Need support?
Our support engineers are available to help with service issues, billing, or account related questions, and can help troubleshoot build configurations. Contact our support engineers by opening a ticket.
You can also visit our support site to find support articles, community forums, and training resources.
CircleCI Documentation by CircleCI is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
