Changelog

Before Upgrading

See the CircleCI server 4.2 release notes for upgrade notes for this release.

When upgrading from server v3.x you will need to install v4.0 before installing this version.

From server v4.2, CircleCI server auto-generates some secrets if they are not present in the Kubernetes namespace. If you are upgrading from v4.1.x to 4.2, and you have managed Kubernetes secrets yourself, you will need to follow the steps outlined in the CircleCI server 4.2 upgrade guide to ensure the system knows some secrets already exist.

What’s New in Release 4.2.0

New Features

• Config policies now available as a preview on server. Use config policies to create organization-level policies to impose rules and scopes to govern which configuration elements are required, allowed, not allowed etc. See the Config policies overview for guidance on setting up config policies. To use config policies CLI commands with server v4.2, append all commands shown in the docs with --policy-base-url https://<CIRCLECI-SERVER-DOMAIN>.com
• You can now find your current server version by running: curl http(s)://<CIRCLECI-SERVER-DOMAIN>/version
• The 2xlarge+ resource class (32 vCPUs, 64 GB RAM) is now available for use with the Linux VM execution environment

Changes

• Introduced the ability to disable the org-level concurrency limit in distributor. For more information, see the docs
• By default, Server 4.2 auto-generates some secrets. If you wish to manage these secrets yourself, you should create these secrets (if not done already) and ensure Helm is aware so that auto-generation is skipped. See the CircleCI server 4.2 upgrade guide for steps.
• RabbitMQ version upgraded from 8.6.4 to 11.15.1.

New Services

The following new services are introduced in this release.

• policy-service

Database Migrations

The following databases will run migrations when upgrading to this version

• builds_service
• cron_service_production
• domain
• insights
• permissions
• conductor_production

Fixes

• Patched various security vulnerabilities.

Known Issues

• Vault may not refresh its client token after a month of uptime.
• Retry with SSH for jobs using the machine executor advertises a private IP address. For this reason, retry with SSH for jobs using the machine executor works as standard for public installations, but for private installs you would need to ensure that you can access the private IP advertised. For example, by using a VPN into your VPC.
• CircleCI 1.0 builds are not supported. If an attempt is made to run a 1.0 build, no feedback will be available in the application to indicate the cause of the issue. If a build is run on your installation and does not show up in the CircleCI application, users should be directed to use the CircleCI CLI to validate the project configuration and get details of the possible cause of the issue.

To learn more about Server 4.2 installation, migration, or operations please see our documentation.

What’s New

The “Rerun failed tests” feature is now generally available. Instead of executing all tests in the testing job from a failed workflow (including tests that passed), only failed tests are executed again. Rerun failed tests helps shorten time-to-feedback when a workflow has flaky tests.

Customers who have adopted the feature are seeing a savings of around 8 minutes per job, and 60% reduction in runtime. Learn more about rerun failed tests in our official documentation.

What’s New

The quarterly update for Linux images is now available. Note that Ubuntu 20.04 and 22.04 are both available on amd64 and arm64 architecture under the edge tag, and are scheduled to be promoted to the GA release next week.

More details on this release, including software updates, can be found in the Linux 2023 Q3 release Discuss post.

What’s New

Image updates are available for the June 2023 snapshot of Windows Server 2022, as well as the current tag. This includes Windows security and OS updates, and package updates for Windows Server 2022.

For a full list of updated packages and image usage examples, visit the Discuss thread for the June 2023 image.

What’s New

CircleCI Personal and Project API access tokens have a new format. Previously, token values included a 40-character hex string. Newly created tokens will now include a CCIPAT or CCIPRJ prefix for personal access tokens or project access tokens, respectively:

CCIPAT_<base58-UUID>_<40-char-hex string>

For example: CCIPAT_ULUhR6rJLxYbyzyrP19iMZ_8fb1e3510325f7f09361de22b9420346c53ca2cb

Existing tokens are not impacted.

What’s New

Starting this week, we are introducing updates to the UI for steps in CircleCI:

• Vastly improved performance and rendering quality
• No cap on log output
• Terminal control codes are now processed in the UI
• URLs are clickable from the step output

More details are available in our Discuss post.

Before Upgrading

See the CircleCI server 4.1 release notes for upgrade notes for this release.

When upgrading from server v3.x you will need to install v4.0 before installing this version.

What’s New in Release 4.1.4

Changes

• Introduced the ability to disable the org-level concurrency limit in distributor. For more information, see the docs

Fixes

• Fixed a bug that caused potential notification failures for pipelines and workflows. The bug affected communication between the legacy-notifier and builds-service services.

Known Issues

• Vault may not refresh its client token after a month of uptime.
• Retry with SSH for jobs using the machine executor advertises a private IP address. For this reason, retry with SSH for jobs using the machine executor works as standard for public installations, but for private installs you would need to ensure that you can access the private IP advertised. For example, by using a VPN into your VPC.
• CircleCI 1.0 builds are not supported. If an attempt is made to run a 1.0 build, no feedback will be available in the application to indicate the cause of the issue. If a build is run on your installation and does not show up in the CircleCI application, users should be directed to use the CircleCI CLI to validate the project configuration and get details of the possible cause of the issue.

To learn more about Server 4.1 installation, migration, or operations please see our documentation.

What’s New

The “created at” date is now available for newly created project environment variables. This allows you to more easily identify which env vars have been recently updated, and change out those that are getting stale. This helps you implement best practices on secrets rotation within CircleCI.

Please note:

• The “created at” date is not available for existing project env vars. Existing env vars will display “older than {x} days” instead of a “created at” date.
• Env vars only support create and delete operations, so there is no “updated at” date associated with project env vars.

Before Upgrading

See the CircleCI server 4.1 release notes for upgrade notes for this release.

When upgrading from server v3.x you will need to install v4.0 before installing this version.

What’s New in Release 4.1.3

Fixes

• Query parameters are now supported on the task-agent download endpoint.

Known Issues

• Vault may not refresh its client token after a month of uptime.
• Retry with SSH for jobs using the machine executor advertises a private IP address. For this reason, retry with SSH for jobs using the machine executor works as standard for public installations, but for private installs you would need to ensure that you can access the private IP advertised. For example, by using a VPN into your VPC.
• CircleCI 1.0 builds are not supported. If an attempt is made to run a 1.0 build, no feedback will be available in the application to indicate the cause of the issue. If a build is run on your installation and does not show up in the CircleCI application, users should be directed to use the CircleCI CLI to validate the project configuration and get details of the possible cause of the issue.

To learn more about Server 4.1 installation, migration, or operations please see our documentation.

Before Upgrading

See the What’s new in server 3.x doc for upgrade notes for this release.

What’s New in Release 3.4.7

Fixes

• When using CircleCI runner, a repository can now be directly cloned with the checkout step in .circleci/config.yml.

Known Issues

• webhook-service is not operational when Vault is externalized.
• Vault may not refresh its client token after a month of uptime.
• Before upgrading to v3.4.x, delete the deployment named circleci-server-kube-state-metrics.
• Prometheus is broken in KOTS versions 1.65.0 - 1.67.0. If you rely on this feature, do not upgrade your KOTS version until this issue has been resolved.
• Retry with SSH for jobs using the machine executor advertises a private IP address. For this reason, retry with SSH for jobs using the machine executor works as standard for private installations, but for public installations you would need to ensure that you can access the private IP advertised (for example, by using a VPN into your VPC).
• CircleCI 1.0 builds are not supported. If an attempt is made to run a 1.0 build, no feedback will be available in the application to indicate the cause of the issue. If a build is run on your installation and does not show up in the CircleCI application, users should be directed to use the CircleCI CLI to validate the project configuration and get details of the possible cause of the issue.
• The KOTS admin console cannot be upgraded if your installation is set up to be behind a proxy. The proxy settings will be deleted and cause the KOTS admin console to break.

To learn more about Server 3.x installation, migration, or operations please see our documentation.

What’s New

The quarterly update across Linux images is now available. In addition to standard software updates, the following changes have been made:

• Heroku has been removed
• Java 11 has been removed and replaced by Java 17

For more details on this release, refer to the Discuss post.

What’s New

CircleCI’s CLI now includes an env subst command, which serves as a wrapper around the popular envsubst CLI tool. The command is used to substitute environment variables in a text string or file.

circleci env subst is available in both the local CLI and inside all CircleCI jobs (coming to CircleCI server v4.2+). More information and usage examples can be found in the docs as well as in this Discuss post.

Before Upgrading

See the CircleCI server 4.1 release notes for upgrade notes for this release.

When upgrading from server v3.x you will need to install v4.0 before installing this version.

What’s New in Release 4.1.2

Changes

• We have updated our PostgreSQL password requirements to allow special characters (except single or double quotes) in addition to alphanumeric characters.

Fixes

• For runner installations on server, you can now directly clone a repository with just the checkout step in .circleci/config.yml.

Known Issues

• Vault may not refresh its client token after a month of uptime.
• Retry with SSH for jobs using the machine executor advertises a private IP address. For this reason, retry with SSH for jobs using the machine executor works as standard for public installations, but for private installs you would need to ensure that you can access the private IP advertised. For example, by using a VPN into your VPC.
• CircleCI 1.0 builds are not supported. If an attempt is made to run a 1.0 build, no feedback will be available in the application to indicate the cause of the issue. If a build is run on your installation and does not show up in the CircleCI application, users should be directed to use the CircleCI CLI to validate the project configuration and get details of the possible cause of the issue.

To learn more about Server 4.1 installation, migration, or operations please see our documentation.

Before upgrading

See the CircleCI server 4.0 release notes for upgrade notes for this release.

What’s new in release 4.0.4

Changes

• We have updated our PostgreSQL password requirements to allow special characters (except single or double quotes) in addition to alphanumeric characters.

Fixes

• For runner installations on server, you can now directly clone a repository with just the checkout step in .circleci/config.yml.

Known issues

• When migrating from 3.x to 4.x, workflow configuration migrations will run as an init job and the workflows-conductor-event-consumer and the workflows-conductor-grpc-handler pods will not start until the migration completes. Customers with large PostgreSQL databases should plan accordingly. Migration time can be reduced by scaling the workflow-conductor pods.
• Vault may not refresh its client token after a month of uptime.
• Retry with SSH for jobs using the machine executor advertises a private IP address. For this reason, retry with SSH for jobs using the machine executor works as standard for public installations, but for private installs you would need to ensure that you can access the private IP advertised. For example, by using a VPN into your VPC.
• CircleCI 1.0 builds are not supported. If an attempt is made to run a 1.0 build, no feedback will be available in the application to indicate the cause of the issue. If a build is run on your installation and does not show up in the CircleCI application, users should be directed to use the CircleCI CLI to validate the project configuration and get details of the possible cause of the issue.

To learn more about Server 4.0 installation, migration, or operations please see our documentation.

What’s New

We now offer a way for you to immediately delete a DLC cache’s contents. You can do so via the CircleCI web UI, or the CLI. Refer to this support article for details.

When using Docker layer caching (DLC), a project’s cache can occasionally become corrupted, leading to failed jobs. Previously, the only solution was to stop using DLC, wait 3 days for the cache to become invalid, then turn DLC back on.

Before upgrading

See the CircleCI server 4.0 release notes for upgrade notes for this release.

What’s new in release 4.0.3

Changes

• In the checkout step in a CircleCI configuration, GitHub host keys are appended to ~/.ssh/known_hosts. Due to a change to GitHub’s RSA SSH key, CircleCI have updated the appended key to match. If you are using GitHub.com, and your installation of server has the old host key hardcoded into any projects, you will also need to update these to match.

Known issues

• When migrating from 3.x to 4.x, workflow configuration migrations will run as an init job and the workflows-conductor-event-consumer and the workflows-conductor-grpc-handler pods will not start until the migration completes. Customers with large PostgreSQL databases should plan accordingly. Migration time can be reduced by scaling the workflow-conductor pods.
• Vault may not refresh its client token after a month of uptime.
• Retry with SSH for jobs using the machine executor advertises a private IP address. For this reason, retry with SSH for jobs using the machine executor works as standard for public installations, but for private installs you would need to ensure that you can access the private IP advertised. For example, by using a VPN into your VPC.
• CircleCI 1.0 builds are not supported. If an attempt is made to run a 1.0 build, no feedback will be available in the application to indicate the cause of the issue. If a build is run on your installation and does not show up in the CircleCI application, users should be directed to use the CircleCI CLI to validate the project configuration and get details of the possible cause of the issue.

To learn more about Server 4.0 installation, migration, or operations please see our documentation.

Before Upgrading

See the What’s new in server 3.x doc for upgrade notes for this release.

What’s New in Release 3.4.6

Changes

• In the checkout step in a CircleCI configuration, GitHub host keys are appended to ~/.ssh/known_hosts. Due to a change to GitHub’s RSA SSH key, CircleCI have updated the appended key to match. If you are using GitHub.com, and your installation of server has the old host key hardcoded into any projects, you will also need to update these to match.

Known Issues

• webhook-service is not operational when Vault is externalized.
• Vault may not refresh its client token after a month of uptime.
• Before upgrading to v3.4.x, delete the deployment named circleci-server-kube-state-metrics.
• Prometheus is broken in KOTS versions 1.65.0 - 1.67.0. If you rely on this feature, do not upgrade your KOTS version until this issue has been resolved.
• Retry with SSH for jobs using the machine executor advertises a private IP address. For this reason, retry with SSH for jobs using the machine executor works as standard for private installations, but for public installations you would need to ensure that you can access the private IP advertised (for example, by using a VPN into your VPC).
• CircleCI 1.0 builds are not supported. If an attempt is made to run a 1.0 build, no feedback will be available in the application to indicate the cause of the issue. If a build is run on your installation and does not show up in the CircleCI application, users should be directed to use the CircleCI CLI to validate the project configuration and get details of the possible cause of the issue.
• The KOTS admin console cannot be upgraded if your installation is set up to be behind a proxy. The proxy settings will be deleted and cause the KOTS admin console to break.

To learn more about Server 3.x installation, migration, or operations please see our documentation.

Before Upgrading

See the CircleCI server 4.1 release notes for upgrade notes for this release.

When upgrading from server v3.x you will need to install v4.0 before installing this version.

What’s New in Release 4.1.1

Changes

• In the checkout step in a CircleCI configuration, GitHub host keys are appended to ~/.ssh/known_hosts. Due to a change to GitHub’s RSA SSH key, CircleCI have updated the appended key to match. If you are using GitHub.com, and your installation of server has the old host key hardcoded into any projects, you will also need to update these to match.

Known Issues

• Vault may not refresh its client token after a month of uptime.
• Retry with SSH for jobs using the machine executor advertises a private IP address. For this reason, retry with SSH for jobs using the machine executor works as standard for public installations, but for private installs you would need to ensure that you can access the private IP advertised. For example, by using a VPN into your VPC.
• CircleCI 1.0 builds are not supported. If an attempt is made to run a 1.0 build, no feedback will be available in the application to indicate the cause of the issue. If a build is run on your installation and does not show up in the CircleCI application, users should be directed to use the CircleCI CLI to validate the project configuration and get details of the possible cause of the issue.

To learn more about Server 4.1 installation, migration, or operations please see our documentation.

Before Upgrading

See the CircleCI server 4.1 release notes for upgrade notes for this release.

When upgrading from server v3.x you will need to install v4.0 before installing this version.

What’s New in Release 4.1.0

New features

• Air-gapped installation is now supported on server v4.1.
• Self-signed certificates or private certificate authorities are now supported when using GitHub Enterprise.
• For new installations, Tink can be used as the encryption backend instead of Vault.

Changes

• The minimum Telegraf version is now set to 1.8.14.
• Nomad and Vault images are now stored in templates, rather than hardcoded in charts.
• The Nomad Terraform configurations now have the Nomad port hardcoded. Previously this needed to be entered manually.

Database Migrations

The following databases will run migrations when upgrading this version.

• builds_service
• branch_service
• contexts_service_production
• orbs
• orb_analytics
• permissions
• webhook_service
• conductor_production

Fixes

• The Trigger Pipeline button in the web app now works for forked PRs, and an error message will be visible if the trigger fails.
• Fixed a bug that was preventing forked PRs from building if both the fork and original were building on CircleCI.
• Fixed a bug that was causing intermittent status updates for jobs in GitHub.

Known Issues

• Vault may not refresh its client token after a month of uptime.
• Retry with SSH for jobs using the machine executor advertises a private IP address. For this reason, retry with SSH for jobs using the machine executor works as standard for public installations, but for private installs you would need to ensure that you can access the private IP advertised. For example, by using a VPN into your VPC.
• CircleCI 1.0 builds are not supported. If an attempt is made to run a 1.0 build, no feedback will be available in the application to indicate the cause of the issue. If a build is run on your installation and does not show up in the CircleCI application, users should be directed to use the CircleCI CLI to validate the project configuration and get details of the possible cause of the issue.

To learn more about Server 4.x installation, migration, or operations please see our documentation.