Changelog

Before Upgrading

See the What’s new in server 3.x doc for upgrade notes for this release.

What’s New in Release 3.4.3

Changes

• Anonymous statistics reporting for Kong is now set to disabled by default.

Fixes

• Fixed a bug that was preventing email notifications from being sent with the following setting enabled: Notifications > Email Notifications> Build Notifications > My Branches.
• Fixed a bug that was causing a “Failed to load webhooks” error on project Webhooks pages in the CircleCI web app.

Known Issues

• webhook-service is not operational when Vault is externalized.
• Vault may not refresh its client token after a month of uptime.
• Before upgrading to v3.4.x delete the deployment named circleci-server-kube-state-metrics.
• Prometheus is broken in KOTS versions 1.65.0 - 1.67.0. If you rely on this feature, do not upgrade your KOTS version until this issue has been resolved.
• Retry with SSH for jobs using the machine executor advertises a private IP address. For this reason, retry with SSH for jobs using the machine executor works as standard for private installations, but for public installs you would need to ensure that you can access the private IP advertised. For example, by using a VPN into your VPC.
• CircleCI 1.0 builds are not supported. If an attempt is made to run a 1.0 build, no feedback will be available in the application to indicate the cause of the issue. If a build is run on your installation and does not show up in the CircleCI application, users should be directed to use the CircleCI CLI to validate the project configuration and get details of the possible cause of the issue.
• The KOTS admin console cannot be upgraded if your installation is set up to be behind a proxy. The proxy settings will be deleted and cause the KOTS admin console to break.

To learn more about Server 3.x installation, migration, or operations please see our documentation.

Before upgrading

See the CircleCI server 4.x release notes for upgrade notes for this release.

What’s new in release 4.0.1

Changes

• When migrating to server v4.x, Telegraf configuration data is now decoded and available in your values.yaml file.
• Anonymous statistics reporting for Kong is now set to disabled by default.
• TLS settings are now available for Postgres.

Fixes

• Fixed a bug that was preventing email notifications from being sent with the following setting enabled: Notifications > Email Notifications> Build Notifications > My Branches.
• Fixed a bug that prevented webhook-service from working when Vault is externalized.

Known issues

• When migrating from 3.x to 4.x, workflow configuration migrations will run as an init job and the workflows-conductor-event-consumer and the workflows-conductor-grpc-handler pods will not start until the migration completes. Customers with large PostgreSQL databases should plan accordingly. Migration time can be reduced by scaling the workflow-conductor pods.
• Vault may not refresh its client token after a month of uptime.
• Retry with SSH for jobs using the machine executor advertises a private IP address. For this reason, retry with SSH for jobs using the machine executor works as standard for public installations, but for private installs you would need to ensure that you can access the private IP advertised. For example, by using a VPN into your VPC.
• CircleCI 1.0 builds are not supported. If an attempt is made to run a 1.0 build, no feedback will be available in the application to indicate the cause of the issue. If a build is run on your installation and does not show up in the CircleCI application, users should be directed to use the CircleCI CLI to validate the project configuration and get details of the possible cause of the issue.

To learn more about Server 4.x installation, migration, or operations please see our documentation.

What’s New

Container agent is a new type of self-hosted runner in open preview. Container agent adds support for the Docker executor with self-hosted runners, as well as basic auto-scaling and integration with Kubernetes.

Read more in the Discuss post.

Refer to our documentation for more information.

What’s New

August updates are available for Android images. These updates include the removal of Android API levels 23 to 26. These API levels are no longer supported by Android and their removal helps eliminate bloat.

The latest updates can be accessed by using the image tag android:2022.08.1.

For more details, please refer to the August 2022 Update Discuss post.

CircleCI server v4.0.0 is now generally available. The feature set for this release are equal to server v3.x. The installation and upgrade processes for v4.x have changed significantly.

Server v4.x offers greatly improved security handling, installation, and update processes. Server v4.x is installed using helm charts and images that can be pulled ahead of time to comply with your security processes. Installation processes can also use artifacts pulled from customer-managed registries. Server v4.x makes improved use of Kubernetes secrets and removes the requirement to grant permissions and network access to third-party tools.

Before upgrading

See the CircleCI server 4.x release notes for upgrade notes for this release.

What’s new in release 4.0.0

New features

• Two new Arm resource classes are now available xlarge and 2xlarge. For more information on using Arm, see the Arm docs.
• CircleCI server v4.x images and helm charts are available for licenced users.

Fixes

• Fixed a bug that was causing a huge quantity of error logs in the distributer-internal pod.
• Prometheus is no longer included by default in CircleCI server v4.x.
• The default concurrency limit has been increased to avoid it being reached.
• Docker layer caching volumes are now deleted after 3 days if unused. Previously they were deleted after 7 days.
• kong is now placed behind NGINX to simplify loadbalancer setup.
• The loadbalancer has been renamed from circleci-traefik to circleci-proxy.
• Workflow configurations that are stored in PostgreSQL will migrated to the Object Storage.

Known issues

• Telegraf needs to have its configuration overwritten and base64 encoded.
• When migrating from 3.x to 4.x, workflow configuration migrations will run as an init job and the workflows-conductor-event-consumer and the workflows-conductor-grpc-handler pods will not start until the migration completes. Customers with large PostgreSQL databases should plan accordingly. Migration time can be reduced by scaling the workflow-conductor pods.
• Webhook-service may not work with an externalized Vault.
• Vault may not refresh its client token after a month of uptime.
• Retry with SSH for jobs using the machine executor advertises a private IP address. For this reason, retry with SSH for jobs using the machine executor works as standard for public installations, but for private installs you would need to ensure that you can access the private IP advertised. For example, by using a VPN into your VPC.
• CircleCI 1.0 builds are not supported. If an attempt is made to run a 1.0 build, no feedback will be available in the application to indicate the cause of the issue. If a build is run on your installation and does not show up in the CircleCI application, users should be directed to use the CircleCI CLI to validate the project configuration and get details of the possible cause of the issue.

To learn more about Server 4.x installation, migration, or operations please see our documentation.

What’s New

GitLab is now generally available for CircleCI. You are able to connect to GitLab and run pipelines from your GitLab repos. GitLab support also offers a new model for roles and permissions in CircleCI, independent of the VCS. This gives you full control over users and their access permissions.

We will continue to release GitLab-related features in Q3. These include project roles, restricted contexts, scheduled pipelines, and auto-cancel redundant workflows.

Each user can create up to three free organizations. A free organization can have up to 10 projects. If you need additional organizations or projects, consider upgrading to a paid plan, or contact our Support team.

What’s New

We have rolled out the Arm Large resource class as a standard offering in our Free plan.

To access this resource class, use the arm.large value with the resource_class key in your configuration. More details can be found in our documentation.

What’s New

This release includes updates to the image current tag for Windows Server 2019, Windows Server 2019 - Cuda, and Windows Server 2022.

For more detailed information about this release, see the Windows image update Discuss Post.

What’s New

CircleCI has updated our Android images for June. This release updates key software versioning for the Android Docker image cimg/android, in addition to the Android machine image android:2022.06. The Android machine image has a new naming scheme.

For more information on this update, and the new naming scheme, please see the Android Images Discuss post.

You may also directly access the GitHub repository for this Android update by going to the cimg repository.

Before Upgrading

See the What’s new in server 3.x doc for upgrade notes for this release.

What’s New in Release 3.4.2

Fixes

• Links and redirects within the CircleCI server web app now correctly send users to their server login page. Previously, these links were directing people to the CircleCI cloud login page.
• Improvements to the user experience when upgrading and migrating server versions.
• Root volumes created by VM service are now tagged with the custom tag set for instances in the KOTS admin console. This covers machine executor and remote Docker VMs.
• A bug was fixed that was presenting users who need to re-autorize with a 404.
• CircleCI cloud marketing items have been removed.

Known Issues

• Vault may not refresh its client token after a month of uptime.
• Before upgrading to v3.4.x delete the deployment named circleci-server-kube-state-metrics.
• Prometheus is broken in KOTS versions 1.65.0 - 1.67.0. If you rely on this feature, do not upgrade your KOTS version until this issue has been resolved.
• Retry with SSH for jobs using the machine executor advertises a private IP address. For this reason, retry with SSH for jobs using the machine executor works as standard for private installations, but for public installs you would need to ensure that you can access the private IP advertised. For example, by using a VPN into your VPC.
• CircleCI 1.0 builds are not supported. If an attempt is made to run a 1.0 build, no feedback will be available in the application to indicate the cause of the issue. If a build is run on your installation and does not show up in the CircleCI application, users should be directed to use the CircleCI CLI to validate the project configuration and get details of the possible cause of the issue.
• The KOTS admin console cannot be upgraded if your installation is set up to be behind a proxy. The proxy settings will be deleted and cause the KOTS admin console to break.

To learn more about Server 3.x installation, migration, or operations please see our documentation.

What’s New

The Q2 standard Linux images have been updated. A patch update was released for Heroku and Git fixes. The updated Ubuntu 20.04 and Ubuntu 22.04 images can be accessed with the following tags:

• ubuntu-2004:2022.04.2
• ubuntu-2204:2022.04.2

More details can be found in the 2022 April (Q2) Update Discuss post.

What’s New

The Windows Server 2019 edge images include OS updates and Windows Server 2022 edge image includes several updated packages (nvm, docker-engine) as well as OS updates.

For more information about this update, please see our most recent Windows image updates post on Discuss.

What’s New

The new Runner UI simplifies the configuration and installation of self-hosted runners, reducing the startup time from upwards of 40 minutes to 5 minutes or less.

Once you have completed installation, a new in-app inventory screen displays active self-hosted runners while providing you a level of visibility you previously did not have.

For more information, please see the Self-Hosted Runner Installation blog post.

What’s New

Today, CircleCI launched the CircleCI Public Roadmap, which showcases some of the product features and focus areas we are investing in. This product roadmap includes a set of features customers have expressed interest in and requested.

If you would like to provide feedback or suggestions on how we can improve the roadmap, please submit your request in the CircleCI Public Roadmap Ideas page.

What’s New

The orb source, orb info, and orb/config validate CLI commands have been updated to work successfully with private orbs.

While the orb source and orb info commands operate the same as they do for public orbs, the orb/config validate command is different.

For more information on how to use the updated orb/config validate tool, please refer to the Private Orbs technical documentation.

What’s New

CircleCI now supports using Test Splitting on jobs that run on self-hosted runners. With this release, Runner jobs can take advantage of CircleCI’s parallelism and Test Splitting features in a manner identical to how jobs that run on CircleCI-hosted compute use the features.

For more detailed information on Test Splitting for Runner, please see the Canny and CircleCI Discuss posts.

What’s New

In making recent updates to the IP ranges feature, a change was made which inadvertently added several new IPs and removed several existing IPs from the static list of IPs associated with CircleCI.

The static list of IPs on our docs page and the DNS records have been updated to reflect these new IPs.

Consumers of the DNS A record do not need to take any action. If you were relying on the static IP list, please update your records to the new list.

What’s New

A brand new convenience image focused on deployment, cimg/deploy, has been deployed in Beta. We are actively seeking feedback from users during this Beta period

For more information about this image, please refer to the announcement thread on Discuss.

You may also find this image in the CircleCI GitHub repo.

What’s New

CircleCI has updated our Android images for April. This release updates key software versioning for the Android Docker Image cimg/android, in addition to the Android Machine Image android:2022.04.1.

For more information on this update, please see the Android Images Discuss post.

You may also directly access the GitHub repository for this Android update by going to the cimg repository,

Before Upgrading

See the What’s new in server 3.x doc for upgrade notes for this release.

What’s New in Release 3.4.1

Fixes

• Kong resource limits have been increased. The previous limits were affecting the stability of some server installations.
• The KOTS config has been updated to allow private loadbalancers when ACM is in use.
• Fixed a bug that was preventing distributor-external from running in certain environments.

Known Issues

• Before upgrading to v3.4.x delete the deployment named circleci-server-kube-state-metrics.
• Prometheus is broken in KOTS versions 1.65.0 - 1.67.0. If you rely on this feature, do not upgrade your KOTS version until this issue has been resolved.
• Retry with SSH for jobs using the machine executor advertises a private IP address. For this reason, retry with SSH for jobs using the machine executor works as standard for private installations, but for public installs you would need to ensure that you can access the private IP advertised. For example, by using a VPN into your VPC.
• CircleCI 1.0 builds are not supported. If an attempt is made to run a 1.0 build, no feedback will be available in the application to indicate the cause of the issue. If a build is run on your installation and does not show up in the CircleCI application, users should be directed to use the CircleCI CLI to validate the project configuration and get details of the possible cause of the issue.
• The KOTS admin console cannot be upgraded if your installation is set up to be behind a proxy. The proxy settings will be deleted and cause the KOTS admin console to break.

To learn more about Server 3.x installation, migration, or operations please see our documentation.

What’s New

The Windows Server 2022 edge image has been updated to include nvm.portable 1.1.9 and docker engine 20.10.14.

For more information about this update, please see the Windows Server 2022 Update Dicuss Post.

What’s New

This feature allows you to set up preferred retention days across object types, while keeping storage costs predictable as CircleCI begins enabling plans to become eligible for overage charges. Beginning May 1, Performance and Free plans will be subject to overage charges, while on June 1, Scale and Custom plans will be subject to overages. Rolling out this UI feature today ensures you have 30-60 days until you will begin receiving overage charges.

As part of this rollout, CircleCI sent out communications to all organizations and added updated Network and Storage documentation that describes these changes.

What’s New

This update includes CircleCI’s first ‘current’ release of Windows Server 2022, in addition to regular updates for Windows Server 2019 and Windows Server 2019 - Cuda.

For more detailed information about this update, please see the Windows image update Dicuss Post.

Before Upgrading

See the What’s new in server 3.x doc for upgrade notes for this release.

What’s New in Release 3.4.0

New Features

• Android images for the machine executor are now available. For more information, see the Using Android Images with the Machine Executor guide.
• The CircleCI Build Agent can now be pulled from a custom Docker registry if required. For more information contact customer support.
• You can now use Amazon Certificate Manager (ACM) to manage your TLS certificates. For more information see the installation docs.
• Kubernetes v1.16 - v1.23 are now supported.
• Workload Identities for installations on GCP are now supported. Workload Identities can be used as an alternative to static credentials. For more informations see the installation docs

Fixes

• Fixed a bug that was causing the jobs page and step output access to run very slow in the CircleCI web app.
• Fixed a bug that was causing the workflows conductor pod to crash and issues with StatsD metrics.

Known Issues

• Before installing v3.4.0 delete the deployment named circleci-server-kube-state-metrics.
• Prometheus is broken in KOTS versions 1.65.0 - 1.67.0. If you rely on this feature, do not upgrade your KOTS version until this issue has been resolved.
• Retry with SSH for jobs using the machine executor advertises a private IP address. For this reason, retry with SSH for jobs using the machine executor works as standard for private installations, but for public installs you would need to ensure that you can access the private IP advertised. For example, by using a VPN into your VPC.
• CircleCI 1.0 builds are not supported. If an attempt is made to run a 1.0 build, no feedback will be available in the application to indicate the cause of the issue. If a build is run on your installation and does not show up in the CircleCI application, users should be directed to use the CircleCI CLI to validate the project configuration and get details of the possible cause of the issue.
• The KOTS admin console cannot be upgraded if your installation is set up to be behind a proxy. The proxy settings will be deleted and cause the KOTS admin console to break.
• Config files will now be stored in the chosen object store (e.g., AWS S3, Google Cloud Storage or Minio). Users can run into errors accessing certain pipelines, if these config files are deleted due to retention policies (e.g., AWS S3 Lifecycle).

To learn more about Server 3.x installation, migration, or operations please see our documentation.

What’s New in Release 2.19.15

Notes

In March 2022 updates to CircleCI server v2.x will stop.

Fixes

• Removed out-dated truststore for root CAs.
• Patched various security vulnerabilities.

Known Issues

• On static (non-AWS) installations with NONE selected under storage driver settings, tests can be split only by file name or file size. If a user attempts to split tests by timing data, static instances will split them by file name instead.
• If any changes have been made to your networking configuration from the default, you should run the following steps to ensure you can use SSH to inspect your builds (for further information on SSH reruns see the SSH Rerun Guide):
  • For customers using AWS, make sure that you have the latest Launch Configuration configured for Nomad clients, and that exiting Nomad clients were spun up using the Launch Configuration.
  • On each Nomad Client machine, create /etc/circleci/public-ipv4
  • This file should contain the public (if applicable) or private IP of the nomad client

What’s New

The Android Docker and Android VM images have been updated for March, including the inclusion of Android Tiramisu.

For more information about this update, please refer to the CircleCI Developer Hub page.

You may also find more information in the CircleCI Discuss post.

What’s New

On February 15, 2022, the Ubuntu 14.04 and 16.04 Linux VM images used with our machine executor were deprecated, meaning there will be no new releases or support for these images moving forward. When these images reach end of life (EOL) on May 31, 2022, any pipelines that utilize Ubuntu 14.04 or 16.04 images will fail.

In order to prevent unexpected failures and enable you to begin moving to newer images, CircleCI’s Ubuntu 14.04 and 16.04 Linux virtual machine (VM) images will be temporarily unavailable in late March 2022 and late April 2022, and will become permanently unavailable as of May 31, 2022. During these temporary “brown-out” periods, if you are currently using either of these images, be sure to update your config with a newer Ubuntu machine image before the 14.04 and 16.04 images are removed at the end of May 2022.

For more information about how CircleCI will manage these updates, please see the Deprecating Ubuntu 14.04 and 16.04 blog post.

What’s New

CircleCI now supports Windows Server 2022. CircleCI has released a new Windows Server 2022 image which enables customers using CircleCI’s Windows executors to take advantage of the latest operating system, IDE (Visual Studio 2022), and software from Microsoft.

At this time, the Windows Server 2022 image will co-exist with the Windows Server 2019 image allowing you to have the option to upgrade while not losing support to your existing image.

For more information about this update, please refer to the Discuss post that has details on how to adopt the image.

What’s New in Release 2.19.14

Notes

In March 2022 updates to CircleCI server v2.x will stop. Support for customers using CircleCI server v2.x will continue.

• There has been a change to how you will checkout using tags in jobs. Branch tags will now be referenced by either CIRCLE_TAG or +refs/heads/tag/path. For example, git checkout --force "$CIRCLE_TAG" or git fetch --force origin '+refs/heads/master:refs/remotes/origin/master'

Fixes

• Security patches for the exim image
• Replaced deprecated GitHub API endpoint for teams

Known Issues

• On static (non-AWS) installations with NONE selected under storage driver settings, tests can be split only by file name or file size. If a user attempts to split tests by timing data, static instances will split them by file name instead.
• If any changes have been made to your networking configuration from the default, you should run the following steps to ensure you can use SSH to inspect your builds (for further information on SSH reruns see the SSH Rerun Guide):
  • For customers using AWS, make sure that you have the latest Launch Configuration configured for Nomad clients, and that exiting Nomad clients were spun up using the Launch Configuration.
  • On each Nomad Client machine, create /etc/circleci/public-ipv4
  • This file should contain the public (if applicable) or private IP of the nomad client

What’s New

On March 15, 2022, GitHub will be changing which keys are supported in SSH and removing unencrypted Git protocol. As a result of this change, any image you have used in a job from a project created between November 2, 2021 and January 13, 2022 will require a version of OpenSSH greater than or equal to version 7.2.

CircleCI has updated the version of OpenSSH for supported machine images and next-generation Docker convenience images in anticipation of this breaking change.

To avoid pipeline failures on Tuesday, March 15, 2022, CircleCI recommends that if you are using deprecated Machine images or deprecated Docker convenience images, upgrade to a supported image. If upgrading an image is not possible, you can regenerate SSH keys for impacted projects to avoid pipeline failures on March 15, 2022.

For more information regarding this change and how to regenerate SSH keys, please refer to this support article.

Before Upgrading

See the What’s new in server 3.x doc for upgrade notes for this release.

What’s New in Release 3.3.1

Fixes

• CircleCI runner can now be used when server is installed behind a proxy.
• Fixed a bug that was causing some intermittent broken dependency caches.
• Removed outdated cert trust stores from some images. They were found to be causing access issues to some server installations.
• Fixed a bug that was affecting runner updates.
• SSH reruns now work when server is installed behind a proxy.
• Replaced deprecated GitHub API endoint for teams.

Known Issues

• Retry with SSH for jobs using the machine executor advertises a private IP address. For this reason, retry with SSH for jobs using the machine executor works as standard for private installations, but for public installs you would need to ensure that you can access the private IP advertised, for example, by using a VPN into your VPC.
• CircleCI 1.0 builds are not supported. If an attempt is made to run a 1.0 build, no feedback will be available in the application to indicate the cause of the issue. If a build is run on your installation and does not show up in the CircleCI application, users should be directed to use the CircleCI CLI to validate the project configuration and get details of the possible cause of the issue.
• The KOTS admin console cannot be upgraded if your installation is set up to be behind a proxy. The proxy settings will be deleted and cause the KOTS admin console to break.

To learn more about Server 3.x installation, migration, or operations please see our documentation.

What’s New

We’ve redesigned the project setup experience and made it easier for both new and existing users to set up new projects with CircleCI.

When you set up a new project, you can now quickly select an org and run through setup on a single page. We offer three different ways to set up your projects: use the .circleci/config.yml in your repository; commit a starter CI pipeline to a new branch; or go to a config.yml template that you can edit. We also indicate a time frame for each of the three methods (Fastest, Faster, and Fast, respectively), so you can choose the quickest project setup that also meets your needs.

What’s New

Network transfer information is now available for all jobs using a Docker executor (excluding remote Docker) on our Cloud platform. This feature can help you predict the cost of using the IP ranges feature as well as optimize IP ranges usage.

To view network transfer information, open the Resources tab within a given job’s details page in the CircleCI web app.

Read more about this update in our Discuss post.