Changelog

What’s New

On December 7, 2021; CircleCI upgraded the underlying operating system used to run customer containers as part of the Docker executor on CircleCI Cloud. This change is critical to ensuring the underlying infrastructure that runs your jobs continues to provide reliable and performant execution.

The upgrade of the underlying OS and kernel resulted in improved spin-up performance for jobs that use the Docker executor. For more information on the performance improvements, please see the following Discuss post.

What’s New

We have released a new generation of macOS resources. These new resources can speed up builds by up to 50%, and are available to customers on Performance, Custom, and Scale plans.

For more information about these resources, please see the Discuss post.

For detailed technical documentaton about macOS resources, please see the macOS documentation.

What’s New in Release 2.19.13

Notes

In March 2022 updates to CircleCI server v2.x will stop. Support for customers using CircleCI server v2.x will continue.

Fixes

• Various security and vulnerability patches.

Known Issues

• On static (non-AWS) installations with NONE selected under storage driver settings, tests can be split only by file name or file size. If a user attempts to split tests by timing data, static instances will split them by file name instead.
• If any changes have been made to your networking configuration from the default, you should run the following steps to ensure you can use SSH to inspect your builds (for further information on SSH reruns see the SSH Rerun Guide):
  • For customers using AWS, make sure that you have the latest Launch Configuration configured for Nomad clients, and that exiting Nomad clients were spun up using the Launch Configuration.
  • On each Nomad Client machine, create /etc/circleci/public-ipv4
  • This file should contain the public (if applicable) or private IP of the nomad client

What’s New

SSH reruns have historically been run as independent jobs, rather than as part of a workflow. This makes those jobs difficult to track and manage, causing them to block customers with limited concurrency. We are now running SSH reruns as workflow reruns, so they will appear in the same way that “rerun from failed” workflows appear.

This will make reruns easier to manage and track since they will be connected to the pipelines and workflows they were in when originally run, both in the API and UI.

Note The CircleCI v2 API now supports enabling SSH for re-running workflows.

Before Upgrading

See the What’s new in server 3.x doc for upgrade notes for this release.

What’s New in Release 3.2.2

Notes

• The rerun workflow endpoint now returns workflow ID rather than the message accepted.

Fixes

• TLS is terminated outside of frontend so the SSL server has been completely removed from the frontend container.
• Moved the default certificate logic from KOTS to helm.
• Fixed the build agent image version used in server v3.x. The image used previously was causing problems with runner.

Known Issues

• Retry with SSH for jobs using the machine executor advertises a private IP address. For this reason, retry with SSH for jobs using the machine executor works as standard for private installations, but for public installs you would need to ensure that you can access the private IP advertised, for example, by using a VPN into your VPC.
• It is currently possible for multiple organizations under the same CircleCI server account to have contexts with identical names. This should be avoided as doing so could lead to errors and unexpected behavior.
• CircleCI 1.0 builds are not supported. If an attempt is made to run a 1.0 build, no feedback will be available in the application to indicate the cause of the issue. If a build is run on your installation and does not show up in the CircleCI application, users should be directed to use the CircleCI CLI to validate the project configuration and get details of the possible cause of the issue.
• The KOTS admin console cannot be upgraded if your installation is set up to be behind a proxy. The proxy settings will be deleted and cause the KOTS admin console to break.
• Runner cannot be used when server is installed behind a proxy.
• Let’s Encrypt certificate generation does not work. You will need to provide your own certificates or use the default certificates provided.

To learn more about Server 3.x installation, migration, or operations please see our documentation.

What’s New

You can now interact with private orbs via the CircleCI web app. A list of your organization’s orbs has been added on the Organization Settings page, and new private orb details pages that open in the orb registry have also been added as part of this release. The orbs list shows all public and private orbs authored by your organization / in your organization’s namespace(s), while the private orb details pages open in the Orbs registry and are flagged with a new “private” badge. Please note you will only be able to see these pages if you are logged in and authenticated; if you are not logged in, you will receive a 404 error.

With this update, you may select your preferred way to interact with orbs, which can be either the CircleCI CLI or the CircleCI App, while also being able to easily share private orb information with other users in your organization.

Note: This update will not add private orbs to the searchable Orb Registry, and un-published orbs will not have linked details pages.

What’s New

You can now review the CPU & memory utilization for docker jobs on CircleCI’s hosted cloud compute. A new Resources tab on the job details page displays this data and easy-to-consume graphs for docker jobs.

Early anecdotal evidence has shown that some customers that have used this page to upgrade their resource classes and noticed their jobs were completed quicker. Future updates will include enhancing the UX for highly parallelized jobs and adding a CTA for jobs for high CPU utilization.

What’s New

A new remote docker version (v20.10.11) has been released and is now available for use.

For more detailed information about the currently supported remote docker versions and how to run them, refer to the CircleCI docker technical documentation.

What’s New

CircleCI’s quarterly Linux machine images are now available. The following updated Ubuntu 20.04 images are available for both amd64 and arm64:

• ubuntu-2004:202111-01

This November refresh updates software included in the image, which can be found in this Discuss post.

Refer to the machine executor docs for more information about how to use the machine executor and these images.

What’s New

The test parallelism timing experience feature is now generally available and enables you to view information on parallel test times and bottlenecks so you can take this information and act on it to run your tests faster.

An example of what the timing tab looks like can be seen here.

What’s New

The new MySQL convenience image cimg/mysql is now in Beta. This image is designed to replace circleci/mysql which will soon be deprecated. We are actibely looking feedback from users during the Beta period.

For more information about this image, please refer to the Announcement thread on Discuss.

You may also find more information in the GitHub repo for this image.

What’s New

Scheduling work within CircleCI is an important way for you to run pipelines. In addition to periodic rebuilds of the application, scheduled pipelines can be useful for security scans and certain utility jobs such as restarting services.

While Scheduled Workflows may address some of these issues, there are a number of limitations. Scheduled pipelines will be more consistent with other methods of triggering pipelines (webhooks/APIs), which includes:

• consolidating common schedules
• resolving open issues in functionality for Scheduled Workflows (no dynamic configuratin, limitations in handling restricted contexts)
• increased reliability

For more information about scheduled pipelines, please refer to the blog post and technical documentation.

What’s New

This update enables you to view information on parallel test times and bottlenecks so you can take this information and act on it to run your tests faster.

An example of what the timing tab looks like can be seen here.

The timing tab has been enabled for 25% of users in Optimizely. The plan is to roll out this new feature to a small segment of users to get early feedback.

Note This feature is currently in “preview” mode and is not available to all customers at this time.

What’s New

Jobs that enabled IP ranges had a known limitation which caused them to receive TCP reset (RST) packets intermittently when uploading/downloading files larger than ~10 MB from a destination hosted by the CDN Fastly. CircleCI resolved this issue and is using a technical workaround.

• For more information about IP ranges, refer to the IP ranges technical documentation.
• For information about this update and technical workaround, refer to the Discuss post.

What’s New

CircleCI is upgrading the kernel version of the operating system used to run customer containers as part of the Docker executor on CircleCI. This change is critical to ensuring the underlying infrastructure that runs your jobs continues to provide reliable and performant execution.

CircleCI will be gradually rolling out this change for all Docker executor customers beginning on December 1, 2021.

This upgrade should be effectively invisible to customers. For more information on this update, refer to the following Discuss post.

What’s New

SSH reruns are now supported for Runner jobs. Although you have been able to SSH into cloud jobs to debug and inspect the system during and after the run, you may now use these same techniques for your Runner jobs.

For security and control purposes, the Runner launch agent requires a specific configuration to enable these SSH connections. Full details are available in the Runner documentation.

What’s New

The new MariaDB convenience image cimg/mariadb is now in Beta. This image is designed to replace circleci/mariadb which will soon be deprecated. During this Beta perios, we are looking for feedback.

For nore information about this image, please refer to the following post in Discuss

You may also find more information in the GitHub repo.

What’s New

The new Redis convenience image cimg/redis is now in Beta. This image is designed to replace circleci/redis which will soon be deprecated. During this Beta periodwWe are looking for feedback from users.

For more information about this image, please refer to the Announcement thread on Discuss.

You may also find more information about this image in the redis GitHub repo.

Before Upgrading

See the What’s new in server 3.x doc for upgrade notes for this release.

What’s New in Release 3.2.1

New Features

• Private VMs are now supported for installations on GCP.

Fixes

• mTLS is now disabled by default.
• SSH timeout for VMs has been increased to 10 minutes.
• Private VMs now request private IPs.

Known Issues

• Retry with SSH for jobs using the machine executor advertises a private IP address. For this reason, retry with SSH for jobs using the machine executor works as standard for private installations, but for public installs you would need to ensure that you can access the private IP advertised, for example, by using a VPN into your VPC.
• It is currently possible for multiple organizations under the same CircleCI server account to have contexts with identical names. This should be avoided as doing so could lead to errors and unexpected behavior.
• CircleCI 1.0 builds are not supported. If an attempt is made to run a 1.0 build, no feedback will be available in the application to indicate the cause of the issue. If a build is run on your installation and does not show up in the CircleCI application, users should be directed to use the CircleCI CLI to validate the project configuration and get details of the possible cause of the issue.
• The KOTS admin console cannot be upgraded if your installation is set up to be behind a proxy. The proxy settings will be deleted and cause the KOTS admin console to break.
• Runner cannot be used when server is installed behind a proxy.
• Let’s Encrypt certificate generation does not work. You will need to provide your own certificates or use the default certificates provided.

To learn more about Server 3.x installation, migration, or operations please see our documentation.

What’s New

Many customers have expressed concerns about the permissions required for an account, especially if they are collaborators on public projects and do not use CircleCI outside of viewing pipeline results. This makes CircleCI less desirable compared to alternatives like GitHub Actions.

With this release, if you go to the pipelines, workflows, or jobs page for a public project, you will be able to see a similar view compared to if you are logged in.

Note that most features, such as reruns or Insights, are not available unless you are already logging in.

What’s New

Many customers have been asking for an easy way to manage the orbs they’ve authored, both public or private. To meet this need, we are rolling out a set of orbs usability enhancements inside the Circle app.

The first feature enhancement is a new orbs list that is available in Organization Settings. You can now see all orbs created by your organization along with key stats such as version, type, and usage.

What’s New

Scheduling within CircleCI is an important way for you to run pipelines. In addition to periodic rebuilds of the application, schedules are useful for security scans and certain utility jobs such as restarting services.

While we currently have a solution, Scheduled Workflows, it has a number of limitations. Scheduled workflows will be more consistent with other methods of triggering pipelines (webhooks/APIs), consolidate common schedules, resolve some open issues in functionality for Scheduled Workflows (no dynamic config, limitations in handling restricted contexts), and will be more reliable.

More information about this update can be found here.

What’s New

CircleCI will be deprecating all legacy convenience images on December 31, 2021 and moving to supporting only next-gen convenience images. This change is being made to better serve our customers, while ensuring that these next-gen convenience images are consistently reliable. The next-gen CircleCI convenience images are designed from the ground up for an execution environment, and designed to be faster, more efficient, and most importantly, more reliable.

For more information about this upcoming change, please refer to the following Discuss post, which can be found here

What’s New

The new PostgreSQL convenience image (cimg/postgres) is now generally available. This image replaces the soon-to-be deprecated legacy image (circleci/postgres).

• Refer to the following CircleCI Discuss post for more information about this update, which can be found here.
• You may also find more information about this release in the following CircleCI GitHub repository, which can be found here.

What’s New

The new Android convenience image (cimg/android) is now generally available. This image replaces the soon-to-be deprecated legacy image (circleci/android).

Refer to the following resources for more detailed information about this update.

• Developer Hub page
• CircleCI Discuss post
• GitHub repository

Before Upgrading

See the What’s new in server 3.x doc for upgrade notes for this release.

What’s New in Release 3.2.0

New Features

• Customers who require a fully private installation can now access a setting in the KOTS admin console to ensure public IPs are not assigned to VMs. Note that with this non-public IP setting enabled, a work-around will be needed if SSH access to running jobs is required, for example, by using a VPN into your VPC.
• Customers that manage outbound traffic through a proxy can now configure proxy settings through the KOTS admin console. Please see our documentation for specifics on proxy support for server.
• We have expanded the machine execution environment options available to include additional resource classes, sizes, and executors. You now have access to Arm (medium, large), Linux (medium, large, X large, and XX large), and Windows (medium, large, XX large) resource classes.
• The insights API is now available to all server customers. Leverage build and other data to better understand the performance of teams and the health of your build and testing efforts.
• We have revamped the admin UI, and updated our installation instructions, making it easier to set up and manage server.
• You can now supply a custom Linux AMI for VM service.
• SSL termination can now be disabled - If you have put server login behind a firewall, this will enable SSL termination at the firewall.
• You can now control the size of persistent volumes. For larger customers, the initial persistent volume size was too small, by default. You can now set this at install time, providing an easier migration for those customers that require it. For further information see the Internal Database Volume Expansion doc.
• We have added an auto-scaling example to the nomad client terraform.
• You can now choose to serve ‘unsafe’ build artifacts. Previously this option was hidden, meaning potentially unsafe artifacts were rendered as plain text. For more information see the Build Artifacts doc.

Fixes

• The default windows executor was not as documented, we have increased the size to align with documentation and cloud.

Known Issues

• Retry with SSH for jobs using the machine executor advertises a private IP address. For this reason, retry with SSH for jobs using the machine executor works as standard for private installations, but for public installs you would need to ensure that you can access the private IP advertised, for example, by using a VPN into your VPC.
• It is currently possible for multiple organizations under the same CircleCI server account to have contexts with identical names. This should be avoided as doing so could lead to errors and unexpected behavior.
• CircleCI 1.0 builds are not supported. If an attempt is made to run a 1.0 build, no feedback will be available in the application to indicate the cause of the issue. If a build is run on your installation and does not show up in the CircleCI application, users should be directed to use the CircleCI CLI to validate the project configuration and get details of the possible cause of the issue.
• The KOTS admin console cannot be upgraded if your installation is set up to be behind a proxy. The proxy settings will be deleted and cause the KOTS admin console to break.
• Runner cannot be used when server is installed behind a proxy.
• Let’s Encrypt certificate generation does not work. You will need to provide your own certificates or use the default certificates provided.

To learn more about Server 3.x installation, migration, or operations please see our documentation.

What’s New

The Xcode 13 RC image is now available on CircleCI macOS resource classes. This image replaces Xcode 13 Beta 5. For further details, please see the Image Manifest.

Note: This image contains potentially breaking changes for NodeJS and Ruby 2.5 users upgrading from Xcode 12 images. For more information about these potential changes, refer to the following Discuss post.

What’s New

Webhooks are now available to all users. A webhook enables you to connect a platform you manage (either an API you create yourself, or a third party service) to a stream of future events.

Setting up a Webhook on CircleCI enables you to receive information (referred to as events) from CircleCI, as they happen. This can help you avoid polling the API or manually checking the CircleCI web application for desired information.

For more detailed information about webhooks, please see the Webhooks documentation, which can be found here.

Several new features have been released to improve the user experience when setting up configurations.

What’s New

Configuration sandbox

We have shipped a configuration sandbox so you can get immediate feedback while building your configuration. In this new experience, you can see the effect of your configuration change in a sandbox environment and offer corrections in the context of CircleCI configuration to get up and running faster.

Guide me - with orbs

We know users enjoy using orbs, therefore, we have made it easier to use and build a configuration with orbs. For new users, you will be able to walk through a series of steps that will take you through constructing a configuration using orbs.

Project setup for users w/existing configurations

We have made it easier for users who are ready to go and just want to build their projects on CircleCI. With this update, users now have the option to directly commit from any branch.

There have been a number of feature and user experience updates made to the CircleCI platform, which are described below.

What’s New

Feature updates

Status badge generator in project settings

Status badges are now available in Project Settings. This feature enables users to create a status badge more quickly for showing statuses on GitHub.

Add parameters to manual pipeline run

Users can now add one or more parameters when running a pipeline manually from the UI.

Iteration on copy step line link

We’ve made it easier for you to copy your code! When you hover over any part of a line in the step output, you’ll be able to see a blue copy icon.

UX updates

Follow project from branch drop down

If you happen to not have a project selected on CircleCI previously, you might have not been able to navigate to the page you’re looking for. Now when you select the drop-down, you can directly follow projects directly from the branch drop down and ensure you get to the project and page data you’re looking for.

Workflow status filter

Users may now filter to any combination of specific workflow statuses. Note that you may need to click the “See more” button if there are additional results.

What’s New

Xcode 13 Beta 4 image is now available on our macOS resource classes. This image replaces Xcode 13 Beta 2.

For more detailed information about this release, please see the Image Manifest.

Note: This image contains potentially breaking changes for NodeJS and Ruby 2.5 users. Please see the Release Notes for more details.

What’s New

The IP ranges feature is now available in open preview for all all customers on a Paid Plan. This feature was designed for customers who need to configure IP-based access to their restricted environments. As part of this feature, CircleCI provides a list of well-defined IP address ranges associated with the CircleCI service. CircleCI jobs that have this feature enabled will have their traffic routed through one of these defined IP address ranges.

Pricing will be calculated based on network data usage of jobs that have opted in to using the IP ranges feature. Although there is no charge while the feature is in open preview, pricing details will be shared soon and will take effect when the feature is generally available.

Note: IP ranges is currently only available for Docker jobs, not including remote_docker.

For more information about the IP ranges feature, please see the following Discuss post. You may also refer to the technical documentation for this feature, which can be found here.

What’s New

CircleCI has now made job and workflow webhooks feature available in open preview for all customers. “Job completed” and “Workflow completed” webhooks send status data, which may include states like success, failed, or canceled whenever a job or workflow reaches a terminal state.

For more detailed information about the webhooks feature, refer to the webhooks technical documentation, which can be found here.

If you would like to submit feedback on the webhooks feature, please submit your comments and feedback here.

What’s New

CircleCI’s quarterly Linux machine images are now available. The following updated Ubuntu 20.04 image is now available:

• ubuntu-2004:202107-01

This July refresh updates software included in the image. Refer to the machine executor docs for more information about how to use the machine executor and this image.

Please see the following Discuss post for more detailed information about these updates.

Before Upgrading

IMPORTANT: With this release, the frontend-external load balancer has been removed. The traefik load balancer now handles all incoming traffic. When updating from a previous server 3.x version, you will need to update the DNS record that was pointing to the frontend-external load balancer and have it point to the circleci-server-traefik load balancer instead. Remember, you can retrieve the external IP address or DNS name of your traefik load balancer by typing kubectl get svc/circleci-server-traefik in a terminal that has access to the cluster.

For further information see the What’s New doc.

What’s New in Release 3.1.0

New Features

• Telegraf plugins can now be added to server and customized to use third party monitoring solutions, for example, Datadog. For more information, see the Metrics and Monitoring doc.
• The option to use only private load balancers has been introduced for customers who want a fully private installation. For more information see the Load Balancers guide.
• Server 3.x hosts build artifacts, test results, and other state in object storage. We support any S3-compatible storage and Google Cloud Storage. For more information, see the Installation guide for further information.
• Dynamic config via setup workflows is now available on server installations. For more information see our blog post and the Dynamic Configuration docs page.
• Runner is now available on server. For further information, including installation steps, see the Runner docs. Runner allows the use of the macOS executor in server installations and VM service functionality for customers with server installed in a private data centre.
• The frontend load balancer from v3.0 has been removed and replaced with an Ingress resource and the Traefik Ingress controller. This is a breaking change requiring you to reconfigure your DNS. See the What’s New in server docs for further information and guidance.
• The following services can now be externalized. For setup information, see the server v3.x installation guide:
  • Postgres
  • MongoDB
  • Vault
• Backup and restore functionality is now available. For more information see the Backup and Restore guide.
• Prometheus is now deployed by default with server to monitor your cluster health and usage. Prometheus can be managed and configured from the KOTS admin UI. For further information, see the Metrics and Monitoring doc.
• Server now supports the 2XL resource class. The Nomad cluster needs to be made large enough to account for larger resource classes.
• The lifecycle of build artifacts and test results can now be configured from the KOTS admin console under Storage Object Expiry, including the option to disable the expiration and retain artifacts and test results indefinitely.

Fixes

• Resolved a collection of bugs that were causing sensitive information to be leaked into CircleCI support bundles:
  • Instances of faulty and partial redactions of secrets were detected, in part due to 3rd party bugs.
  • PostgresDB leaking sensitive information to STDOUT.
  • Several CircleCI services were logging secrets.
• Tightened network security in the Nomad terraform module.
• Terraform v0.15.0 and up are now supported.
• Updated installation scripts to use functions supported by most recent versions of Terraform.
• Resolved a bug that was leading to machine large builds being run on the wrong machine type. Machine large builds now correctly use 4 vCPUs and 16GB of RAM.
• Resolved a bug that caused contexts-service to fail on expiration of Vault client tokens.
• Resolved a bug that was causing legacy-notifier to report readiness prematurely.
• The JVM heap size parameter has been removed for all services. The heap size is set to be half of the memory limit.
• Changes to networking config and certs are now picked up automatically by Traefik. Previously, a restart would have been required.
• Minimum requirements for CPU and memory have changed. For the new values, see the Installation Prerequisites doc.

Known Issues

• Retry with SSH for jobs using the machine executor advertises a private IP address. For this reason, retry with SSH for jobs using the machine executor works as standard for private installations, but for public installs you would need to ensure that you can access the private IP advertised, for example, by using a VPN into your VPC.
• It is currently possible for multiple organizations under the same CircleCI server account to have contexts with identical names. This should be avoided as doing so could lead to errors and unexpected behavior.
• CircleCI 1.0 builds are not supported. If an attempt is made to run a 1.0 build, no feedback will be available in the application to indicate the cause of the issue. If a build is run on your installation and does not show up in the CircleCI application, users should be directed to use the CircleCI CLI to validate the project configuration and get details of the possible cause of the issue.

To learn more about Server 3.0 installation, migration, or operations please see our documentation.

What’s New

In order to keep up to date with changes to Xcode, the following updates have been made:

Xcode 12.5.1 is now available

Xcode 12.5.1 is now available on CircleCI macOS resource classes. This image supersedes Xcode 12.5.0 which will be removed, and job requests will now be re-routed to Xcode 12.5.1.

Xcode 13 Beta 1 is now available

Xcode 13 Beta 1 image is now available on our macOS resource classes. This image enables developers to build and test their apps against iOS 15, tvOS 15 and watchOS 8 betas.

Xcode 13 Beta 2 is now available

Xcode 13 Beta 2 image is now available on our macOS resource classes. This image replaces Xcode 13 Beta 1.

For more information on these macOS updates, refer to the CircleCI macOS Containers page.