CircleCI Server v2.19.x What’s New

We now require a minimum 32GB of RAM for the Services Machine.

We made some changes to our Redis configuration in v2.18. If you have externalized Redis then you will need to update your configuration. Please contact your Customer Success Manager if you are upgrading from pre v2.18 to v2.19.

We have made changes to our Postgres version and require at least postgreSQL v9.5.16. If you have externalized postgreSQL then please update to at least that version in 2.17.x before upgrading to v2.19.

There has been a change to how you will checkout using tags in jobs. Branch tags will now be referenced by either CIRCLE_TAG or +refs/heads/tag/path. For example, git checkout --force "$CIRCLE_TAG" or git fetch --force origin '+refs/heads/master:refs/remotes/origin/master'

On static (non-AWS) installations with NONE selected under storage driver settings, tests can be split only by file name or file size. If a user attempts to split tests by timing data, static instances will split them by file name instead.

If you are using an HTTP proxy for your installation, upgrade to v2.19.02 or above. We had a known issue around job step logging and docker commands customization that made pre-2.19.2 releases incompatible with an HTTP proxy setup of CircleCI Server.

The first user to access a CircleCI Server installation should be granted admin access. Currently, there is an issue preventing this from happening when LDAP authentication is in use. Follow this guide for a workaround.

If any changes have been made to your networking configuration from the default, you may need to run the following steps to ensure you can use SSH to inspect your builds:

Classic Load Balancer is no longer available from v2.19.02 due to the ciphersuite changes listed in the notes below. CircleCI no longer accepts requests from Classic Load Balancer, so you should move to Network Load Balancer (NLB) or Application Load Balancer (ALB).

Removed out-dated truststore for root CAs.

Patched various security vulnerabilities.

Security patches for the exim image.

Replaced deprecated GitHub API endpoint for teams.

Various security and vulnerability patches.

Removed support for 1.0 builder graceful shutdown on auto-scaling events.

Fixed a bug that didn’t allow more than 50 aliases in a configuration.

Fixed a bug that caused 'git checkout' to fail in go-git.

Fixed a bug in the YAML parser in picard dispatcher to support config-str to have more than 50 YAML aliases.

Fixed a bug where s3 retention policy deleted test results from an older job.

Reduced the likelihood of 'out-of-sequence' commit statuses.

General security updates.

Better handling of non-alphanumeric characters in authentication passwords for Docker executors. Fixed a bug that was preventing the use of ~ and ? in the Docker ID password field under the auth key.

General security updates.

Fixed a bug that was causing parallelism to fail for static installations with NONE selected under storage driver settings.

Fixed a bug that was preventing authentication to China region ECR images in the docker executor.

Removed the recursive chown from startup of the fileserverd service. In instances of heavy usage this was causing the startup process to take a long time, or in some cases, startup was blocked.

Fixed a bug that was causing workflow statuses to be displayed on GitHub in the wrong order.

Introduced performance improvements to workflows-conductor that dramatically reduce CPU usage and latency.

Fixed a bug that was preventing use of an S3 storage region other than us-east-1 via an IAM user.

Fixed formatting type of SMTP passwords to ensure they are masked during setup.

Fixed a bug that could lead to the VM database ending up in an incorrect state in the event of a Services Machine crash or manual termination of VM Service instances.

Fixed a bug that was leading to support bundle creation timing out before the Services machine logs had been created, leaving only Replicated logs.

S3 connection pool metrics under circle.s3.connection_pool.* have been added to the test results service, making it easier to debug issues with this service.

Added in missing environment variables for the workflows service. The absence of these environment variables was causing excessive stack tracing whenever workflows were run. This in turn lead to excessive log rotation.

Fixed a bug that was causing failure to update GitHub statuses. Some customers experienced this bug when a project had a follower with a broken auth token.

Removed the use of the depecated GitHub.com API endpoint GET applications/%s/tokens/%s.

Distributed tracing is now enabled by default for Server installations. Traces are used in CircleCI support bundles to improve our ability to troubleshoot Server issues. Options for the tracing sampling rate are displayed in the Replicated Management Console, but should only be changed from the default if requested by CircleCI Support.

Fixed an issue that was preventing restore_cache from working with the storage driver set to "none" - i.e not S3.

Fixed an issue that was preventing the output_processor service from using AWS AssumeRole when the role was located in a subfolder. This issue affected customers with security policies forcing the use of a subfolder in this case, and the symptoms included the inability to store artifacts or use timings-based test splitting.

JVM heap size can now be changed using the JVM_HEAP_SIZE environment variable for the following services: vm-service, domain-service, permissions-service and federations-service.

In the LDAP login flow we now use an anonymous form to POST LDAP auth state, rather than sending it as a GET parameter. Previously, when a user authenticated using LDAP, their username and password were sent in plaintext as part of a query parameter in a GET request. As requests are over HTTPS, this left usernames and passwords in request logs, etc. This issue is now fixed.

Optimizely and Zendesk are now removed from Server release images.

Fixed an issue in which setting CIRCLE_ADMIN_SERVER_HTTP_THREADS or CIRCLE_PUBLIC_FACING_SERVER_HTTP_THREADS too high would prevent the frontend container from starting.

Due to changes in the GitHub API we have removed the use of ?client_id=x&client_secret=y for GitHub, and GHE versions 2.17 and later.

Fixed an issue that was causing intermittent failures to spin up VMs with DLC in use.

Fixed an issue that was preventing the customization of proxy settings for Docker containers. See the Nomad Client Proxy and Service Configuration Overrides guides for more infomation.

Fixed a bug that was preventing job steps for non-failing builds being logged when proxy settings were used for the job container.

Removed legacy TLS versions 1.0 and 1.1, in addition, enabled 1.2 and 1.3 TLS, and specified the following ciphersuites

Fixed a statsd configuration issue that meant some services were not emitting Telegraf metrics.

Fixed a bug that was preventing some customers from upgrading due to a schema change in one of our library dependencies.

Fixed a bug that was preventing some customers from inspecting builds via SSH due to a logic change in our build agent.

You can now customize resource classes for your installation to provide developers with CPU/RAM options for the Jobs they configure. For more information see our guide to customizing resource classes in server v2.19.

CircleCI Server installations on AWS can now be configured to work on GovCloud.

The image used to run the RabbitMQ server has been updated to fix vulnerabilities.