CircleCI Server v3.x Migration

today • 4 min read

Server v3.x Server Admin

Migrating from 2.19.x to 3.x requires you to back up your 2.19 instance data (Mongo, Postgres, and Vault) and then restore that data in a waiting Server 3.x instance. If you experience problems, you can fall back to your 2.19 instance. Migration requires an already operating Server 3.x installation.

Depending on the size of your data stores, the migration can take anywhere from a few minutes to a few hours. We recommend using a staging environment before completing this process in a production environment. This not only allows you to gain a better understanding of the migration process, but also gives you an idea of how long the migration will take to complete.

Prerequisites
- External Datastores Only
- Internal Datastore Only
Migration
Frequently Asked Questions
What to read next

Prerequisites

Your current CircleCI Server installation is 2.19.
You have taken a backup of the 2.19 instance. If you are using external datastores, they need to be backed up separately.
You have a new CircleCI Server 3.x installation.
You have successfully run realitycheck with contexts before starting.
The migration script must be run from a machine with:
- kubectl configured for the server 3.x instance
- ssh access to the 2.19 services box

External Datastores Only

Backups have been taken of all external data stores.
Postgres has been updated to version 12.

Internal Datastore Only

You have taken a backup of the 2.19 instance.
You have successfully run realitycheck on the new server 3.x instance with contexts prior to starting.

Migration

Migrating to server v3.x will shut down your v2.19 application. Your v2.19 application will not be restarted, although you may manually restart it using the KOTS Admin Console.

Starting the migration process will cause downtime. It is recommended you schedule a maintenance window.

Running server 2.19 and server 3.x at the same time can cause issues with your 2.19 build data. Server 2.19 should NOT be restarted if server 3.x is running.

Step 1 - Clone the repository and run the migration script

The instructions below will clone the repository containing the server v2.19.x to server v3.x migration script.

The migration script will:

Stop your v2.19.x application.
Perform preflight checks to confirm namespace and datastores for 2.19.x.
Create a tarball of your v2.19.x application’s PostgreSQL and Mongo databases.
Archive existing application data for Vault and CircleCI encryption/signing keys.
Export the 2.19.x tarball to your v3.x installation. Exported data stores are stored in a directory named circleci_export, located relative to wherever the migration script is run from. This can be useful for debugging purposes.
Perform preflight checks to confirm namespace and datastores for 3.x instance.
Scale v3.x application deployments down to zero.
Import the data from the previously exported tarball to your new v3.x instance.
Scale v3.x application deployments up to one.

If you have externalized services, you can run bash migrate.sh -v -p -m. These -v -p -m flags will skip the migration of Vault, Postgres, and Mongo, respectively. Skipping all three will copy your keys from /data/circle/circleci-encryption-keys on the v2.19.x services machine, allowing you to cat these files and upload their contents to the 3.x configuration page.

In a terminal:

Run git clone https://github.com/CircleCI-Public/server-scripts.
Change into the migrate directory: cd server-scripts/migrate.
Run the migration script: ./migrate.sh.
You will be prompted for the following information:
- Username of your server 2.19 installation
- Hostname of your server 2.19 installation
- The path to your SSH key file for your server 2.19.x installation
- Kubernetes namespace of your server 3.x installation
After the script has completed, the Signing and Encryption keys from the 2.19 instance will need to be added to the new 3.0 instance via the KOTS Admin Console. The keys will be located in circleci_export/circle-data.
The 3.x instance will either need to be updated to point at the same storage bucket that the 2.19 instance used, or the data needs to be copied over to a new bucket. The latter option ensures the 2.19 instance continues to work as expected, and so is the recommended approach if this migration is part of a test.

If a different hostname is being used in the 3.x environment, the GitHub webhooks will still be pointing to the hostname used in the 2.19 environment. The easiest way to update this is to click Stop Building and then Set Up Project. After doing this, the contexts and environment variables associated with the project will still be present.

Step 2 - Validate your migration to Server 3.0

Re-run realitycheck with contexts on your new server 3.x environment by pushing a fresh commit.

Step 3 - Update your team

Once you have successfully run realitycheck, notify your team of the new CircleCI UI and URL, if it has changed.

Frequently Asked Questions

Where did all my job and build history go?

All of your existing jobs and build history have been moved to the Legacy Jobs view. You can view the complete job history using one of the following methods:
- Selecting Projects → PROJECT_NAME and selecting the legacy jobs view link at the bottom of the project’s build history
- Using the following URL pattern: https://<APP_DOMAIN>/pipelines/github/<ORG>/<PROJECT>/jobs
- For a specific job, append a job number to the URL: https://<APP_DOMAIN>/pipelines/github/<ORG>/<PROJECT>/jobs/<JOB_NUMBER>

Why does nothing happen when I select "Start Building" on my project after migration?

By default, a newly added project (a project that has never been followed) triggers a build automatically after it has been followed for the first time. If the project was or ever has been followed in 2.0 or 3.0, it will not be considered a new project or first build and a build will not be triggered after a follow. To trigger a build, perform an activity that triggers a GitHub webhook such as pushing up a new commit or branch.

I got an error "Error from server (NotFound):"

The script assumes specific naming patterns for your Postgres and MongoDB. If you get this error, it may indicate a non-standard installation, a missing DB migration, or other issues. In this case, contact support with a support bundle and the output from the migration script.

What to read next

Help make this document better

This guide, as well as the rest of our docs, are open source and available on GitHub. We welcome your contributions.

Suggest an edit to this page (please read the contributing guide first).
To report a problem in the documentation, or to submit feedback and comments, please open an issue on GitHub.
CircleCI is always seeking ways to improve your experience with our platform. If you would like to share feedback, please join our research community.

Need support?

Our support engineers are available to help with service issues, billing, or account related questions, and can help troubleshoot build configurations. Contact our support engineers by opening a ticket.

You can also visit our support site to find support articles, community forums, and training resources.

CircleCI Documentation by CircleCI is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.