CircleCI Server v3.x VM Service

today • 4 min read

Server v3.x Server Admin

CircleCI server’s VM service controls how machine executor (Linux and Windows images) and Remote Docker jobs are run.

This section describes the available configuration options for VM service. These config options are all accessible from the KOTS Admin Console by choosing the Config tab from your dashboard.

VM service settings
VM provider
- AWS EC2
- Google Cloud Platform

We recommend that you leave these options at their defaults until you have successfully configured and verified the core and build services of your server installation. Steps to set up VM service are provided in the server 3.x installation guide for AWS and GCP.

VM service settings

You need to provide the hostname/IP for your VM service load balancer:

Once your server installation is up and running, run the following command:
```
kubectl get svc/vm-service
```
Enter the address listed under External IP in the VM Service Load Balancer Hostname field.

There is also an option to change the port used for VM service. The default is 3000 and this should only be changed if you are guided to do so by a CircleCI support engineer.

VM provider

The following configuration options are for the VM provider: either AWS or GCP.

AWS EC2

You will need to complete the following fields to configure your VM Service to work with AWS EC2.

At this point you can uncheck the Assign Public IPs check box if you need VMs to use private IP addresses.

AWS Region (required): This is the region in which the application is hosted.
AWS Linux AMI ID (optional): If you wish to provide a custom AMI for Linux machine executors, you can supply an AMI ID here. To create a Linux image, use the CircleCI Server Linux Image Builder. If you leave this field blank, a default AMI will be used.
AWS Windows AMI ID (optional): If you require Windows executors, you can supply an AMI ID for them here. To create a Windows image, use the CircleCI Server Windows Image Builder. Leave this field blank if you do not require Windows executors.
Subnets (required): Choose subnets (public or private) where the VMs should be deployed. Note that all subnets must be in the same availability zone.
Security Group ID (required): This is the security group that will be attached to the VMs.

The recommended security group configuration can be found in the Hardening Your Cluster section.

Number of <VM-type> VMs to keep prescaled: By default, this field is set to 0, which creates and provisions instances of a resource type on demand. You have the option of preallocating up to five instances per resource type. Preallocating instances lowers the start time, allowing for faster machine and remote_docker builds.

Preallocated instances are always running and could potentially increase costs. Decreasing this number may also take up to 24 hours for changes to take effect. You have the option of terminating those instances manually, if required.

If Docker Layer Caching (DLC) is used, VM Service instances need to be spun up on demand. For this to happen, either check any preallocated instances are in use, or set both remote Docker and machine preallocated instance fields to 0.

When using preallocated instances, be aware that a cron job is scheduled to cycle through these instances once per day to ensure they do not end up in an unworkable state.

Authentication

One of the following options is required. Either select "IAM Keys" and provide:

Access Key ID (required): Access Key ID for EC2 access.

The Access Key and Secret Key used by VM Service differs from the policy used by object storage in the previous section.
Secret Key (required): Secret Key for EC2 access.

Or select "IAM role" and provide:

Role ARN (required): Role ARN for Service Accounts (Amazon Resource Name) for EC2 access.

Encrypted EBS Volumes

Within the AWS EC2 Dashboard modify the Account Attributes to always encrypt new EBS volumes. Enabling this option will encrypt all new EBS volumes created by VM Service. Please note this option is per-region and needs to be enabled in the region running CircleCI server.

Default AWS AMI list

The default AMIs for server v3.x are based on Ubuntu 20.04.

"us-east-1" "ami-04f249339fa8afc90"
"ca-central-1" "ami-002f61fb4f6cd4f04"
"ap-south-1" "ami-0309e6438340ff3f5"
"ap-southeast-2" "ami-03ac956e1d298b76a"
"ap-southeast-1" "ami-0272b002478c96552"
"eu-central-1" "ami-07266a91e4ef7e3e8"
"eu-west-1" "ami-0bc8a965f9ae82e44"
"eu-west-2" "ami-0bcbed1cffe3866c2"
"sa-east-1" "ami-05291e231356c0387"
"us-east-2" "ami-08735066168c5c8e9"
"us-west-1" "ami-035e0e862838fcb21"
"us-west-2" "ami-0b4970c467d8baaef"
"ap-northeast-1" "ami-0b9233227f60abc2c"
"ap-northeast-2" "ami-08e7a9df6ab2f6b9d"
"eu-west-3" "ami-07f0d51c7621f0c39"
"us-gov-east-1" "ami-0f68718afd37587ae"
"us-gov-west-1" "ami-8e2106ef"

Google Cloud Platform

You need the following fields to configure your VM service to work with Google Cloud Platform (GCP).

We recommend you create a unique service account used exclusively by VM Service. The Compute Instance Admin (Beta) role is broad enough to allow VM Service to operate. If you wish to make permissions more granular, you can use the Compute Instance Admin (beta) role documentation as reference.

At this point you can uncheck the Assign Public IPs check box if you need VMs to use private IP addresses.

GCP project ID (required): Name of the GCP project the cluster resides.
GCP Zone for your VMs (required): GCP zone the virtual machines instances should be created in, for example us-east1-b.
GCP Windows Image (optional): If you require Windows executors, you can supply an AMI ID for them here. To create a Windows image, use the CircleCI Server Windows Image Builder. Leave this field blank if you do not require Windows executors.
GCP VPC Network (required): Name of the VPC Network. If you are deploying CircleCI server in a shared VPC, use the full network endpoint for the host network rather than the name, for example:
```
https://www.googleapis.com/compute/v1/projects/<host-project>/global/networks/<network-name>
```
GCP VPC Subnet (optional): Name of the VPC Subnet. If using auto-subnetting, leave this field blank. If you are deploying CircleCI server in a shared VPC, use the full network endpoint for the shared subnetwork rather than the name, for example:
```
https://www.googleapis.com/compute/v1/projects/<service-project>/regions/<your-region>/subnetworks/<subnetwork-name>
```
GCP IAM Access Type (required): One of the following is required. Either select GCP Service Account JSON file and provide:
- GCP Service Account JSON file (required): Copy and paste the contents of your service account JSON file if using the static GCP IAM service account credential.
  
  Or select IAM Workload Identity and provide:
- GCP IAM Workload Identity (required): Copy and paste the VM service account email address (service-account-name@project-id.iam.gserviceaccount.com ) which you have created here in point 2 & 3.

Number of <VM-type> VMs to keep prescaled: By default, this field is set to 0, which will create and provision instances of a resource type on demand. You have the option of preallocating up to 5 instances per resource type. Preallocating instances lowers the start time allowing for faster machine and remote_docker builds.

If Docker Layer Caching (DLC) is used, VM Service instances need to be spun up on demand. For this to happen, either ensure any preallocated instances are in use, or set both remote Docker and machine preallocated instance fields to 0.

When using preallocated instances be aware that a cron job is scheduled to cycle through these instances once per day to ensure they do not end up in an unworkable state.

Help make this document better

This guide, as well as the rest of our docs, are open source and available on GitHub. We welcome your contributions.

Suggest an edit to this page (please read the contributing guide first).
To report a problem in the documentation, or to submit feedback and comments, please open an issue on GitHub.
CircleCI is always seeking ways to improve your experience with our platform. If you would like to share feedback, please join our research community.

Need support?

Our support engineers are available to help with service issues, billing, or account related questions, and can help troubleshoot build configurations. Contact our support engineers by opening a ticket.

You can also visit our support site to find support articles, community forums, and training resources.

CircleCI Documentation by CircleCI is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.