Continuous integration with the Google Cloud Run orb

This tutorial will show you how to use the Google Cloud Run platform in a CI/CD pipeline. The pipeline will test the application’s code, build a Docker image, and deploy the image as a Google Cloud Run service on the Google Cloud Platform.

Prerequisites

This tutorial assumes a basic understanding of:

Add a project to CircleCI

The project used in this demo can be found in this repo. If you would like to follow along, fork the repo. Then, signup for a free CircleCI account if you don’t already have one. Connect the project to CircleCI by following the instructions for setting up your build on CircleCI.

Google Cloud setup

Let’s create the necessary credentials needed to interact with the Google Cloud Run platform. These credentials will give our CI/CD pipeline the access needed to execute commands on the Google Cloud Platform (GCP).

Create a GCP project

Create a new project in the GCP console. Give your project a memorable name. We want to make it readily identifiable so that it is easy to tear down later. After creating it, be sure to copy the project id as it is different from the project name.

How to find your project id.

Enable required APIs

Before creating service account credentials, you need to enable the Google Cloud APIs that your CI/CD pipeline will use. Go to the APIs & Services Library in your Google Cloud Console and enable the following APIs for your project:

Search for and enable the Cloud Run Admin API to allow your pipeline to deploy and manage Cloud Run services directly.
Enable the Google Container Registry API for pushing and pulling Docker images to Google’s container registry.

The Cloud Build API provides additional container building capabilities if you need them. You can find these APIs by searching for “Cloud Run Admin”, “Google Container Registry”, and “Cloud Build” respectively in the library search box. Then, click the Enable button for each.

Create a service account

Now you’ll create a service account that CircleCI can use to authenticate with Google Cloud on your behalf. A service account is a special type of Google account that belongs to your application rather than an individual user, making it perfect for automated deployments.

Go to the IAM & Admin Service Accounts section in your Google Cloud Console. Click Create Service Account and give it a descriptive name like circleci-deployment and a description like Service account for CircleCI automated deployments.

After creating the service account, you need to grant it the appropriate permissions. Click on the newly created service account, then go to the Permissions tab. The service account needs several roles to successfully deploy to Cloud Run and manage container images.

Add these roles to ensure comprehensive access for your CI/CD pipeline:

Artifact Registry Create-on-Push Repository Administrator - Allows automatic creation of repositories when pushing images
Artifact Registry Writer - Provides write access to push Docker images to Artifact Registry
Cloud Run Admin - Enables creating and managing Cloud Run services
Service Account User - Allows the service account to act on behalf of other service accounts when needed
Storage Admin - Provides access to push Docker images to Google Container Registry (legacy support)
Viewer - Grants general project access to read project metadata

These permissions ensure your pipeline can handle both Google Container Registry and the newer Artifact Registry, providing flexibility and future-proofing your deployment setup.

Generate and download credentials

With the service account created and properly configured, you now need to generate a JSON key file that CircleCI will use for authentication. In the service account details page, go to the Keys tab. Click Add Key, then select Create new key.

Choose JSON as the key type and click Create. This will download a JSON file containing your service account credentials. Save this file in a secure location on your local machine. Use a memorable name like circleci-gcp-credentials.json.

Important security note: The JSON file you just downloaded contains sensitive credentials that provide access to your Google Cloud project. Treat this file with extreme care and never commit it to version control. Anyone with access to this file can authenticate as your service account and potentially create resources that incur charges. Add the filename to your project’s .gitignore file immediately to prevent accidental exposure. Consider setting up file permissions to restrict access to this file on your local system.

CircleCI pipeline setup

Next, update your pipeline configuration file so that you can use the Google Cloud Run platform in your CI/CD pipeline.

Encoding the Google service account file

The service account file must be encoded into a base64 value in order to store this data as an environment variable on CircleCI. Go to the directory where you saved your service account JSON file. Replacing the sample filename with the one you created. Run this command:

base64 circleci-gcp-credentials.json

If you saved your file with a different name or in a different location, adjust the command accordingly. For example, if your file is in your Downloads folder:

base64 ~/Downloads/your-service-account-file.json

The results of this command will be a long string of base64-encoded text that looks similar to this:

ewogICJ0eXBlIjogInNlcnZpY2VfYWNjb3VudCIsCiAgInByb2plY3RfaWQiOiAiY2ljZC13b3Jrc2hvcHMiLAogICJwcml2YXRlX2tleV9pZCI6ICJiYTFmZDAwOThkNTE1ZTE0NzE3ZjE4NTVlOTY1NmViMTUwNDM4YTQ4IiwKICAicHJpdmF0ZV9rZXkiOiAiLS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0tXG5NSUlFdlFJQkFEQU5CZ2txaGtpRzl3MEJBUUVGQUFTQ0JLY3dnZ1NqQWdFQUFvSUJBUURjT1JuRXFla3F4WUlTXG5UcHFlYkxUbWdWT3VzQkY5NTE1YkhmYWNCVlcyZ2lYWjNQeFFBMFlhK2RrYjdOTFRXV1ZoRDZzcFFwWDBxY2l6XG5GdjFZekRJbXkxMCtHYnlUNWFNV2RjTWw3ZlI2TmhZay9FeXEwNlc3U0FhV0ZnSlJkZjU4U0xWcC8yS1pBbjZ6XG5BTVdHZjM5RWxSNlhDaENmZUNNWXorQmlZd29ya3Nob3BzLmlhbS5nc2VydmljZWFjY291bnQuY29tIgp9Cg==

Copy this entire base64 string into your clipboard. You’ll be using it later to create a CircleCI environment variable after you set up the pipeline configuration.

Google Cloud Run (fully managed) vs Google Cloud Run on GKE

Google Cloud Run allows you to run services on a fully managed environment or on a Google Kubernetes Engine (GKE) cluster. In this post, I’ll address running Google Cloud Run fully managed as well as running Google Cloud Run on a GKE cluster. In both cases, I’ll demonstrate how to integrate Google Cloud Run deployments into your CI/CD pipelines using the Google Cloud Run orb.

Creating a CI/CD pipeline with the Google Cloud Run (fully managed) service

Now that you have all of the elements required to use the Google Cloud Run platform in a CircleCI pipeline, update your project’s config.yml file with the following configuration syntax.

version: 2.1
orbs:
  gcp-gcr: circleci/gcp-gcr@0.16.1
  cloudrun: circleci/gcp-cloud-run@1.0.2
jobs:
  build_test:
    docker:
      - image: cimg/python:3.9
    steps:
      - checkout
      - run:
          name: Install Python Dependencies
          command: |
            pip install --user --upgrade pip
            pip install --user -r requirements.txt
      - run:
          name: Run Tests
          command: |
            pytest
  build_push_image_cloud_run_managed:
    docker:
      - image: cimg/python:3.9
    steps:
      - checkout
      - setup_remote_docker:
          docker_layer_caching: false
      - run:
          name: Install/Update Google Cloud SDK
          command: |
            # Remove any existing gcloud installation to avoid conflicts
            sudo rm -rf /home/circleci/google-cloud-sdk || true
            # Download and install Google Cloud SDK non-interactively
            cd /home/circleci
            curl -O https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-cli-458.0.1-linux-x86_64.tar.gz
            tar -xf google-cloud-cli-458.0.1-linux-x86_64.tar.gz
            ./google-cloud-sdk/install.sh --quiet --usage-reporting=false --path-update=true
            echo 'export PATH="$HOME/google-cloud-sdk/bin:$PATH"' >> $BASH_ENV
            source $BASH_ENV
      - run:
          name: Setup Google Cloud credentials and environment
          command: |
            # Decode the service account key.
            echo "${GCP_PROJECT_KEY}" | tr -d '[:space:]' | base64 --decode > $HOME/gcloud-service-key.json
            # Set environment variables for orbs and subsequent steps
            echo 'export GOOGLE_CLOUD_KEYS=$(cat $HOME/gcloud-service-key.json)' >> $BASH_ENV
            echo "export TAG=${CIRCLE_SHA1}" >> $BASH_ENV
            echo "export IMAGE_NAME=${CIRCLE_PROJECT_REPONAME}" >> $BASH_ENV
      - run:
          name: Build app binary and Docker image
          command: |
            # Load environment variables
            source $BASH_ENV
            pip install --user --upgrade pip
            pip install --user -r requirements.txt
            pyinstaller -F hello_world.py
            docker build -t us.gcr.io/$GOOGLE_PROJECT_ID/$IMAGE_NAME:latest -t us.gcr.io/$GOOGLE_PROJECT_ID/$IMAGE_NAME:$TAG .
      - run:
          name: Authenticate with Google Cloud
          command: |
            # Load environment variables
            source $BASH_ENV
            # Use the validated service account key
            gcloud auth activate-service-account --key-file=$HOME/gcloud-service-key.json
            gcloud config set project $GOOGLE_PROJECT_ID
            gcloud auth configure-docker --quiet
      - gcp-gcr/gcr-auth:
          gcloud-service-key: GOOGLE_CLOUD_KEYS
          google-project-id: GOOGLE_PROJECT_ID
          google-compute-zone: GOOGLE_COMPUTE_ZONE
      - gcp-gcr/push-image:
          google-project-id: GOOGLE_PROJECT_ID
          registry-url: "us.gcr.io"
          image: $IMAGE_NAME
      - cloudrun/deploy:
          platform: "managed"
          image: "us.gcr.io/$GOOGLE_PROJECT_ID/$IMAGE_NAME"
          service-name: "orb-gcp-cloud-run"
          region: $GOOGLE_COMPUTE_ZONE
          unauthenticated: true
workflows:
  build_test_deploy:
    jobs:
      - build_test
      - build_push_image_cloud_run_managed:
          requires:
            - build_test

Cloud Run (fully managed) config breakdown

The pipeline syntax implements the Google Cloud Run orb and deploys the application using the Google Cloud Run (fully managed) service.

version: 2.1
orbs:
  gcp-gcr: circleci/gcp-gcr@0.16.1
  cloudrun: circleci/gcp-cloud-run@1.0.2

The configuration starts by declaring CircleCI platform version 2.1 and importing two essential orbs. The gcp-gcr orb at version 0.16.1 handles Google Container Registry operations like authentication and image pushing, while the cloudrun orb manages Google Cloud Run deployments. Using current orb versions ensures you have access to the latest features and security improvements.

Job definitions

Our pipeline consists of two main jobs that work together to test and deploy our application. The configuration uses Python 3.9 and includes manual Google Cloud SDK installation to ensure compatibility and avoid Python module conflicts that can occur with newer Python versions.

Testing job

build_test:
  docker:
    - image: cimg/python:3.9
  steps:
    - checkout
    - run:
        name: Install Python Dependencies
        command: |
          pip install --user --upgrade pip
          pip install --user -r requirements.txt
    - run:
        name: Run Tests
        command: |
          pytest

The build_test job uses the stable cimg/python:3.9 image, which provides excellent compatibility with Google Cloud SDK and CircleCI orbs. Python 3.9 avoids compatibility issues while still following current best practices for dependency installation and testing.

Build and deployment job

The deployment job is more sophisticated and handles Google Cloud SDK installation, credential setup, application building, and Docker image creation. This approach ensures compatibility and reliability:

build_push_image_cloud_run_managed:
  docker:
    - image: cimg/python:3.9
  steps:
    - checkout
    - setup_remote_docker:
        docker_layer_caching: false
    - run:
        name: Install/Update Google Cloud SDK
        command: |
          # Remove any existing gcloud installation to avoid conflicts
          sudo rm -rf /home/circleci/google-cloud-sdk || true
          # Download and install Google Cloud SDK non-interactively
          cd /home/circleci
          curl -O https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-cli-458.0.1-linux-x86_64.tar.gz
          tar -xf google-cloud-cli-458.0.1-linux-x86_64.tar.gz
          ./google-cloud-sdk/install.sh --quiet --usage-reporting=false --path-update=true
          echo 'export PATH="$HOME/google-cloud-sdk/bin:$PATH"' >> $BASH_ENV
          source $BASH_ENV

The first step ensures we have a compatible Google Cloud SDK version by downloading and installing it fresh. This avoids Python compatibility issues that can occur with pre-installed versions.

- run:
    name: Setup Google Cloud credentials and environment
    command: |
      # Decode the service account key.
      echo "${GCP_PROJECT_KEY}" | tr -d '[:space:]' | base64 --decode > $HOME/gcloud-service-key.json
      # Set environment variables for orbs and subsequent steps
      echo 'export GOOGLE_CLOUD_KEYS=$(cat $HOME/gcloud-service-key.json)' >> $BASH_ENV
      echo "export TAG=${CIRCLE_SHA1}" >> $BASH_ENV
      echo "export IMAGE_NAME=${CIRCLE_PROJECT_REPONAME}" >> $BASH_ENV

The credential setup step carefully decodes the service account key and sets up environment variables that will persist across subsequent steps using $BASH_ENV.

- run:
    name: Build app binary and Docker image
    command: |
      # Load environment variables
      source $BASH_ENV
      pip install --user --upgrade pip
      pip install --user -r requirements.txt
      pyinstaller -F hello_world.py
      docker build -t us.gcr.io/$GOOGLE_PROJECT_ID/$IMAGE_NAME:latest -t us.gcr.io/$GOOGLE_PROJECT_ID/$IMAGE_NAME:$TAG .

The build step creates both the Python binary and Docker image with proper tagging for both latest and commit-specific versions.

- run:
    name: Authenticate with Google Cloud
    command: |
      # Load environment variables
      source $BASH_ENV
      # Use the validated service account key
      gcloud auth activate-service-account --key-file=$HOME/gcloud-service-key.json
      gcloud config set project $GOOGLE_PROJECT_ID
      gcloud auth configure-docker --quiet

The authentication step explicitly authenticates with Google Cloud using the fresh SDK installation and configures Docker authentication for pushing images.

This snippet uses the gcp-gcr/gcr-auth: orb to authenticate to Google Container Registry (GCR) using the environment variables set in the previous section:

- gcp-gcr/gcr-auth:
    gcloud-service-key: GOOGLE_CLOUD_KEYS
    google-project-id: GOOGLE_PROJECT_ID
    google-compute-zone: GOOGLE_COMPUTE_ZONE

In the next snippet, the push-image command is used to push the newly created image up to GCR for use within GCP:

- gcp-gcr/push-image:
    google-project-id: GOOGLE_PROJECT_ID
    registry-url: "us.gcr.io"
    image: $IMAGE_NAME

This snippet uses the deploy function from the cloudrun orb to create and deploy the Google Cloud Run service that will serve up the newly packaged Docker image. The platform parameter is set to managed which deploys the service to the fully managed environment on GCP:

- cloudrun/deploy:
    platform: "managed"
    image: "us.gcr.io/$GOOGLE_PROJECT_ID/$IMAGE_NAME"
    service-name: "orb-gcp-cloud-run"
    region: $GOOGLE_COMPUTE_ZONE
    unauthenticated: true

The next section will demonstrate how to deploy the Docker image to Google Cloud Run on GKE.

Creating a CI/CD pipeline with the Google Cloud Run service on GKE

The previous section demonstrated how to deploy the application to the fully managed Google Cloud Run environment. In this section, I’ll demonstrate how to deploy an application to Google Cloud Run on GKE. In the following pipeline example the build_test: job is identical to the previous fully managed Google Cloud Run example, but notice the new job named build_push_image_cloud_run_gke: which deploys an application to a Google Cloud Run service running on a GKE cluster via the Google Cloud Run orb.

version: 2.1
orbs:
  gcp-gcr: circleci/gcp-gcr@0.15.0
  cloudrun: circleci/gcp-cloud-run@1.0.2
jobs:
  build_test:
    docker:
      - image: cimg/python:3.11
    steps:
      - checkout
      - run:
          name: Install Python Dependencies
          command: |
            pip install --user --upgrade pip
            pip install --user -r requirements.txt
      - run:
          name: Run Tests
          command: |
            pytest
  build_push_image_cloud_run_gke:
    docker:
      - image: cimg/python:3.11
    steps:
      - checkout
      - setup_remote_docker:
          docker_layer_caching: false
      - run:
          name: Build app binary and Docker image
          command: |
            echo ${GCP_PROJECT_KEY} | base64 --decode --ignore-garbage > $HOME/gcloud-service-key.json
            export GOOGLE_CLOUD_KEYS=$(cat $HOME/gcloud-service-key.json)
            export TAG=${CIRCLE_SHA1}
            export IMAGE_NAME=$CIRCLE_PROJECT_REPONAME
            pip install --user --upgrade pip
            pip install --user -r requirements.txt
            pyinstaller -F hello_world.py
            docker build -t us.gcr.io/$GOOGLE_PROJECT_ID/$IMAGE_NAME -t us.gcr.io/$GOOGLE_PROJECT_ID/$IMAGE_NAME:$TAG .
      - gcp-gcr/gcr-auth:
          gcloud-service-key: GOOGLE_CLOUD_KEYS
          google-project-id: GOOGLE_PROJECT_ID
          google-compute-zone: GOOGLE_COMPUTE_ZONE
      - gcp-gcr/push-image:
          google-project-id: GOOGLE_PROJECT_ID
          registry-url: "us.gcr.io"
          image: $IMAGE_NAME
      - cloudrun/create_gke_cluster:
          cluster-name: $CIRCLE_PROJECT_REPONAME
          machine-type: "g1-small"
          zone: $GOOGLE_COMPUTE_ZONE
          enable-stackdriver-kubernetes: true
          scopes: "cloud-platform"
      - cloudrun/deploy:
          platform: "gke"
          image: "us.gcr.io/$GOOGLE_PROJECT_ID/$IMAGE_NAME"
          cluster: $CIRCLE_PROJECT_REPONAME
          service-name: $CIRCLE_PROJECT_REPONAME
          cluster-location: $GOOGLE_COMPUTE_ZONE
workflows:
  build_test_deploy:
    jobs:
      - build_test
      - build_push_image_cloud_run_gke:
          requires:
            - build_test

Cloud Run GKE Config Breakdown

The first job of the pipeline example shown above was covered in the fully managed section, so I’ll skip to the new job build_push_image_cloud_run_gke:.

The GKE deployment job follows the same Docker build pattern as the fully managed version, with streamlined environment variable handling and modern Python practices. The key difference lies in the deployment target and cluster management steps that follow.

GKE cluster creation and deployment

- cloudrun/create_gke_cluster:
    cluster-name: $CIRCLE_PROJECT_REPONAME
    machine-type: "g1-small"
    zone: $GOOGLE_COMPUTE_ZONE
    enable-stackdriver-kubernetes: true
    scopes: "cloud-platform"
- cloudrun/deploy:
    platform: "gke"
    image: "us.gcr.io/$GOOGLE_PROJECT_ID/$IMAGE_NAME"
    cluster: $CIRCLE_PROJECT_REPONAME
    service-name: $CIRCLE_PROJECT_REPONAME
    cluster-location: $GOOGLE_COMPUTE_ZONE

The GKE deployment involves two key steps. First, the create_gke_cluster command provisions a new Kubernetes cluster with Stackdriver monitoring enabled. Then, the deploy command targets the “gke” platform instead of “managed”, deploying your application as a Cloud Run service within the GKE cluster. This approach gives you more control over the underlying infrastructure while maintaining Cloud Run’s serverless benefits.

Note that GKE deployments typically take longer than fully managed deployments due to cluster provisioning time.

Testing the pipeline

Now that you have your pipeline configuration ready, it’s time to test it by pushing your code to GitHub and connecting it to CircleCI. This process will help you understand how the environment variables work in practice.

First, commit and push your updated configuration to your GitHub repository. If you’re following along with the forked repository, make sure your .circleci/config.yml file contains one of the pipeline configurations shown above.

Log into CircleCI and go to the Projects dashboard. Find your orb-gcp-cloud-run repository in the project list and click Set Up Project next to your repository.

CircleCI will detect the .circleci/config.yml file in your repository. Make sure to select the branch that contains your configuration file, then click Set Up Project to start the initial pipeline run.

CircleCI set up project

The pipeline will start immediately. The build_test job will complete successfully, but the build_push_image_cloud_run_managed job will fail because the necessary environment variables haven’t been configured yet. This is completely expected!

Failed pipeline

Don’t worry about this deployment job failure, it’s a normal part of the setup process. The failure occurs during the Docker build step when CircleCI tries to create an image tag like us.gcr.io/$GOOGLE_PROJECT_ID/$IMAGE_NAME but $GOOGLE_PROJECT_ID is empty, resulting in an invalid tag format. This demonstrates why the environment variables are crucial, without them, CircleCI can’t authenticate with Google Cloud Platform or access your project resources.

Create project variables

Now you’ll create the project-level environment variables that your CI/CD pipeline needs to execute commands on GCP. Go to your project settings in the CircleCI dashboard and add the following environment variables:

GOOGLE_PROJECT_ID should be set to your Google Cloud project ID. This value can be retrieved from the project dashboard in your Google Cloud Console, as shown in the earlier screenshot.
GCP_PROJECT_KEY should contain the base64 encoded string you generated from your service account JSON file in the previous section. Paste the entire base64 string here.
GOOGLE_COMPUTE_ZONE should specify the region where you want to deploy your Cloud Run service. Common values include us-central1, us-east1, or europe-west1.

After adding these environment variables, rerun the pipeline. This time, your pipeline should authenticate successfully with Google Cloud Platform and complete the deployment process.

Successful pipeline

Conclusion

In this tutorial, we’ve shown how CircleCI orbs streamline Google Cloud Run deployments while addressing the real-world challenges you’ll encounter along the way. You’ve learned that Python version compatibility and proper Google Cloud SDK installation are crucial for reliable deployments; why your configuration uses Python 3.9 and includes manual SDK setup steps.

The service account configuration requires comprehensive permissions spanning both legacy Google Container Registry and modern Artifact Registry to ensure your pipeline works seamlessly across Google’s evolving infrastructure. While orbs do simplify many deployment tasks, production-ready pipelines benefit from explicit credential handling and structured environment variable management.

Understanding that initial pipeline runs will fail due to missing environment variables is part of the learning process, and these expected failures actually help demonstrate why proper configuration is essential. Whether you’re deploying microservices, web applications, or batch processing jobs, the patterns we’ve established provide a robust foundation for automating your Google Cloud Run deployments with CircleCI.

Site

Blog