TutorialsLast Updated Nov 3, 202611 min read

How do I use Infrastructure as Code?

Angel Rivera

Developer Advocate, CircleCI

Infrastructure as code (IaC) is the process of managing and provisioning cloud and IT resources via machine-readable definition files and is a part of modern continuous integration pipelines. IaC enables organizations to provision, manage, and destroy compute resources using modern DevOps tools. IaC enables this by statically defining and declaring these resources in code, then deploying and dynamically maintaining these resources via code.

In recent interactions with developers, I’ve realized that many are either not aware of IaC, or are aware but have abandoned their initial efforts because ramping up was too cumbersome. In this post, I will discuss the benefits of IaC and some of the tooling options currently available, along with a brief demonstration of how to use them, in hopes of easing the transition to using IaC. I will focus on two of the most widely used IaC tools: HashiCorp’s Terraform and Pulumi.

Prerequisites

Before you get started, you’ll need to have these things:

The virtual machine (VM) in this tutorial is an e2-micro in the us-central1 region with a 10 GB standard disk, which fits within the Compute Engine Free Tier. New Google Cloud customers also get a $300 credit to use over 90 days.

To enable the Compute Engine API for your project, run this command, replacing <your-project-id> with your project ID:

gcloud services enable compute.googleapis.com --project <your-project-id>

Sign in with Application Default Credentials

Terraform and Pulumi need Google Cloud credentials to create resources in your project. For local development, Google recommends Application Default Credentials (ADC) instead of downloading service account keys, and many organizations now block service account key creation by default.

To create your ADC file, run this command and sign in with your Google account in the browser window that opens:

gcloud auth application-default login

This saves your credentials to ~/.config/gcloud/application_default_credentials.json. You’ll share that directory with the IaC101 container later, so that Terraform and Pulumi can use it.

Build the IaC101 Docker image

The IaC101 Docker image has the Terraform and Pulumi CLI tools installed, plus example code for each tool. You’ll build it from the sample repository so that it runs natively on your machine, whether it has an Intel or Apple silicon processor.

In a terminal, clone the repository and build the image:

git clone https://github.com/CIRCLECI-GWP/infrastructure-as-code101.git
cd infrastructure-as-code101
docker build -t iac101 .

You can verify the build by running docker images. You should see the iac101 image listed in the results.

Run the IaC101 container

Now you’ll create a new Docker container from the iac101 image. The command mounts your local ~/.config/gcloud/ directory into the container as read-only, so that the IaC tools can find your ADC file. It also sets GOOGLE_CLOUD_PROJECT, which both Terraform and Pulumi read to determine which project to create resources in.

Run this command, replacing <your-project-id> with your project ID:

docker run -it --name iac101 \
  -v "$HOME/.config/gcloud:/root/.config/gcloud:ro" \
  -e GOOGLE_CLOUD_PROJECT=<your-project-id> \
  iac101

On Windows, the gcloud configuration directory is %APPDATA%\gcloud instead of ~/.config/gcloud.

IaC101 container is running

After running the previous docker run command, your IaC101 container is up and running, and your terminal shell drops you into the running container. Every command you run in your terminal shell will be executed in the Docker container from now on until you manually exit the container. The container’s project files are mapped as follows:

project/
|_ terraform/gcp/compute/   # Contains the Terraform code files
|_ pulumi/gcp/compute/      # Contains the Pulumi code files

If you exit the container before you finish, run docker start -ai iac101 to get back into it. Your Terraform and Pulumi state is stored inside the container, so use the same container to destroy the resources you create.

HashiCorp Terraform

HashiCorp Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently. Terraform can manage existing service providers as well as custom in-house solutions. Terraform 1.6 and later are source-available under the Business Source License (BSL 1.1). If you prefer an open source license, OpenTofu is a community fork of Terraform, and the commands in this tutorial work the same way with tofu in place of terraform.

Configuration files describe the components needed to run a single application or your entire datacenter on Terraform. Terraform generates an execution plan describing what it will do to reach the desired state, and then executes it to build the described infrastructure. As the configuration changes, Terraform is able to determine what changed and create incremental execution plans which can be applied.

The infrastructure Terraform can manage includes low-level components such as compute instances, storage, and networking, as well as high-level components such as DNS entries, SaaS features, etc.

Provisioning infrastructure with Terraform

Let’s start with provisioning some resources in Google Cloud using Terraform code. The main.tf file in the terraform/gcp/compute/ directory is the code that has our infrastructure defined. In this file, we’re creating a new Compute Engine VM that runs a sample web app packaged in a Docker container. The Terraform code also creates a dedicated VPC network for the VM, so the tutorial doesn’t depend on your project’s default network, and a firewall rule that allows public access to the app over port 8080.

The VM boots from Google’s Container-Optimized OS, which comes with Docker installed. A startup script in startup.sh starts the app container each time the VM boots:

#!/bin/bash
# Runs at every boot on Container-Optimized OS: starts the sample web app on port 8080.
CONTAINER_NAME="hello-app"
IMAGE="us-docker.pkg.dev/google-samples/containers/gke/hello-app:2.0"

iptables -w -A INPUT -p tcp --dport 8080 -j ACCEPT
docker stop "$CONTAINER_NAME" || true
docker rm "$CONTAINER_NAME" || true
docker run --detach --restart=always --name="$CONTAINER_NAME" -p 8080:8080 "$IMAGE"

Using a startup script to run a container is the approach Google recommends now that the older container startup agent is deprecated. In main.tf, the VM resource points at the cos-stable image family and passes it the startup script:

resource "google_compute_instance" "app" {
  name         = "iac101-terraform"
  machine_type = "e2-micro"
  tags         = ["iac101-terraform"]

  boot_disk {
    initialize_params {
      image = "cos-cloud/cos-stable"
      size  = 10
      type  = "pd-standard"
    }
  }

  network_interface {
    subnetwork = google_compute_subnetwork.app.id
    access_config {
      # Ephemeral public IP
    }
  }

  metadata_startup_script = file("${path.module}/startup.sh")
}

To provision this, run this command in the terminal:

cd ~/project/terraform/gcp/compute/

Initializing Terraform

While in ~/project/terraform/gcp/compute/, run this command:

terraform init

You will see results similar to this.

Initializing the backend...

Initializing provider plugins...
- Reusing version 8.8.1 of hashicorp/google from the dependency lock file
- Installing hashicorp/google v8.8.1...
- Installed hashicorp/google v8.8.1 (signed by HashiCorp)

Terraform has been successfully initialized!

Previewing with Terraform

Terraform has a command that allows you to dry run and validate your Terraform code without actually executing anything. The command is called terraform plan, and it also graphs all the actions and changes that Terraform will execute against your existing infrastructure. In the terminal run:

terraform plan

You will see results similar to this (shortened here, with ... marking removed lines).

Terraform will perform the following actions:

  # google_compute_firewall.allow_app will be created
  + resource "google_compute_firewall" "allow_app" {
      + name               = "iac101-terraform-allow-8080"
      + network            = "iac101-terraform"
      + source_ranges      = [
          + "0.0.0.0/0",
        ]
      + target_tags        = [
          + "iac101-terraform",
        ]
      ...
    }

  # google_compute_instance.app will be created
  + resource "google_compute_instance" "app" {
      + machine_type            = "e2-micro"
      + name                    = "iac101-terraform"
      + zone                    = "us-central1-a"
      ...
    }

  # google_compute_network.app will be created
  ...

  # google_compute_subnetwork.app will be created
  ...

Plan: 4 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + app_url   = (known after apply)
  + public_ip = (known after apply)

As you can see, Terraform is going to create new Google Cloud resources for you based on the code in the main.tf file.

Terraform apply

Now you’re ready to create the new infrastructure and deploy the application. Run this command in the terminal:

terraform apply

Terraform will prompt you to confirm your command. Type yes and hit Enter.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

Terraform will build your infrastructure, and you will have an application up and running on Google Cloud shortly after it completes. Please note that it can take a minute or two for the application to come online after Terraform completes, while the VM boots and pulls the app’s container image.

Once done, you will see results similar to this (shortened here).

google_compute_network.app: Creation complete after 53s
google_compute_firewall.allow_app: Creation complete after 11s
google_compute_subnetwork.app: Creation complete after 32s
google_compute_instance.app: Creation complete after 18s

Apply complete! Resources: 4 added, 0 changed, 0 destroyed.

Outputs:

app_url = "http://136.80.19.220:8080"
public_ip = "136.80.19.220"

The app_url value is the address that your app is running on. Open it in a browser, or run curl with it from your local terminal, and you will see your app running on Google Cloud:

$ curl http://136.80.19.220:8080
Hello, world!
Version: 2.0.0
Hostname: 9333c39d2637

Terraform destroy

Now that you have proof that your Compute Engine VM and your application work, run the terraform destroy command and type yes when prompted to destroy the resources that you created in this tutorial. You can leave them up and running, but be aware that you’re liable for the costs of any resources running on Google Cloud beyond the Free Tier. It’s up to you, but running terraform destroy will terminate any running resources.

Next, I’ll demonstrate how to perform infrastructure provisioning using Pulumi.

Pulumi SDK

Pulumi SDK is an open source framework for defining and deploying cloud apps and IaC. Pulumi enables developers to write code in their favorite language such as TypeScript, JavaScript, Python, and Go. This enables modern approaches to cloud applications and infrastructure without needing to learn another YAML or DSL dialect. This unlocks abstractions and reuse, as well as access to your favorite IDEs, refactoring, and testing tools. Master one toolchain and set of frameworks and go to any cloud (AWS, Azure, GCP, or Kubernetes) with ease.

Provisioning infrastructure with Pulumi

You’ve experienced building infrastructure using Terraform. Now you’ll learn how to provision and deploy new infrastructure using Pulumi. The Pulumi example creates the same kind of Google Cloud resources as the Terraform example, using Pulumi code and tools. Pulumi enables you to define your infrastructure in programming languages, which provides lots of flexibility. In this example, we’re using Python as our language. The __main__.py file in the ~/project/pulumi/gcp/compute/ directory is where we define our infrastructure. Here’s the VM resource:

instance = gcp.compute.Instance(
    "iac101-pulumi",
    machine_type="e2-micro",
    tags=[VM_TAG],
    boot_disk={
        "initialize_params": {
            "image": "cos-cloud/cos-stable",
            "size": 10,
            "type": "pd-standard",
        },
    },
    network_interfaces=[
        {
            "subnetwork": subnet.id,
            "access_configs": [{}],  # Ephemeral public IP
        }
    ],
    metadata_startup_script=startup_script,
)

It’s the same VM as in the Terraform example, written as a Python object. It uses a copy of the same startup.sh script, and the program creates its own VPC network, subnet, and firewall rule, too.

The Pulumi example can be found in the ~/project/pulumi/gcp/compute/ directory. Run this command to change into the Pulumi example directory:

cd ~/project/pulumi/gcp/compute/

Setting up the Pulumi project

Pulumi keeps track of the state of your infrastructure in a backend. To keep this tutorial self-contained, you’ll use a local backend, which stores the state in a directory inside the container and doesn’t need a Pulumi account. To store state in Pulumi Cloud instead, run pulumi login and follow the prompts.

In the terminal, run these commands to log in to the local backend and set a passphrase. Pulumi uses the passphrase to encrypt any secrets in your stack configuration:

pulumi login --local
export PULUMI_CONFIG_PASSPHRASE="<choose-a-passphrase>"

Since we’re defining our IaC specs using Python, we’ll need to install the Pulumi Python SDK dependencies. The pulumi install command creates a Python virtual environment in the project directory and installs the packages listed in requirements.txt:

pulumi install

Finally, create a stack called dev. A stack is an isolated instance of your Pulumi program, such as a development or production environment:

pulumi stack init dev

Previewing with Pulumi

Pulumi has a dry-run command called pulumi preview. This command displays a preview of the updates to an existing stack whose state is represented by an existing state file. The new desired state is computed by running a Pulumi program, and extracting all resource allocations from its resulting object graph. These allocations are then compared against the existing state to determine what operations must take place to achieve the desired state. No changes to the stack will actually take place.

Run this command in the terminal:

pulumi preview

You will see results similar to this.

Previewing update (dev):

 +  pulumi:pulumi:Stack compute-dev create
 +  gcp:compute:Network iac101-pulumi create
 +  gcp:compute:Subnetwork iac101-pulumi create
 +  gcp:compute:Firewall allow-8080 create
 +  gcp:compute:Instance iac101-pulumi create
Outputs:
    app_url  : [unknown]
    public_ip: [unknown]

Resources:
    + 5 to create

Pulumi up

To execute the Pulumi code, use the command pulumi up. This command creates or updates resources in a stack. The new desired goal state for the target stack is computed by running the current Pulumi program and observing all resource allocations to produce a resource graph. This goal state is then compared against the existing state to determine what create, read, update, and/or delete operations must take place to achieve the desired goal state in the most minimally disruptive way. Afterward, this command records a full transactional snapshot of the stack’s new state so that the stack may be updated incrementally again later on.

In the terminal run:

pulumi up

Select the yes option and hit Enter. The Pulumi app will execute, and shortly after you’ll have a complete server running an application on Google Cloud. You will see results similar to this.

Updating (dev):

 +  gcp:compute:Network iac101-pulumi created (21s)
 +  gcp:compute:Subnetwork iac101-pulumi created (11s)
 +  gcp:compute:Firewall allow-8080 created (11s)
 +  gcp:compute:Instance iac101-pulumi created (27s)
 +  pulumi:pulumi:Stack compute-dev created (62s)
Outputs:
    app_url  : "http://136.80.19.220:8080"
    public_ip: "136.80.19.220"

Resources:
    + 5 created

Duration: 1m3s

In these results, you’ll see an app_url output. Its value is the address of the public-facing application on port 8080. Just like the previous Terraform example, you can access the application in a browser. Remember to allow a minute or two for the VM to bring everything online.

Pulumi destroy

Pulumi has a command to terminate all of the cloud resources called pulumi destroy. This command deletes all of the resources in an existing stack. The current state is loaded from the stack’s state in the backend. After running to completion, all of this stack’s resources will be gone, and the stack itself remains so that you can deploy it again.

Run pulumi destroy in the terminal and select the yes option when prompted to permanently destroy the Google Cloud resources created.

When you’re done with both examples, run exit to leave the container. You can then remove the container with docker rm iac101.

Summary

Congratulations! You’ve just leveled up and now have experience provisioning and deploying applications to Google Cloud using the modern IaC tools Terraform and Pulumi. The sample repository also includes tests for both examples that use mocked providers, so they run without credentials, and a CircleCI configuration that runs them on every commit. The following resources will help you expand your knowledge from here: