Enforcing web performance budgets in CI/CD with Sitespeed.io and Slack

Keeping your website fast as new features are introduced is a challenge. Performance regression is common issue that continues to plague websites, especially those of SaaS companies. In performance regression, newly shipped features introduce bloat, leading to slow page loads and reduced user conversion rates.

This is exactly what setting performance budgets helps prevent. Performance budgets are limits placed on a site’s performance metrics to keep things that could lead to site regression in check.

In this tutorial, you will learn how to enforce a web performance budget in a Continuous Integration and Deployment (CI/CD) pipeline using Sitespeed.io and Slack alerts.

Prerequesites

To follow through with this article, you need:

1. A CircleCI account.
2. A GitHub account.
3. Node.js version 14 or higher.
4. A Slack workspace.
5. Docker installed.
6. A Vercel account.

Project setup

To get started with enforcing a web performance budget in CI/CD, begin by cloning the sample project using the following:

git clone --branch starter https://github.com/Slimdan20/landing-page
cd landing-page

Note: The project used in this tutorial is a service landing page built with Next.js and Tailwind CSS. You can follow along with it or use a similar project.

Next, ensure your project is version-controlled with Git and pushed to a GitHub repository. You can do this by following the steps outlined in this guide.

Defining a performance budget

Before defining performance budgets for your project, it is important to first determine which metrics matter most for your site. While there are many metrics used to evaluate web performance, this tutorial focuses on two practical categories: time-based and quantity-based metrics.

Time-based metrics

Time-based metrics are used to evaluate a site’s load speed and the key events that happen during the loading process. Most of the metrics in this category are part of Google’s Core Web Vitals.

Key examples of time-based metrics include:

• First Contentful Paint (FCP): is a time-based metric that measures how long it takes the browser to display the first piece of content (either image or text) on the user’s screen. It has a recommended threshold of 1.8 seconds or less.
• Largest Contentful Paint (LCP): measures how long it takes the browser to render the largest content, be it an image or text, to end users. This important metric determines how fast users can see important content irrespective of the content’s size. LCP has a recommended threshold of 2.5 seconds or less.
• Interaction to Next Paint (INP): measures the speed at which a website responds to users’ input, like clicking a button or filling out a form. It comes with a recommended threshold of 200 milliseconds or less.
• Cumulative Layout Shift (CLS): measures the stability of a website’s layout upon page load. It checks for unexpected layout shifts, such as texts moving while fonts load or button movements after image appearance. Its recommended threshold is 0.1 second or less.
• Total Blocking Time (TBT): measures how long a webpage remains blocked before it begins to respond to the user’s input. This gives developers an understanding of how JavaScript or other tasks can lead to a slow site. Its recommended threshold is 200 milliseconds or less.

Quantity-based metrics

Unlike time-based metrics, which focus on speed, quantity-based metrics focus on size and numbers. Simply put, they measure how much data a website loads as well as the number of requests it makes. Quantity-based metrics are easier to manage during web page development and can help prevent sites from getting too heavy.

Some quantity-based metrics to set limitations on include:

• Total page weight: is the combined size of images, fonts, videos, and more that a webpage is required to load completely in a browser. It is an important metric, especially for mobile users, as lighter pages load faster and consume less data.
• Number of HTTP requests: The more complex a website is, the higher the number of HTTP requests it has to make, as every image, script, or stylesheet requires a separate network request.
• Font, stylesheet, and script size: Another important quantity-based metric that requires limitation is the size of fonts, stylesheets, and scripts. This is because imported fonts, unnecessary styles, and large scripts can easily cause webpage bloat and affect performance.

Implementing a performance budget with Sitespeed.io

Now that you understand the key performance metrics, it is time to enforce a web performance budget on your project using Sitespeed.io, a popular tool for web performance testing.

The first step is to define your budget. To do this, create a budget.json file at the root of your project and update its content with this:

{
  "budget": {
    "googleWebVitals": {
      "firstContentfulPaint": 1800,
      "largestContentfulPaint": 2500,
      "cumulativeLayoutShift": 0.1,
      "totalBlockingTime": 200
    },
    "requests": {
      "total": 50,
      "javascript": 10,
      "css": 5,
      "image": 25,
      "font": 5
    },
    "transferSize": {
      "total": 5500000,
      "javascript": 150000,
      "css": 100000,
      "image": 4500000,
      "font": 650000
    }
  }
}

This file defines a threshold for both time-based and quantity-based performance metrics.

Commit and push changes to GitHub.

Next, deploy your site to Vercel. To do this, log into your Vercel account.

Note: While Sitespeed.io can run on localhost, this tutorial uses a live URL from Vercel for simplicity and reliability in CI/CD.

In your Vercel dashboard, go to Add New Project.

Look for your project’s GitHub repo name (in this tutorial, it is landing-page). Click Import.

The deployment settings will be shown to you. Click Deploy to continue.

There you have it, your project has now been successfully deployed to Vercel. To view your deployed site, click the image thumbnail, then copy your site’s URL.

Finally, use Sitespeed.io with Docker to run your performance test against the thresholds defined in budget.json. In your terminal, run this command:

docker run --rm -v "$(pwd):/sitespeed.io" sitespeedio/sitespeed.io:37.9.0 https://landing-page-iota-virid.vercel.app/ --budget.configPath budget.json

If you are using a Windows terminal (PowerShell or CMD), use this format instead:

docker run --rm -v "C:\Users\Lenovo\landing-page:/sitespeed.io" sitespeedio/sitespeed.io:37.9.0 https://landing-page-iota-virid.vercel.app/ --budget.configPath budget.json

Note: Before executing this command, ensure Docker Desktop is running. Replace https://landing-page-iota-virid.vercel.app/ with your actual deployed site URL. Replace "C:\Users\Lenovo\landing-page with your actual project directory.

The first run will take a couple of minutes as Docker downloads the necessary Sitespeed.io image. Once complete, a sitespeed-result folder will be created in your project directory containing the test report.

To view the report, open your project folder in File Explorer, navigate to the sitespeed-result folder, and open the index.html file to see a summary of the results.

Integrating CircleCI into your project

To automatically run Sitespeed.io tests whenever changes are pushed to your codebase, you need to integrate a CI/CD tool. A perfect CI/CD tool for this purpose is CircleCI.

To integrate CircleCI, create a .circleci folder at the root of your project. Inside it, create a config.yml file. Add this content:

version: 2.1

jobs:
  performance-budget-check:
    machine: true 
    steps:
      - checkout 

      - run:
          name: Run Sitespeed.io against deployed site
          command: |
            docker run --rm -v "$(pwd):/sitespeed.io" sitespeedio/sitespeed.io:37.9.0 \
              https://landing-page-iota-virid.vercel.app/ \
              --budget.configPath /sitespeed.io/budget.json

      - store_artifacts:
          path: sitespeed-result
          destination: sitespeed-report

workflows:
  version: 2.1
  enforce-performance-budget:
    jobs:
      - performance-budget-check

The above configuration defines a performance-budget-check job that runs in a Linux virtual machine environment using machine: true, which provides full Docker support required to execute the Sitespeed.io Docker container.

The job first checks out the project code using the checkout step. It then runs Sitespeed.io via Docker against your deployed website and stores the results as artifacts upon completion.

Finally, the enforce-performance-budget workflow triggers the performance-budget-check job automatically whenever a new pipeline is run.

Note: Replace https://landing-page-iota-virid.vercel.app with the actual URL of your deployed site.

Save your changes and push them to your GitHub repository.

Setting up a project in CirceCI

With your config.yml in place, go to CircleCI and log into your account.

Next, follow the steps outlined in CircleCI’s official guide to set up your project.

With everything properly configured, your project should build green.

There you have it! Your project is now set up to automatically run Sitespeed.io tests against your defined budget whenever you push changes to GitHub.

Setting up Slack alerts

You can take your performance budget enforcement a step further by setting up Slack alerts. This allows your team to receive instant notifications whenever any metric exceeds the limits defined in your performance budget.

To do this, you will need to set up a Slack app with the right permissions, add its credentials to CircleCI, and update your config.yml file so alerts can be posted based on Sitespeed.io test results.

To get started, log into Slack.

Next, visit the Slack API webpage. Click the Create New App button to continue.

In the pop-up configuration page that follows, select the From Scratch option.

Next, enter a valid app name followed by the workspace you wish to develop your app in and click the Create App button to continue.

You will be directed to a Basic Information page. From the features tab, click OAuth & Permissions.

This will direct you to the OAuth & Permissions page. Scroll down to the Scopes section and click Add an OAuth scope. Include the following:

• chat:write.
• chat:write.public.
• files:write.

Next, scroll to the OAuth Tokens section and install it in your workspace.

A page requesting permission to access your workspace will appear. Click the Allow button to continue.

You will be redirected to your OAuth & Permissions page, where a unique Bot User OAuth Token will be provided for you in the OAuth Tokens section. Copy this token so you can use later on in this tutorial.

Go to your Slack workspace. Right-click the channel where you want to receive alerts, and copy the channel link.

Paste the copied link somewhere, then extract the last part of the URL(e.g, C07LSQ64D9P). This will serve as your channel ID.

Next, go to your project dashboard on CircleCI. Click Project Settings -> Environment Variables. Click Add environment variable, and enter this:

• Name: SLACK_ACCESS_TOKEN.
• Value: Bot User OAuth Token.
• Name: SLACK_DEFAULT_CHANNEL.
• Value: Channel ID.

Click the Add environment variable button to continue.

Finally, update your .circleci/config.yml file to handle Slack notifications.

Note: CircleCI offers a Slack orb that makes integrating Slack with CircleCI straightforward. In this tutorial, we will be utilizing that orb.

orbs:
  slack: circleci/slack@5.1.1

workflows:
  version: 2.1
  enforce-performance-budget:
    jobs:
      - performance-budget-check:
          post-steps:
            - slack/notify:
                event: fail
                template: basic_fail_1

The above configuration integrates the Slack orb from CircleCI (circleci/slack@5.1.1) into your workflow to enable Slack notifications.

It modifies the performance-budget-check job by adding a post-steps section. Within this section, the slack/notify step is configured with event: fail, which triggers a notification whenever the CircleCI build fails. The template: basic_fail_1 setting sends a predefined message to your Slack channel to alert you of build failure.

Commit and push changes to GitHub.

This will trigger a CircleCI pipeline build, and since your website performance metrics are well within the thresholds defined in your budget.json file, your pipeline should pass successfully and appear green in your CircleCI dashboard.

Testing Slack alert on a failed Sitespeed.io test

So far, you have successfully set up Slack alerts for your project. As a result, you should now receive a notification in Slack whenever your performance budget thresholds are exceeded and the test fails.

To test this setup, modify the transferSize section on your budget.json file by reducing some of the threshold values:

  "transferSize": {
      "total": 40000,
      "javascript": 15000,
      "css": 10000,
      "image": 5000,
      "font": 5000
    }

Commit and push changes to GitHub.

This will trigger a pipeline build in your CircleCI dashboard, and since your web performance metrics are higher than the threshold value set in your budget.json file, your CircleCI build will fail.

A notification will be sent to your Slack channel to inform you of the failure.

Conclusion

Setting and enforcing a web performance budget is a powerful way to keep your site fast, efficient, and user-friendly. Users expect pages to load quickly, and delays, even by a few seconds, can lead to frustration or loss of engagement.

In this tutorial, you have learned how to define a performance budget, using Sitespeed.io to enforce it, and integrating Slack notifications to stay updated when threshold limits are exceeded. Moving on, endeavor to implement all you have learned in both new and existing projects.

You can view the complete code for this tutorial on GitHub.