Runner Release 3.1.0

Container Runner:

• Version 3.1.0 introduces substantial performance and reliability improvements to container runner. These updates are designed to enhance the overall efficiency and stability of operations at scale. For more background and technical details on these changes, you can visit our Runner Init repository and documentation.
• To upgrade, ensure that the Helm chart is at least version v101.1.2. The chart is pinned to the major version tag of the runner, enabling automatic upgrades on redeployment. If you wish to remain on the existing architecture (3.0.x), you can do so by setting the tag to kubernetes-3.0. This version will continue to receive only critical bug fixes and security patches. You can check the configuration details in the Helm chart repository.

Customers have been asking for more flexibility when building dependency requirements between jobs. To date “requires” was based on the success of the required job. That is no longer the case. “Requires” now supports job status; success, failed, and canceled. So you can now build out complex dependency maps and fan in/out scenarios.

• When running complex fan-in/fan-out workflows, you may want to continue work even when there are test failures.
• When certain jobs fail or are canceled, you may want to execute cleanup jobs, or decide not to do a deployment.

Example Config

jobs:
      - a
      - b
      - c:
          requires:
            - a canceled
            - b:
              - failed
              - success
      - d:
          requires:
            - a: #defaults to success
            - b:
              - failed
              - success

Flexible requires, along with flexible when statements and expression based job filters, opens up more opportunities to optimize your pipelines for fast feedback and greater control of costs.

For more information on the feature visit the documentation.

The page Organization Settings > Overview now includes a new “Organization slug” section, which lets users view and copy in one click the slug of their organization.

Please note that all the following slug formats are valid:

• github/org-name
• bitbucket/org-name
• circleci/FMckQH4sfM6PsIPQEG1RLd

Updates:

• We’ve removed the ‘Scheduled Pipelines’ tab from Project Settings and added the ability to create scheduled pipelines via the ‘Trigger Type’ dropdown on the Project Settings > Triggers page.
• For users in organizations that are integrated with GitHub OAuth App, we have added a note on the ‘Add Trigger’ and ‘Add Pipeline’ pages that states they can now use GitHub App pipelines, triggers, and custom webhooks.
• If a user does not have a GitHub App pipeline for their project, we display a message on the Project Settings > Triggers page to direct users to create a GitHub App pipeline first before they can create a GitHub App trigger.
• When users delete a trigger, we’ve implemented an automatic page refresh so the deleted trigger is immediately removed from the list.

The pipeline value pipeline.trigger_parameters.webhook.body is now present for all GitHub App and Bitbucket Data Center pipelines, no matter how they are triggered.

This value was previously undefined for pipelines triggered via API, UI, or via Bitbucket Data Center push. This prevented customers from using this pipeline value in their config file if they wanted their pipeline to be triggerable from these event sources as well as from a custom webhook.

For pipelines triggered via custom webhook, the value of pipeline.trigger_parameters.webhook.body is the stringified event payload. For all other methods of triggering, the value is now set to {}.

NOTE: This value is still not set for OAuth pipelines.

Updates:

• Added a ‘Remove all’ button to allow users to easily remove parameters for specific projects
• We display in-modal error states when users input duplicate parameter names, have a missing config file in their selected branch, etc.
• When users manually add a parameter, the new row will now appear at the top of the list instead of the bottom, making it easier to spot and fill in

GitHub App triggers, previously uneditable, can now be edited.

Specifically, users can edit the following fields:

• Trigger name
• Config branch (if present)
• Checkout branch (if present)

Updates:

• For projects with a single pipeline, the modal now defaults to that pipeline, so users no longer need to select it from the dropdown
• Automatically populate the branch in the dropdown when users are on the Project > Branch page
• Added the ability to search in the branch dropdown within the modal
• Rolled out a fix so the text no longer duplicates when users copy and paste into the parameter inputs

The page Project Settings > Overview now includes a new “Project Slug” section, which lets users view and copy in one click the slug of their project.

Please note that both the following project slug formats are valid but mutually exclusive by project:

• github/org-name/project-name
• circleci/FMckQH4sfM6PsIPQEG1RLd/4CNEcx&6Yk6wkpwo2ft9Uo

IP Ranges now support parameters providing more dynamic control of when ip ranges is used in builds.

Example

version: 2.1

parameters:
  ip_ranges:
    type: boolean
    default: true

executors:
  base:
    docker:
      - image: cimg/base:stable

jobs:
  job_a:
    executor: base
    circleci_ip_ranges: << pipeline.parameters.ip_ranges >>
    steps:
      - run: echo "Running job a"

workflows:
  workflow-a:
    jobs:
      - job_a

As a continuation of CircleCI’s update to enable access to “GitHub App functionality” to all CircleCI users (including users of the GitHub OAuth App integration), the “Triggers” tab in Project Settings of the CircleCI web app now shows a “GitHub OAuth App” trigger and a badge to indicate the integration type. This is intended to make it easier for all users to understand what is currently set up within their CircleCI project.

As part of CircleCI’s update to enable access to “GitHub App functionality” to all CircleCI users (including users of the GitHub OAuth App integration), the “Pipelines” tab in Project Settings of the CircleCI web app now shows a “GitHub OAuth App” or “Bitbucket Cloud” pipeline and a badge to indicate the integration type. This is intended to make it easier for all users to understand what is currently set up within their CircleCI project. A subsequent update will do the same for the “Triggers” tab.

Known issues: The organization name will be removed from the “Config Source” column to only show the repository name.

A new v2 API endpoint for triggering pipelines via API is now available to all organizations that integrate through GitHub (GitHub OAuth and GitHub App) and Bitbucket (Cloud and Data Center). This functionality is not yet available to organizations that integrate through GitLab.

The endpoint is:

POST /api/v2/project/{provider}/{organization}/{project}/pipeline/run

This is now the recommended API to be used to trigger pipelines.

The pre-existing endpoint for triggering new pipelines - which is only available to organizations integrating through GitHub oAuth or Bitbucket Cloud - continues to be supported.

Job Filters now support expression based conditions, much like contexts. This enables you to conditionally run jobs based on pipeline values. This supports you optimizing pipelines to lower costs, decrease time to feedback, or run specific jobs based on context of the source of change. Previously filtering was extremely limited to branches and evaluating to only and ignore.

Example for a Deploy workflow:

workflows:
  deploy:
    jobs:
      - init-service
      - build-service-image:
          requires:
            - init-service
      - dry-run-service:
          requires:
            - init-service
          filters: pipeline.git.branch != "main" and pipeline.git.branch != "canary"
      - publish-service:
          requires:
            - build-service-image
            - test-service
          filters: pipeline.git.branch == "main" or pipeline.git.tag starts-with "release"
      - deploy-service:
          context:
            - org-global
          requires:
            - publish-service
          filters: pipeline.git.branch == "main" and pipeline.git.commit.subject starts-with "DEPLOY:"

We will be following on this with expression based evaluations for When Statements which enable more flexibility in how you choose when steps or workflows run. When Statements are also currently limited in their evaluation options. Improving When Statements will expand your options optimizing for cost and speed.

GitHub App users who granted their App access to more than 100 repositories have been running into issues when creating Pipelines and Triggers on CircleCI, because the repository dropdown would only display a limited number of repositories.

We have now replaced the dropdown inputs with a search component, which lets users search repositories by keyword.

Note: at the moment the search results include all of an organization’s public repositories. We plan to adjust this behaviour in the near future.

Orbs are the leading method to abstract away shared aspects of your pipelines. They make it possible to simplify the complex, share, and maintain jobs and commands across your organization. For large organizations, private orbs cab provide efficiency and scale.

Orb security settings now allow organizations to enable only private orbs. Previously orb settings required enabling uncertified orbs in order to enable private orbs. This prevented some organizations, whom were concerned about uncertified orb security, from benefiting from the private orbs.

For more information on orbs and creating your own private orbs see our documentation.

We’ve added additional meta data using the “tag” key to audit logs to help customers identify how workflow reruns were triggered.

Workflow tags include:

• setup
• rerun-single-job
• rerun-workflow
• rerun-workflow-from-beginning
• rerun-workflow-from-failed
• rerun-workflow-from-job-ids-sparse-tree
• rerun-single-job-with-ssh

Example audit log entries

Workflow Started:

workflow.start
{"job":{},
 "workflow":{"id":"247...","name":"test-and-build","tag":null}}

workflow.job.context.request
{"job":{"context_uuids":[],"contexts":[],"id":"408...","job_name":"test","job_status":"pending"},
 "workflow":{"id":"247...","name":"test-and-build","tag":null}}

workflow.job.start
{"job":{"contexts":[],"id":"408...","job_name":"test","job_status":"running"},
 "workflow":{"id":"247...","name":"test-and-build","tag":null}}

workflow.job.finish
{"job":{"contexts":[],"id":"408...","job_name":"test","job_status":"failed"},
 "workflow":{"id":"247...","name":"test-and-build","tag":null}}

Retry the original workflow with rerun job with SSH enabled:

workflow.retry
{"job":{},
 "workflow":{"id":"247...","name":"test-and-build","tag":null}}

SSH enabled rerun started:

workflow.start
{"job":{},
"workflow":{"id":"811...","name":"test-and-build","tag":"rerun-single-job-with-ssh"}}

workflow.job.context.request
{"job":{"context_uuids":[],"contexts":[],"id":"c6a...","job_name":"test","job_status":"pending"},
 "workflow":{"id":"811...","name":"test-and-build","tag":"rerun-single-job-with-ssh"}}

workflow.job.start
{"job":{"contexts":[],"id":"c6a...","job_name":"test","job_status":"running"},
 "workflow":{"id":"811...","name":"test-and-build","tag":"rerun-single-job-with-ssh"}}

workflow.job.finish
{"job":{"contexts":[],"id":"c6a...","job_name":"test","job_status":"canceled"},
 "workflow":{"id":"811...","name":"test-and-build","tag":"rerun-single-job-with-ssh"}}

The pipelines page can load slowly for some users. The number of pipelines and number of projects can contribute to that slowness. A recent change to one of the underlying data services has been made resulting in a 30%-40% improvement in performance.

We continue to work on improving the pipelines page experience. Our current focus is on load time. We do have plans to introduce new filter and search capabilities in the future.

We have added an API and UI component to approval jobs that allows you to “cancel” the approval job. Previously it was not possible to cancel approval jobs. This limitation forced you to either approve the job or let it expire, which takes 90 days. This led to a suboptimal experience with jobs left in a non-terminal state. This could impact performance metrics, as well pipelines views. It also led to non-intuitive actions being taken like cancelling the workflow which cancels all remaining jobs.

We are following up this feature with more flexible require statements. So you can have a job depend on an approval while another depends on cancellation. This is great for cleanup jobs, notifications, or other tasks that may only need to happen or not happen upon cancellation.

For CircleCI organizations that integrate with GitLab or GitHub App:

• The user count on the People tile reflects the number of users in the organization instead of active users in the billing period.

• Organization admins have the ability to invite additional teammates directly from the Organization Homepage by clicking the “Invite your teammates” button.

The CircleCI Automation Actor will be used to attribute automated workflow and job cancelations on September 17th, 2024. This change will impact two functionalities:

• Auto-cancel redundant workflows
• Jobs canceled via the v1 Cancel Job API using a project token

This change is intended to better indicate that the action was performed by a machine.

Once the change is completed, the canceled_by field of the getWorkflowById and the listWorkflowJobs API response will return the CircleCI Automation Actor ID. You only need to take action if you rely on this field to have a static value.

Previously: The pipeline.trigger_parameters.webhook.body pipeline value was only populated when a custom webhook triggered the pipeline. If a config used pipeline.trigger_parameters.webhook.body and the pipeline was triggered with a GitHub App event, an errored pipeline would occur. This made it difficult to use one configuration file with a custom webhook & GitHub App triggers.

New behavior: The pipeline.trigger_parameters.webhook.body pipeline value will be populated with an empty string if the pipeline is triggered from a GitHub App event. This simplifies using one configuration file with a custom webhook & GitHub App triggers.

Three new fields will be introduced to the Usage API on August 23, 2024 to help you better understand how your builds are running. These fields are:

• job_run_queued_at: the timestamp (UTC) of when the job started to queue (if applicable)
• job_run_started_at: the timestamp (UTC) of when the job started to run
• job_run_stopped_at: the timestamp (UTC) of when the job stopped (only logged for success and failed job build statuses)

These three new fields will be introduced into the CSV output in the order above immediately following the job_run_date and prior to the job_build_status fields.

NOTE: If you have an automated ingestion process that relies on the format of the CSV remaining consistent, please update your ingestion process to accommodate these three new fields prior to August 23, 2024.