Building CI/CD pipelines using the CircleCI AWS ECR Orb

CircleCI orbs are designed to get you up and running quickly on CircleCI. You can easily integrate your DevOps tools with trusted orbs provided by our technology partners.

In this post I’ll demonstrate and explain the AWS ECR Orb and its usage within the CircleCI configuration file. For this, I’ll use CircleCI Workflows, push an image to Docker Hub and leverage a specified AWS ECR.

What are CircleCI orbs?

Orbs are packages of CircleCI configuration that can be shared across projects. Orbs allow you to make a single bundle of jobs, commands, and executors that can reference each other and can be imported into a CircleCI build configuration and invoked in their own namespace. Orbs are registered with CircleCI, with revisions expressed using the semver pattern. CircleCI orbs are hosted and maintained on the CircleCI Orbs Registry which is a centralized repository for CircleCI orbs.

Prerequisites

Before you get started you’ll need:

GitHub account
Fork or clone of the orbs-ecr-python repo locally
Docker Hub account
AWS account
AWS IAM User Account with Access Keys & Secrets
A new AWS ECR Repository with the name orbs-ecr-python

Using the CircleCI AWS ECR orb

The config.yml file defines a pipeline. Here’s the one I created for this project:

version: 2.1

orbs:
  aws-cli: circleci/aws-cli@5.1.0
  aws-ecr: circleci/aws-ecr@9.6.0

jobs:
  build_test:
    docker:
      - image: cimg/python:3.11
    steps:
      - checkout
      - run:
          name: Setup VirtualEnv
          command: |
            pip install --user --upgrade pip
            pip install --user --no-cache-dir -r requirements.txt
      - run:
          name: Run Tests
          command: |
            python test_hello_world.py
  docker_hub_build_push_image:
    docker:
      - image: cimg/python:3.11
    steps:
      - checkout
      - setup_remote_docker:
          docker_layer_caching: false
      - run:
          name: Build and push Docker image to Docker Hub
          command: |
            export TAG=0.1.${CIRCLE_BUILD_NUM}
            export IMAGE_NAME=${CIRCLE_PROJECT_REPONAME}
            docker build -t ${DOCKER_LOGIN}/${IMAGE_NAME}:${TAG} -t ${DOCKER_LOGIN}/${IMAGE_NAME}:latest .
            echo "${DOCKER_PWD}" | docker login -u "${DOCKER_LOGIN}" --password-stdin
            docker push ${DOCKER_LOGIN}/${IMAGE_NAME}:${TAG}
            docker push ${DOCKER_LOGIN}/${IMAGE_NAME}:latest

workflows:
  build_test_deploy:
    jobs:
      - build_test
      - docker_hub_build_push_image:
          requires:
            - build_test
      - aws-ecr/build_and_push_image:
          account_id: ${AWS_ACCOUNT_ID}
          auth:
            - aws-cli/setup
          region: ${AWS_REGION}
          repo: ${CIRCLE_PROJECT_REPONAME}
          tag: ${CIRCLE_BUILD_NUM}
          requires:
            - build_test

Specifying orbs and workflows

orbs:
  aws-cli: circleci/aws-cli@5.1.0
  aws-ecr: circleci/aws-ecr@9.6.0

The orbs: section declares the external orbs that will be used in this pipeline. This imports two complementary orbs:

aws-cli for AWS authentication setup
aws-ecr for ECR-specific operations

The version numbers (5.1.0 and 9.6.0) ensure you’re using stable, current versions of these orbs. Think of these orb declarations as import statements; they make external functionality available within your configuration.

workflows:
  build_test_deploy:
    jobs:
      - build_test
      - docker_hub_build_push_image:
          requires:
            - build_test
      - aws-ecr/build_and_push_image:
          account_id: ${AWS_ACCOUNT_ID}
          auth:
            - aws-cli/setup
          region: ${AWS_REGION}
          repo: ${CIRCLE_PROJECT_REPONAME}
          tag: ${CIRCLE_BUILD_NUM}
          requires:
            - build_test

Understanding the workflow structure

The workflows: section orchestrates the execution of your pipeline jobs. Our build_test_deploy workflow defines a clear progression:

Test your application
Build and push images to both Docker Hub and AWS ECR in parallel

jobs:
  - build_test
  - docker_hub_build_push_image:
      requires:
        - build_test
  - aws-ecr/build_and_push_image:
      requires:
        - build_test

Both the Docker Hub push job and the AWS ECR orb job depend on the build_test job completing successfully. This ensures that you deploy only those images that have passed your tests. The parallel execution of the two push jobs maximizes efficiency - there’s no need to wait for one registry push to complete before starting the other.

AWS ECR orb configuration

The AWS ECR orb simplifies the process of building and pushing Docker images to your ECR repository. Here is its configuration:

- aws-ecr/build_and_push_image:
    account_id: ${AWS_ACCOUNT_ID}
    auth:
      - aws-cli/setup
    region: ${AWS_REGION}
    repo: ${CIRCLE_PROJECT_REPONAME}
    tag: ${CIRCLE_BUILD_NUM}
    requires:
      - build_test

This orb call demonstrates several key concepts:

Authentication: The auth parameter uses the aws-cli/setup command from the AWS CLI orb to handle AWS credentials.
Dynamic parameters: Environment variables like ${AWS_ACCOUNT_ID}, ${AWS_REGION}, and built-in variables like ${CIRCLE_PROJECT_REPONAME} make the configuration reusable across different projects.
Dependency management: The requires key ensures this job only runs after successful tests.
Automatic tagging: Using ${CIRCLE_BUILD_NUM} provides unique, sequential tags for each build.

The orb handles all the complexity of ECR authentication, Docker building, and image pushing behind the scenes. See the AWS ECR Orb docs for complete parameter documentation.

Custom job definitions

Take a moment to examine the two custom jobs that handle testing and Docker Hub deployment. These jobs work alongside the AWS ECR orb to create a comprehensive CI/CD pipeline:

The build_test job
The docker_hub_build_push_image job

The build_test job:

build_test:
  docker:
    - image: cimg/python:3.11
  steps:
    - checkout
    - run:
        name: Setup VirtualEnv
        command: |
          pip install --user --upgrade pip
          pip install --user --no-cache-dir -r requirements.txt
    - run:
        name: Run Tests
        command: |
          python test_hello_world.py

This job uses the modern cimg/python:3.11 image and employs current Python best practices. Instead of manually creating virtual environments, it leverages pip’s --user flag for clean dependency installation. The simplified approach reduces complexity while maintaining isolation.

The docker_hub_build_push_image job:

docker_hub_build_push_image:
  docker:
    - image: cimg/python:3.11
  steps:
    - checkout
    - setup_remote_docker:
        docker_layer_caching: false
    - run:
        name: Build and push Docker image to Docker Hub
        command: |
          export TAG=0.1.${CIRCLE_BUILD_NUM}
          export IMAGE_NAME=${CIRCLE_PROJECT_REPONAME}
          docker build -t ${DOCKER_LOGIN}/${IMAGE_NAME}:${TAG} -t ${DOCKER_LOGIN}/${IMAGE_NAME}:latest .
          echo "${DOCKER_PWD}" | docker login -u "${DOCKER_LOGIN}" --password-stdin
          docker push ${DOCKER_LOGIN}/${IMAGE_NAME}:${TAG}
          docker push ${DOCKER_LOGIN}/${IMAGE_NAME}:latest

This job demonstrates manual Docker operations, creating both versioned and latest tags. The use of export statements makes the script more readable, and the secure password handling with --password-stdin follows current security best practices. The script pushes both a specific version tag and the latest tag, giving users flexibility in how they reference the image.

Bringing it all together

This configuration showcases the power of combining custom jobs with CircleCI orbs. The custom jobs give you full control over your testing and Docker Hub deployment process, while the AWS ECR orb eliminates boilerplate code for ECR operations.

The result is a robust pipeline that:

Tests your application thoroughly before deployment
Pushes images to both Docker Hub and AWS ECR simultaneously
Uses current best practices for Python development and Docker security
Leverages CircleCI’s built-in environment variables for maintainable configuration

After successful execution, you’ll have identical, tested Docker images available in both Docker Hub and your AWS ECR repository. These are ready for deployment to your preferred container orchestration platform.

Push the code to GitHub

After you have completed all the previous steps, push the code to your GitHub repository.

Connect CircleCI to your repository

Log into CircleCI and go to the Projects dashboard. Find your orbs-ecr-python repository in the project list and click Set Up next to it.

CircleCI will detect the .circleci/config.yml file in your repository. Select the branch that contains your configuration file, then click Set Up Project to start the initial pipeline run.

CircleCI set up project

The pipeline will start immediately, but it will fail on the first run because the environment variables you need haven’t been configured yet. This is completely expected!

Don’t worry about this initial failure, it’s a normal part of the set-up process. It actually enables you to access the project settings where you’ll configure the required environment variables.

Create environment variables in CircleCI

From your CircleCI dashboard, go to your project settings and add the following environment variables:

Name: AWS_ACCESS_KEY_ID Value: Your AWS IAM Account's Access Key ID
Name: AWS_ACCOUNT_ID Value: Your AWS Account ID (12-digit number)
Name: AWS_ECR_ACCOUNT_URL Value: Your AWS ECR Registry URL
Name: AWS_REGION Value: Your AWS region (e.g., us-east-1, us-west-2)
Name: AWS_SECRET_ACCESS_KEY Value: Your AWS IAM Account's Secret Access Key
Name: DOCKER_LOGIN Value: Your Docker Hub User Name
Name: DOCKER_PWD Value: Your Docker Hub Password

After adding these variables, re-run the pipeline. It should now pass successfully.

CircleCI pipeline success

With CI in place, every push will trigger tests and Docker builds to both Docker Hub and AWS ECR. This feedback loop helps you catch regressions early and ensures your application images are always ready for deployment.

Conclusion

Throughout this tutorial, you’ve explored how CircleCI orbs transform complex deployment workflows into simple, maintainable configurations. The AWS ECR orb exemplifies this perfectly by condensing dozens of lines of manual Docker and AWS CLI commands into just a few parameters. Instead of wrestling with authentication tokens, registry URLs, and error handling, you can focus on the core logic of your application.

The power of orbs becomes even more apparent when combined with custom jobs. You’ve shown how testing and Docker Hub deployment can run alongside AWS ECR operations, creating a comprehensive pipeline that serves multiple deployment targets simultaneously. This hybrid approach gives you the flexibility to handle application-specific requirements while using proven solutions for infrastructure operations.

Orbs represent more than just convenience - they embody best practices developed by the CircleCI community and technology partners. When you use the AWS ECR orb, you’re not just saving time; you’re adopting security patterns, error handling strategies, and authentication methods that have been tested across thousands of projects. This shared knowledge accelerates your CI/CD maturity without requiring deep expertise in every tool and platform.

The ecosystem continues to grow as teams identify common patterns and contribute solutions back to the community. Whether you’re working with cloud providers, testing frameworks, or deployment platforms, there’s likely an orb that can simplify your configuration. And when you develop your own reusable patterns, you have the opportunity to create custom orbs that benefit the broader CircleCI community. The Orb Registry serves as both a resource for finding solutions and a platform for sharing innovations that make CI/CD more accessible for everyone.

Site

Blog