CircleCI Orb-Based Pipeline

Overview

CircleCI 2.1 introduced a powerful config format that allows teams to share and reuse pipeline components via orbs — versioned packages of jobs, commands, and executors published to the CircleCI Orb Registry. Instead of writing twenty lines of shell commands to install Node.js at the right version, configure npm caching, and handle edge cases, you simply reference the official circleci/node orb and call its install-packages command. The orb maintainer has already solved those problems for you.

This pipeline implements a common two-job pattern: a build-and-test job that runs on every branch, and a deploy job that runs only on main. The workflow graph connects them with a requires dependency, so the deploy job waits for the build to succeed. The compiled dist/ folder is passed from the build job to the deploy job via CircleCI's workspace mechanism.

Complete YAML

version: 2.1

orbs:
  node: circleci/[email protected]
  aws-cli: circleci/[email protected]

workflows:
  build-and-deploy:
    jobs:
      - build-and-test
      - deploy:
          requires:
            - build-and-test
          filters:
            branches:
              only: main

jobs:
  build-and-test:
    docker:
      - image: cimg/node:20.10
    steps:
      - checkout
      - node/install-packages:
          pkg-manager: npm
      - run:
          name: Run tests
          command: npm test
      - run:
          name: Build
          command: npm run build
      - persist_to_workspace:
          root: .
          paths:
            - dist

  deploy:
    executor: aws-cli/default
    steps:
      - attach_workspace:
          at: .
      - aws-cli/setup
      - run:
          name: Deploy to S3
          command: aws s3 sync dist/ s3://$S3_BUCKET --delete

Orbs

The orbs block at the top of the config imports two certified orbs from the CircleCI Orb Registry. Each orb reference follows the format namespace/name@version — pinning to a specific semver version (like 5.2 rather than 5) ensures your pipeline behaves consistently even if the orb publishes breaking changes in a minor patch. You can browse available orbs at circleci.com/developer/orbs.

The circleci/node orb contributes the node/install-packages command used in the build job. This command handles npm dependency installation with built-in caching — it generates a cache key from your package-lock.json hash, restores a cache on the next run if the lockfile hasn't changed, and saves a new cache when dependencies change. This can reduce dependency installation from 60+ seconds to under 5 seconds on cache hits.

The circleci/aws-cli orb contributes both the aws-cli/default executor (a pre-built Docker image with the AWS CLI already installed) and the aws-cli/setup step (which reads AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_DEFAULT_REGION from CircleCI environment variables and writes the AWS credentials file). You define those variables in the CircleCI project settings under "Environment Variables" so they are never hardcoded in your config.

Workflows and job graph

CircleCI workflows define a directed acyclic graph (DAG) of jobs. In this config the workflow build-and-deploy lists two jobs. The deploy job has a requires list containing build-and-test, which means CircleCI will not start deploy until build-and-test finishes with a success status. If the build job fails, the deploy job is cancelled automatically.

The filters block on the deploy job restricts it to run only when the pipeline was triggered on the main branch. build-and-test has no filters, so it runs on every branch. This means feature branch pushes run the full test suite but never touch production infrastructure — exactly the right balance between fast feedback and safe deployments.

You can scale this pattern further by adding more jobs to the workflow and chaining them with requires. For example, you could add an integration test job that runs after build-and-test and before deploy, creating a three-node pipeline graph. CircleCI will parallelise any jobs at the same "level" of the DAG — if two jobs both require only build-and-test, they run in parallel after the build finishes.

Workspace persistence

Each CircleCI job runs in a fresh, isolated container. By default, files created in one job are not visible to any other job. The persist_to_workspace step at the end of the build job solves this by uploading a snapshot of selected paths to CircleCI's temporary workspace storage. The root: . sets the workspace root to the current working directory, and paths: [dist] specifies that only the dist/ directory should be persisted — keeping the workspace small and fast to upload and download.

In the deploy job, attach_workspace: at: . downloads the persisted files into the job's current directory. After this step, the dist/ folder is available exactly as the build job left it, and the aws s3 sync command can reference it directly. This mechanism is CircleCI's equivalent of GitLab CI artifacts or GitHub Actions' upload-artifact / download-artifact actions.

The aws s3 sync dist/ s3://$S3_BUCKET --delete command synchronises the local dist/ folder to an S3 bucket, uploading new and modified files and deleting files in S3 that no longer exist locally. The --delete flag is important for keeping the bucket contents in sync with your build output; without it, old files removed from your project would persist in S3 indefinitely. The $S3_BUCKET variable is read from CircleCI's project environment variables.