CI/CD Pipeline Best Practices — Stages, Testing, Security Scanning, and Deployment Strategies

In this tutorial, you'll learn about CI/CD Pipeline Best Practices. We cover key concepts, practical examples, and best practices to help you understand and apply this topic effectively.

CI/CD pipeline best practices help teams automate the build, test, and deployment lifecycle with staged workflows, integrated security scanning, artifact versioning, and deployment strategies that eliminate downtime.

What You'll Learn

Why It Matters

Manual deployments are slow, error-prone, and impossible to scale. A well-designed CI/CD pipeline turns deployment from a stressful manual process into a reliable automated one — catching bugs early, enforcing quality gates, and shipping updates multiple times a day without incident.

Real-World Use

DodaZIP uses a multi-stage CI/CD pipeline that lints and type-checks on every commit, runs unit and integration tests in parallel, scans dependencies for vulnerabilities, builds optimized Docker images, and deploys to staging then production using blue-green releases — all without human intervention.

flowchart LR
    A[Code Commit] --> B[Lint & Type Check]
    B --> C[Unit Tests]
    B --> D[Build Artifacts]
    C --> E[Integration Tests]
    D --> E
    E --> F[Security Scan]
    F --> G[Staging Deploy]
    G --> H[E2E Tests]
    H --> I[Production Deploy]
    I --> J[Health Check]
    style A fill:#4CAF50,color:#fff
    style J fill:#4CAF50,color:#fff

Core Pipeline Stages

Stage 1 — Lint and Static Analysis

The first stage runs linters and Static Analysis tools to catch code quality issues before any tests execute. This provides fast feedback — typically completing in under a minute.

# .github/workflows/ci.yml
name: CI Pipeline

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm ci
      - run: npm run lint
      - run: npm run typecheck

Expected output: The pipeline fails fast if any lint or type errors are found, preventing developers from wasting CI minutes on tests that would fail anyway.

Stage 2 — Unit and Integration Tests

After linting passes, the pipeline runs tests in parallel to maximize speed. Unit tests run in isolation while integration tests spin up dependent services.

test:
    needs: lint
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:16-alpine
        env:
          POSTGRES_DB: test
          POSTGRES_PASSWORD: test
        ports:
          - 5432:5432
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm ci
      - run: npm run test:unit
      - run: npm run test:integration
        env:
          DATABASE_URL: postgres://postgres:test@localhost:5432/test

Expected output: Unit tests complete in seconds. Integration tests connect to the PostgreSQL service container and verify database interactions work correctly.

Stage 3 — Build and Artifact Management

Once tests pass, the pipeline builds deployable artifacts and versions them uniquely for traceability.

build:
    needs: test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npm run build

      - name: Build Docker image
        run: |
          docker build -t myapp:${{ github.sha }} .
          docker tag myapp:${{ github.sha }} myapp:latest

      - name: Save artifact metadata
        run: |
          echo "SHA=${{ github.sha }}" > artifact-metadata.txt
          echo "BUILD_TIME=$(date -u +%Y-%m-%dT%H:%M:%SZ)" >> artifact-metadata.txt
          echo "BRANCH=${{ github.ref_name }}" >> artifact-metadata.txt

      - uses: actions/upload-artifact@v4
        with:
          name: build-output
          path: |
            dist/
            artifact-metadata.txt

Expected output: Each build produces a uniquely tagged Docker image and a metadata file. Artifacts are stored for later stages, ensuring the same artifact that passed tests is what gets deployed.

Stage 4 — Security Scanning

Security gates are embedded in the pipeline to catch vulnerabilities before they reach production. Tools scan dependencies, container images, and code for known exploits.

security-scan:
    needs: build
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Scan dependencies
        run: npm audit --audit-level=high

      - name: Scan Docker image
        uses: aquasecurity/trivy-action@master
        with:
          image-ref: myapp:${{ github.sha }}
          format: table
          exit-code: 1
          severity: CRITICAL,HIGH

Expected behavior: The pipeline blocks the build if any high or critical vulnerabilities are found in dependencies or the container image.

Stage 5 — Deploy with Zero Downtime

The final stage deploys the verified artifact using a strategy that eliminates downtime.

deploy-staging:
    needs: [build, security-scan]
    runs-on: ubuntu-latest
    environment: staging
    steps:
      - name: Deploy to staging
        run: |
          kubectl set image deployment/myapp \
            myapp=myapp:${{ github.sha }} \
            --record
          kubectl rollout status deployment/myapp

  deploy-production:
    needs: deploy-staging
    runs-on: ubuntu-latest
    environment: production
    steps:
      - name: Blue-green deploy
        run: |
          kubectl apply -f k8s/blue-green/
          kubectl set image deployment/myapp-blue \
            myapp=myapp:${{ github.sha }}
          kubectl rollout status deployment/myapp-green
          kubectl label svc/myapp active=green

Expected output: The application deploys to staging first for final validation. Production uses a blue-green strategy — the new version is deployed alongside the old one, and traffic switches only after the new pods pass health checks.

Deployment Strategies

Rolling Update

The default Kubernetes strategy that gradually replaces old pods with new ones. Best for stateless applications where a mix of old and new versions during transition is acceptable.

apiVersion: apps/v1
kind: Deployment
spec:
  replicas: 5
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1
      maxSurge: 1

Expected behavior: One old pod is terminated and one new pod is created at a time. The application remains available throughout, though both versions serve traffic briefly.

Blue-Green Deployment

Two identical environments (blue and green) run simultaneously. Only one serves live traffic at a time, making rollbacks instantaneous.

apiVersion: v1
kind: Service
metadata:
  name: myapp
spec:
  selector:
    app: myapp
    active: green  # Switch this label to roll back

Expected behavior: Switching the service selector from active: green to active: blue instantly routes all traffic to the other environment. Rollback is a single label change.

Canary Deployment

The new version receives a small percentage of traffic initially, gradually increasing as confidence grows.

# Service sends 10% of traffic to canary
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "10"

Expected behavior: 10% of users hit the new version while 90% stay on the old version. If error rates remain low, the weight increases to 50%, then 100%.

Pipeline Optimization Patterns

Caching Dependencies

Installation time drops from minutes to seconds when dependencies are cached across pipeline runs.

- name: Cache npm
        uses: actions/cache@v4
        with:
          path: ~/.npm
          key: ${{ runner.os }}-npm-${{ hashFiles('package-lock.json') }}
          restore-keys: |
            ${{ runner.os }}-npm-
      - run: npm ci

Expected output: Second and subsequent runs restore the cache in under 2 seconds instead of re-downloading all packages.

Parallel Job Execution

Independent stages run simultaneously to minimize total pipeline time.

jobs:
  lint:
  test-unit:
    needs: lint
  test-integration:
    needs: lint
  build:
    needs: [test-unit, test-integration]

Expected output: Unit and integration tests run in parallel after linting. The build stage waits for both to complete before proceeding.

Common Errors

1. Running all stages sequentially: Lint, test, build, and security scan can run in parallel where dependencies allow. Sequential execution wastes minutes on each pipeline run.

2. Deploying untested artifacts: Building the artifact in a deploy job means what you test is different from what you deploy. Build once, promote the same artifact through every environment.

3. Missing security gates: Without security scanning in the pipeline, vulnerable dependencies reach production. Add at minimum dependency and container image scanning.

4. Hardcoded environment configuration: Environment-specific values (database URLs, API keys) should come from secret stores or environment variables, not pipeline YAML files committed to the repository.

5. Ignoring failed health checks: Deploying new pods without verifying they serve traffic correctly leads to silent outages. Always add post-deploy health check probes.

6. No rollback plan: Every deployment must have an automated rollback strategy. Blue-green and canary deployments make this straightforward — just switch traffic back to the previous version.

Practice Questions

1. What is the difference between a rolling update and a blue-green deployment? Answer: A rolling update gradually replaces pods in-place while both versions serve traffic briefly. Blue-green keeps two full environments and switches traffic atomically, enabling instant rollback.

2. Why should you build artifacts once and promote them through environments? Answer: Building in each environment creates variance — different versions of dependencies, different build timestamps. A single artifact tested in staging is identical to the one deployed in production.

3. What purpose do security scanning stages serve in a CI/CD pipeline? Answer: Security scanning catches vulnerable dependencies, misconfigured containers, and hardcoded secrets before they reach production, acting as an automated compliance gate.

4. How does test parallelization reduce pipeline duration? Answer: Independent test suites (unit, integration, lint) run simultaneously on separate runners, completing all checks in the time of the slowest test group instead of sequentially.

Challenge

Build a complete CI/CD pipeline for a microservice application: implement lint, unit test, integration test, build, security scan, and deploy stages. Use parallel execution where possible, cache dependencies, promote a single artifact through staging and production, and implement a blue-green deployment strategy with automated rollback.

Mini Project

Set up a CI/CD pipeline for a sample web application using GitHub Actions. Create a workflow that runs lint and Type Checking on every Pull Request, runs unit and integration tests with a PostgreSQL service container, builds a Docker image tagged with the commit SHA, scans the image with Trivy, deploys to a staging Kubernetes namespace, and deploys to production using a blue-green strategy only after manual approval.

Built by the developers of Doda Browser, DodaZIP, and Durga Antivirus Pro.