Documentation Build Pipeline — Complete Guide

A documentation build pipeline automates converting Markdown to HTML, running linters, checking links, and deploying the output. Learn how to build a pipeline that runs on every commit and pull request.

What You'll Learn

You will learn the stages of a documentation build pipeline, how to configure each stage, and how to tie everything together in a CI/CD workflow.

Why It Matters

A manual build process guarantees drift. The deployed documentation will differ from the Repository. An automated pipeline ensures that what is in the repository is what is deployed, and it catches errors before readers see them.

Real-World Use

DodaTech runs a build pipeline that processes 15,000+ pages. It runs markdownlint, cspell, HTMLProofer for link checking, and Hugo for building. The entire pipeline completes in under five minutes.

flowchart LR
  A[Git Push] --> B[CI Trigger]
  B --> C[Install Dependencies]
  C --> D[Lint: markdownlint]
  D --> E[Spell Check: cspell]
  E --> F[Link Check: HTMLProofer]
  F --> G[Build: Hugo]
  G --> H[Deploy to Netlify]
  A:::current
  classDef current fill:#f90,color:#fff,stroke:#333,stroke-width:2px

Pipeline Stages

Stage 1: Install Dependencies

npm install -g markdownlint-cli2 cspell
pip install htmlproofer

Stage 2: Lint with markdownlint

markdownlint-cli2 content/

Expected output on success: No output (exit code 0).

Expected output on failure:

content/api/authentication.md: 1:1 MD041/first-line-heading/first-line-h1
First line in a file should be a top-level heading

Stage 3: Spell Check with cspell

cspell "content/**/*.md"

Expected output on success: No output (exit code 0).

Expected output on failure:

content/api/authentication.md:23:15 - Unknown word (authenticaion)
  Suggestions: authentication, authentications

Stage 4: Link Check with HTMLProofer

# Build first, then check links in the output
hugo --gc --minify
htmlproofer public/ --check-html --check-img-http

Expected output on success:

Running 29 checks on 142 files...
HTML-Proofer finished successfully.

Stage 5: Build

hugo --gc --minify

Expected output:

Start building sites ...
hugo v0.134.0
Total in 2342 ms

Stage 6: Deploy

- name: Deploy to Netlify
  uses: nwtgck/actions-netlify@v2
  with:
    publish-dir: public
    production-branch: main

Complete Pipeline Configuration

# .github/workflows/docs-pipeline.yml
name: Documentation Pipeline
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - uses: peaceiris/actions-hugo@v2
        with:
          hugo-version: 0.134.0
      - name: Install tools
        run: |
          npm install -g markdownlint-cli2 cspell
      - name: Lint
        run: markdownlint-cli2 content/
      - name: Spell check
        run: cspell "content/**/*.md"
      - name: Build
        run: hugo --gc --minify
      - name: Link check
        run: |
          npm install -g htmlproofer
          htmlproofer public/ --disable-external
      - name: Deploy
        if: github.ref == 'refs/heads/main'
        run: npx netlify-cli deploy --dir=public --prod

Local Build Commands

Run the same checks locally before pushing:

# Install tools globally
npm install -g markdownlint-cli2 cspell htmlproofer

# Run all checks
markdownlint-cli2 content/
cspell "content/**/*.md"
hugo --gc --minify
htmlproofer public/ --disable-external

Common Mistakes

1. Skipping the Build Step in CI

Some pipelines run linters but skip the actual build. A file with broken shortcodes or invalid frontmatter will only fail during building.

2. Running Only External Link Checking

External link checks are slow and unreliable (sites may block crawlers). Run internal link checks in CI and external checks on a schedule.

3. Ignoring Exit Codes

A script that prints errors but exits with code 0 will not fail the CI. Always check that tools return non-zero on failure.

4. Building Without Cache

Running git gc --minify on every build is slower but ensures a clean build. Use cached builds only when you understand the trade-offs.

5. Hardcoding Deployment Credentials

Store deployment tokens in GitHub secrets, not in the repository. Hardcoded credentials leak when the repository is forked.

Practice Questions

1. What are the six stages of a documentation build pipeline?

Install dependencies, lint, spell check, link check, build, deploy.

2. Why should linting run before building?

Linting catches formatting and style errors early. A file with lint errors may still build, but fixing earlier saves time.

3. What tool checks for broken links in built documentation?

HTMLProofer, broken-link-checker, or lychee.

4. How do you ensure the CI pipeline fails when a tool finds errors?

Tools return non-zero exit codes on errors. CI systems treat non-zero exits as failures.

5. Challenge: Create a GitHub Actions workflow that builds a Hugo documentation site, runs markdownlint, checks spelling with cspell, and deploys to Netlify or GitHub Pages.

FAQ

Mini Project

Create a GitHub Actions workflow for a documentation repository that runs markdownlint and cspell on pull requests, builds the site with Hugo, runs HTMLProofer for internal link checking, and deploys only when changes merge to main.

What's Next

Now that you have a build pipeline, learn how to set up CI/CD for Docs with different platforms. Then explore Linting Docs with Vale and markdownlint.