Link Checking with HTMLProofer — Complete Guide

Automated link checking prevents broken links in documentation. Learn how to configure HTMLProofer, lychee, and other tools to check internal and external links in your documentation pipeline.

What You'll Learn

You will learn how to set up automated link checking, how to handle external link validation, how to ignore false positives, and how to integrate link checking into your CI pipeline.

Why It Matters

Broken links frustrate readers and erode trust. Studies show that users encountering a 404 are 60 percent less likely to return to the site. Automated link checking catches broken links before readers find them.

Real-World Use

DodaTech runs HTMLProofer on every build, checking all internal links and a subset of external links. The tool catches moved pages, renamed files, and expired external references before deployment.

flowchart LR
  A[Build Documentation] --> B[HTMLProofer]
  B --> C[Check Internal Links]
  B --> D[Check External Links]
  B --> E[Check Images]
  C --> F{Valid?}
  D --> F
  E --> F
  F -->|Yes| G[Deploy]
  F -->|No| H[Fail CI]
  A:::current
  classDef current fill:#f90,color:#fff,stroke:#333,stroke-width:2px

Setting Up HTMLProofer

Install HTMLProofer:

gem install html-proofer

Run it against the built documentation:

hugo --gc --minify
htmlproofer public/ --check-html --check-img-http

Expected output on success:

Running 29 checks on 142 files...
HTML-Proofer finished successfully.

Expected output on failure:

- public/getting-started/index.html
  *  linking to internal path #installation that does not exist (line 34)
  *  image public/images/screenshot.png does not exist (line 67)
- public/api/index.html
  *  external link https://example.com/docs/ failed with 404 (line 89)

Configuration Options

htmlproofer public/ \
  --disable-external \
  --check-html \
  --check-img-http \
  --allow-hash-href \
  --alt-ignore "icon" \
  --url-ignore "/fonts.googleapis.com/,/linkedin.com/"

Using lychee (Rust-based alternative)

lychee is a faster alternative written in Rust:

# Install lychee
brew install lychee  # macOS
# or: cargo install lychee  # from source

# Run link checking
lychee public/

Expected output:

🔍 142 links checked (135 OK, 5 errors, 2 excluded)
📝 Found 5 errors:
  - public/index.html → https://example.com/docs/ 404 Not Found
  - public/guide.html → images/logo.png File not found

Ignoring False Positives

Some external sites block automated crawlers. Add ignore patterns for known issues:

# .lycheeignore
https://linkedin.com
https://fonts.googleapis.com
http://localhost

Integrating into CI

- name: Link check
  run: |
    hugo --gc --minify
    htmlproofer public/ --disable-external --check-html

For external link checking on a schedule:

- name: Weekly external link check
  schedule:
    - cron: "0 6 * * 1"  # Every Monday at 6 AM
  steps:
    - uses: actions/checkout@v4
    - run: |
        hugo --gc --minify
        htmlproofer public/ --check-html

Common Mistakes

1. Skipping Link Checking in CI

Without automated checking, broken links accumulate until someone manually audits them. Run link checking on every build.

2. Checking External Links on Every Build

External link checking is slow and unreliable. Run internal link checks on every build and external checks on a weekly schedule.

3. Ignoring Image Links

Broken images look unprofessional. Configure HTMLProofer to check image references with --check-img-http.

4. Not Testing the Built Output

Link checkers must run against the generated HTML output, not the Markdown source. Relative links may resolve differently after building.

5. Using Fragments Without Validation

Links to page sections like #installation break when heading IDs change. Check that fragment targets exist in the output.

Practice Questions

1. Why should link checking run against the built HTML output, not the Markdown source?

Relative links and shortcodes may resolve differently after the static site generator processes them.

2. What is the difference between internal and external link checking?

Internal links point to pages within the same site. External links point to other domains. Internal checks are faster and more reliable.

3. How do you handle external sites that block automated link checkers?

Add them to an ignore list. Run external checks separately with user-agent headers that identify the checker.

4. What does the --disable-external flag do?

It tells HTMLProofer to skip external link validation and only check internal links and images.

5. Challenge: Set up HTMLProofer to check links in a built documentation site. Create a CI workflow that runs internal link checking on every PR and external checking on a weekly schedule.

FAQ

Mini Project

Create a documentation project with at least five pages that link to each other and to external sites. Build the site, run HTMLProofer to find broken links, fix them, and set up a GitHub Actions workflow that runs link checking on every PR.

What's Next

Link checking ensures your references work. Next, learn about the Review Process for documentation to establish a PR-based review workflow that catches content issues before merge.