Conceptual Documentation — Complete Guide

In this tutorial, you will learn about Conceptual Documentation. We cover key concepts, practical examples, and best practices to help you master this topic.

Conceptual API documentation explains the big picture of how your API works, covering fundamental concepts, architecture, data model, design philosophy, and workflows so developers understand the system before they write code.

What You'll Learn

The difference between conceptual and reference documentation, how to explain API architecture and data models, how to describe workflows and use cases, how to write concept docs that reduce support questions, and how to link concepts to reference pages.

Why It Matters

Reference docs tell developers what parameters to send. Conceptual docs tell them why. Without conceptual documentation, developers misuse endpoints, combine features incorrectly, and generate support tickets asking how things work instead of just what to type.

Real-World Use

The Stripe API has a Concepts section explaining payment flows, Webhook delivery, idempotency, and expandable objects. Developers who read the concepts section are 60 percent less likely to create support tickets about payment flow issues.

Conceptual vs Reference Documentation

flowchart TD
  A[API Documentation] --> B[Conceptual Docs]
  A --> C[Reference Docs]
  B --> D[Architecture Overview]
  B --> E[Data Model]
  B --> F[Workflows]
  B --> G[Use Cases]
  B --> H[Design Decisions]
  C --> I[Endpoint Reference]
  C --> J[Parameter Tables]
  C --> K[Response Schemas]
  C --> L[Error Codes]
  A:::current
  classDef current fill:#f90,color:#fff,stroke:#333,stroke-width:2px

Explaining API Architecture

Start with a high-level diagram showing how the system works, then explain each component.

sequenceDiagram
  participant D as Developer
  participant API as DodaTech API
  participant Q as Job Queue
  participant W as Worker
  participant S as Storage

  D->>API: POST /v2/compress
  API->>Q: Enqueue job
  API-->>D: 202 Accepted + job_id
  Q->>W: Dispatch job
  W->>S: Read source file
  W->>W: Compress
  W->>S: Store result
  API->>D: GET /v2/jobs/{id}
  D-->>API: Job completed

Explaining Data Models

Describe the main entities in the system and how they relate to each other.

## Data Model

The DodaTech Compression API has three main entities:

### File

A **File** represents an uploaded or referenced source file. Files have a
unique ID, original name, size, content type, and storage location.

### Job

A **Job** represents a compression operation. When you submit a file for
compression, the API creates a job with a unique ID. Jobs progress through
states: `pending`, `processing`, `completed`, `failed`.

### Archive

An **Archive** is the result of a completed compression job. Archives have
a format (zip, gzip, sevenz), output size, compression ratio, and download
URL that expires after 24 hours.

### Entity Relationships

- A File can have many Jobs (different compression attempts)
- A Job produces exactly one Archive
- An Archive belongs to exactly one Job
- Each File belongs to one organization

Describing Workflows

Explain common workflows step by step. These are the most valuable conceptual docs.

## Compression Workflow

1. **Upload or reference a file** — Provide a public URL or upload directly
   using the multipart upload endpoint.

2. **Submit a compression job** — Choose the output format (zip, gzip, sevenz)
   and optionally set a password for encryption.

3. **Poll for completion** — The API returns immediately with a `job_id`.
   Poll `GET /v2/jobs/{job_id}` until status is `completed` or `failed`.

4. **Download the result** — Once completed, the response includes a
   `download_url`. This URL expires after 24 hours.

### Async Processing for Large Files

Files over 100 MB are processed asynchronously. The API returns HTTP 202
Accepted with a job_id. Poll the job status endpoint to get the result.

```python
import time

def compress_file(file_url, format="zip"):
    # Step 1: Submit job
    response = client.post("/v2/compress", json={
        "file_url": file_url, "format": format
    })
    job_id = response.json()["job_id"]

Explaining Design Decisions

Document why things work the way they do. This builds trust and reduces confusion.

## Why Jobs Instead of Sync Processing?

File compression can take seconds to minutes depending on file size and
compression level. Synchronous processing would cause HTTP timeouts and
force developers to set unreasonably long timeouts.

Instead, compression is always async with job polling. This gives you:

- Predictable response times (under 100ms for the submit call)
- Progress tracking during long operations
- Parallel processing of multiple files
- Retry capability for failed jobs

Linking Concepts to Reference

Every conceptual section should link to the relevant reference pages.

## Understanding Pagination

All list endpoints use cursor-based pagination. Instead of page numbers,
you pass a `cursor` parameter returned from the previous response.

**Benefits of cursor-based pagination:**
- Stable results when new items are added during iteration
- Better performance for large datasets
- No off-by-one errors with page counting

See the [reference page](/api/files/list) for parameter details and
the [error codes](/api/errors) for pagination-related errors.

Common Mistakes

1. Writing Concepts as Reference

Explaining parameter formats in conceptual docs duplicates the reference and gets outdated quickly. Keep concepts high-level and link to reference for details.

2. No Diagrams

Wall-of-text explanations of architecture are hard to follow. Every conceptual doc should include at least one Mermaid diagram showing the system flow.

3. Assuming Too Much Context

Using terms like webhook, idempotency key, or cursor pagination without explanation excludes less experienced developers. Define every concept in plain language.

4. Only Writing for Happy Path

Showing only the ideal workflow leaves developers unprepared for errors. Include failure scenarios and recovery steps in workflow documentation.

5. Outdated Workflow Descriptions

Conceptual docs that describe old versions of the API erode trust. Review and update conceptual docs whenever the API behavior changes.

6. No Real-World Examples

Abstract explanations without concrete use cases leave developers wondering when to use each feature. Include at least one real-world scenario per concept.

7. Too Many Links Too Early

Overloading the first paragraph with links to reference pages distracts from the conceptual explanation. Use links sparingly and place them after explaining the concept.

Practice Questions

1. What is the difference between conceptual and reference documentation?

Conceptual docs explain how and why the API works, describing architecture, data models, and workflows. Reference docs list specific endpoints, parameters, and schemas for lookup.

2. Why are diagrams important in conceptual documentation?

Diagrams help readers understand system architecture and data flow faster than text alone. A good diagram replaces paragraphs of explanation.

3. How do conceptual docs reduce support tickets?

When developers understand how the system works conceptually, they make fewer mistakes combining features, handling errors, and designing their integration architecture.

4. What should every workflow description include?

Step-by-step instructions, code examples for each step, error handling guidance, and links to the relevant reference documentation for parameters.

5. Challenge: Write a conceptual documentation page for a feature of a public API you use. Include a Mermaid diagram, data model explanation, and workflow description with at least one code example.

FAQ

Mini Project: Write a Conceptual Doc

Write a complete conceptual documentation page for a system you know well. Include an architecture diagram (Mermaid), data model explanation with three entities and their relationships, a step-by-step workflow, a design decision explanation, and links to reference pages.

What's Next

Concepts explain how the API works. Now learn to write Getting Started Guides that onboard new users quickly. Then study Authentication Documentation for securing your API.