Secrets Management for APIs — Complete Secure Storage Guide

In this tutorial, you will learn about Secrets Management for APIs. We cover key concepts, practical examples, and best practices to help you master this topic.

Secrets management is the practice of securely storing, accessing, and rotating sensitive information like API keys, database passwords, TLS private keys, and service credentials.

What You'll Learn

You'll learn how to manage secrets without exposing them in code, configuration files, or logs.

Why It Matters

Hardcoded secrets are the leading cause of credential exposure. GitHub scans alone discover millions of secrets in public repositories each year. Proper secrets management prevents catastrophic breaches.

Real-World Use

A fintech startup used a secrets manager (HashiCorp Vault) to rotate database credentials every 24 hours. When a developer's laptop was stolen, the rotating credentials rendered the stolen .env file useless within hours.

flowchart LR
    A[Application Startup] --> B[Read Config]
    B --> C{Secret Reference?}
    C -->|Yes| D[Request from Vault]
    C -->|No| E[Use Value Directly]
    D --> F[Vault Authenticates App]
    F --> G[Vault Returns Secret]
    G --> H[Use in Application]
    H --> I[Rotate on Schedule]
    I --> J[Update Vault]
    J --> K[App Gets New Secret]

Teacher's Mindset

Secrets management is like having a trusted bank vault for your valuables instead of hiding keys under the doormat. The vault has guards (authentication), logs (audit trail), and automatic key changes (rotation).

Implementing Secrets Management

import os
from dotenv import load_dotenv
import hvac  # HashiCorp Vault client

load_dotenv()

# NEVER do this:
# DB_PASSWORD = "supersecret123"

# Instead, use environment variables:
DB_PASSWORD = os.environ.get("DB_PASSWORD")
if not DB_PASSWORD:
    raise ValueError("DB_PASSWORD not set")

# HashiCorp Vault integration
class VaultManager:
    def __init__(self, vault_addr, role_id, secret_id):
        self.client = hvac.Client(
            url=vault_addr,
            auth=hvac.auth.AppRole(role_id, secret_id)
        )

    def get_secret(self, path, key):
        secret = self.client.secrets.kv.v2.read_secret_version(
            path=path
        )
        return secret["data"]["data"][key]

    def rotate_secret(self, path, key, new_value):
        self.client.secrets.kv.v2.create_or_update_secret(
            path=path,
            secret={key: new_value}
        )

vault = VaultManager(
    vault_addr="https://vault.example.com:8200",
    role_id=os.environ["VAULT_ROLE_ID"],
    secret_id=os.environ["VAULT_SECRET_ID"]
)
db_password = vault.get_secret("database/backend", "password")

# Secret scanning pre-commit hook
import re
import sys

SECRET_PATTERNS = [
    r"(?i)(password|secret|token|apikey|api_key)\s*[:=]\s*['\"].+['\"]",
    r"(?i)(-----BEGIN RSA PRIVATE KEY-----)",
    r"(?i)(ghp_|gho_|ghu_|ghs_)[a-zA-Z0-9]{36}",
    r"(?i)sk_live_[0-9a-z]{32}",
    r"(?i)xox[abp]-[0-9a-z-]{24,}"
]

def scan_for_secrets(file_path):
    with open(file_path) as f:
        for line_no, line in enumerate(f, 1):
            for pattern in SECRET_PATTERNS:
                if re.search(pattern, line):
                    print(f"Secret found in {file_path}:{line_no}")
                    return True
    return False

Common Mistakes

Practice Questions

1. Why should secrets never be in source code?
2. What is the difference between environment variables and a vault?
3. How does secret rotation improve security?
4. What is the principle of Least Privilege for secrets?
5. How do you detect secrets in code?

Challenge

Set up HashiCorp Vault in dev mode. Store a database password. Write an application that authenticates to Vault using AppRole and retrieves the password.

FAQ

Mini Project

Deploy HashiCorp Vault in Docker. Configure AppRole authentication. Store three secrets: database URL, API key, and TLS certificate. Write a Python app that retrieves each on startup.

What's Next

Learn about dependency scanning to detect vulnerable libraries in your API.