Secure Storage: Encrypting Data at Rest and Managing Secrets

In this tutorial, you will learn about Secure Storage: Encrypting Data at Rest and Managing Secrets. We cover key concepts, practical examples, and best practices to help you master this topic.

Secure storage protects sensitive data when it is stored on disk or in databases. This includes encrypting personally identifiable information (PII), payment data, and credentials, as well as managing secrets like API keys, database passwords, and encryption keys.

flowchart TB
    subgraph Data at Rest
        DB[(Database)]
        FS[File System]
        Backup[Backups]
    end
    
    subgraph Protection
        TDE[Transparent Data Encryption]
        CE[Column-Level Encryption]
        FE[File Encryption]
        Secret[Secrets Management]
    end
    
    subgraph Keys
        KMS[AWS KMS / HashiCorp Vault]
        EK[Envelope Encryption]
        Rotation[Key Rotation]
    end
    
    DB --> TDE
    DB --> CE
    FS --> FE
    Backup --> TDE
    Secret --> KMS
    KMS --> EK
    EK --> Rotation

What You'll Learn

• Encryption at rest: TDE, column-level, application-level encryption
• Hashing vs. encryption for sensitive data
• Secrets management with environment variables, Vault, and KMS
• Key management and rotation strategies

Why It Matters

Data breaches often expose stored data. Encryption at rest ensures that even if an attacker gains access to the database or file system, the data remains unreadable. Secrets management prevents hardcoded credentials from being exposed in source code.

Real-World Use

A healthcare application encrypts patient PII (names, SSN, diagnoses) at the column level using AES-256-GCM. Encryption keys are stored in AWS KMS and rotated every 90 days. Database backups are encrypted with a separate KMS key. Secrets (DB passwords, API keys) are stored in HashiCorp Vault.

Secure Storage Implementation

Application-Level Encryption

const crypto = require('crypto');

const ALGORITHM = 'aes-256-gcm';
const KEY = Buffer.from(process.env.ENCRYPTION_KEY, 'hex');

function encrypt(text) {
  const iv = crypto.randomBytes(16);
  const cipher = crypto.createCipheriv(ALGORITHM, KEY, iv);

  let encrypted = cipher.update(text, 'utf8', 'hex');
  encrypted += cipher.final('hex');
  const authTag = cipher.getAuthTag().toString('hex');

  return JSON.stringify({
    iv: iv.toString('hex'),
    data: encrypted,
    tag: authTag
  });
}

function decrypt(encryptedString) {
  const { iv, data, tag } = JSON.parse(encryptedString);
  const decipher = crypto.createDecipheriv(
    ALGORITHM,
    KEY,
    Buffer.from(iv, 'hex')
  );
  decipher.setAuthTag(Buffer.from(tag, 'hex'));

  let decrypted = decipher.update(data, 'hex', 'utf8');
  decrypted += decipher.final('utf8');
  return decrypted;
}

// Column-level encryption middleware
async function encryptPIIColumns(data) {
  if (data.ssn) data.ssn_encrypted = encrypt(data.ssn);
  if (data.email) data.email_hash = crypto.createHash('sha256').update(data.email).digest('hex');
  return data;
}

Expected output:

encrypt('123-45-6789') → '{"iv":"...","data":"...","tag":"..."}' (AES-256-GCM ciphertext)
decrypt(ciphertext) → '123-45-6789'

HashiCorp Vault Integration

const vault = require('node-vault')({
  apiVersion: 'v1',
  endpoint: process.env.VAULT_ADDR,
  token: process.env.VAULT_TOKEN
});

class VaultSecrets {
  static async getSecret(path) {
    try {
      const result = await vault.read(path);
      return result.data.data;
    } catch (err) {
      console.error(`Failed to read secret from ${path}:`, err.message);
      throw err;
    }
  }

  static async rotateSecret(path, newData) {
    try {
      await vault.write(path, newData);
      console.log(`Secret rotated at ${path}`);
    } catch (err) {
      console.error(`Failed to rotate secret at ${path}:`, err.message);
      throw err;
    }
  }
}

// Usage
async function initializeDatabase() {
  const dbSecret = await VaultSecrets.getSecret('secret/data/db-prod');
  const connection = await mysql.createPool({
    host: dbSecret.host,
    user: dbSecret.username,
    password: dbSecret.password,
    database: dbSecret.database
  });
  return connection;
}

Expected output:

Secret values are never hardcoded. DB credentials are fetched from Vault at startup and cached with TTL.

Envelope Encryption with AWS KMS

const { KMSClient, GenerateDataKey, Decrypt } = require('@aws-sdk/client-kms');
const kms = new KMSClient({ region: 'us-east-1' });

class EnvelopeEncryption {
  constructor(keyId) {
    this.keyId = keyId;
    this.dataKeyCache = new Map();
  }

  async generateDataKey() {
    const command = new GenerateDataKey({
      KeyId: this.keyId,
      KeySpec: 'AES_256'
    });
    const response = await kms.send(command);

    return {
      plaintext: response.Plaintext,
      ciphertextBlob: response.CiphertextBlob
    };
  }

  async encrypt(plaintext) {
    const { plaintext: dataKey, ciphertextBlob } = await this.generateDataKey();
    const iv = crypto.randomBytes(12);
    const cipher = crypto.createCipheriv('aes-256-gcm', dataKey, iv);

    let encrypted = cipher.update(plaintext, 'utf8', 'hex');
    encrypted += cipher.final('hex');

    return JSON.stringify({
      ciphertext: encrypted,
      iv: iv.toString('hex'),
      tag: cipher.getAuthTag().toString('hex'),
      encryptedKey: ciphertextBlob.toString('base64')
    });
  }

  async decrypt(payload) {
    const { ciphertext, iv, tag, encryptedKey } = JSON.parse(payload);

    const command = new Decrypt({
      CiphertextBlob: Buffer.from(encryptedKey, 'base64')
    });
    const response = await kms.send(command);
    const plaintextKey = response.Plaintext;

    const decipher = crypto.createDecipheriv(
      'aes-256-gcm',
      plaintextKey,
      Buffer.from(iv, 'hex')
    );
    decipher.setAuthTag(Buffer.from(tag, 'hex'));

    let decrypted = decipher.update(ciphertext, 'hex', 'utf8');
    decrypted += decipher.final('utf8');
    return decrypted;
  }
}

Expected output:

Data encrypted with a unique data key. Data key encrypted with KMS master key. Decryption requires KMS to unwrap the data key.

Common Mistakes

• Storing encryption keys in the same database as encrypted data — keys must be stored separately (e.g., KMS, Vault).
• Using ECB mode encryption, which is deterministic and reveals data patterns.
• Hashing data that needs to be decrypted — use hashing for passwords, encryption for reversible data.
• Not rotating encryption keys regularly — if a key is compromised, all data encrypted with it is exposed.
• Encrypting entire database instead of just sensitive columns — encryption adds overhead; only encrypt what needs protection.

Practice Questions

1. What is the difference between encryption at rest and encryption in transit?
2. Why should PII be encrypted at the column level?
3. What is envelope encryption and why is it useful?
4. How does secrets management differ from storing secrets in environment variables?
5. What is key rotation and why is it important?

Challenge

Design a secure storage system for a user data API. Users have name, email, SSN, and payment info. Encrypt SSN and payment info with AES-256-GCM. Hash email for lookups. Store encryption keys in AWS KMS with monthly rotation. Integrate Vault for database credentials.

FAQ

Mini Project

Build an encrypted user data API. Implement column-level encryption for SSN and payment info. Hash email addresses for lookup queries. Use environment variables with .env encryption (dotenv-safe) for development and Vault integration for production. Write tests verifying encrypted data in the database is unreadable.

What's Next

Continue to JWT Security to learn about securing JSON Web Token implementation.