Structured Logging: JSON Logs, Schema Design, and Field Conventions

In this tutorial, you will learn about Structured Logging: JSON Logs, Schema Design, and Field Conventions. We cover key concepts, practical examples, and best practices to help you master this topic.

Structured logging outputs log entries as structured data (typically JSON) with named fields instead of free-form text. This enables machine Parsing, precise searching, automated alerting, and rich dashboard visualization in log aggregation systems like Elasticsearch and Loki.

flowchart LR
    subgraph Unstructured
        U1["2026-06-28 User 123 logged in from 192.168.1.1"]
    end
    
    subgraph Structured
        S1["{""timestamp"":""2026-06-28"", ""event"":""LOGIN"", ""userId"":123, ""ip"":""192.168.1.1""}"]
    end
    
    Unstructured -->|Hard to search| Pain
    Structured -->|Easy to filter, aggregate, visualize| Gain

What You'll Learn

• JSON log schema design and field conventions
• Log Serialization: objects, errors, and custom serializers
• Standard fields: timestamp, level, message, service, trace, user
• Log context: enrichment with request, environment, and metadata

Why It Matters

Unstructured logs are nearly useless at scale. Searching "find all errors for user 123 in the last hour" requires grep across files. With structured logs, it is a single query: level:error AND userId:123 AND timestamp:>1h-ago.

Real-World Use

A platform team defined a standard log schema across 50 Microservices. Each log entry includes: timestamp, level, message, service, version, environment, correlationId, userId, and duration. This enables cross-service tracing and performance dashboards.

Structured Logging Implementation

Standard Log Schema

const logSchema = {
  // Required fields
  timestamp: '<ISO 8601>',
  level: 'info | warn | error | debug | trace',
  message: 'Human-readable description',
  logger: { name: 'my-app', version: '1.0.0' },

  // Request context (when available)
  request: {
    correlationId: 'uuid',
    method: 'GET',
    url: '/api/users',
    ip: '192.168.1.1',
    userAgent: 'Mozilla/5.0...'
  },

  // Business context
  business: {
    userId: '123',
    tenantId: 'abc',
    eventType: 'ORDER_CREATED',
    duration: '45ms',
    statusCode: 200
  },

  // Error context (when applicable)
  error: {
    name: 'ValidationError',
    message: 'Invalid email format',
    stack: 'Error: ...',
    code: 'VALIDATION_001'
  }
};

function createLogEntry(level, message, context = {}) {
  return {
    timestamp: new Date().toISOString(),
    level,
    message,
    logger: { name: 'my-app', version: process.env.APP_VERSION },
    ...context
  };
}

Expected output:

{"timestamp":"2026-06-28T10:00:00.000Z","level":"info","message":"Order created","logger":{"name":"order-service","version":"1.2.3"},"business":{"userId":"123","eventType":"ORDER_CREATED","duration":"45ms"}}

Custom Serializers for Errors and Objects

const pino = require('pino');

const logger = pino({
  serializers: {
    err: pino.stdSerializers.err,
    req: pino.stdSerializers.req,
    res: pino.stdSerializers.res,
    user: (user) => ({
      id: user.id,
      email: user.email,
      role: user.role
    }),
    error: (error) => ({
      name: error.name,
      message: error.message,
      stack: error.stack,
      code: error.code,
      statusCode: error.statusCode
    }),
    dbQuery: (query) => ({
      sql: query.sql.substring(0, 200),
      params: '[REDACTED]',
      duration: `${query.duration}ms`
    })
  }
});

// Usage
try {
  const user = await db.findUser(id);
  logger.info({ user }, 'Found user');
} catch (err) {
  logger.error({ err, userId: id }, 'Failed to find user');
}

Expected output:

{..., "user": {"id": 123, "email": "user@example.com", "role": "admin"}}
{..., "err": {"type": "Error", "message": "User not found", "stack": "..."}}
Sensitive fields in user object are excluded from serialization.

Log Enrichment Middleware

function enrichLogContext(req, res, next) {
  const start = Date.now();

  // Enrich with request context
  logger.assign({
    correlationId: req.correlationId,
    method: req.method,
    url: req.originalUrl,
    ip: req.ip,
    userAgent: req.headers['user-agent'],
    userId: req.user?.id || 'anonymous',
    tenant: req.headers['x-tenant-id'],
    environment: process.env.NODE_ENV
  });

  res.on('finish', () => {
    logger.assign({
      statusCode: res.statusCode,
      duration: `${Date.now() - start}ms`,
      contentLength: res.get('content-length')
    });
  });

  next();
}

// Usage
app.use(enrichLogContext);
app.get('/api/orders', async (req, res) => {
  logger.info('Fetching orders');
  // All subsequent logs in this request include the enriched context
});

Expected output:

All log entries within the request automatically include correlationId, userId, method, url, and other context fields.

Common Mistakes

• Including too many fields, creating bloated log entries that are expensive to index and store.
• Using inconsistent field names across services (e.g., "user_id" in one, "userId" in another).
• Logging binary data, images, or large objects — log sizes explode and aggregation systems choke.
• Not including essential context (correlationId, userId) — logs become isolated events that cannot be correlated.
• Using dynamic field names (e.g., log[count_${i}] = value) — these create many unique fields that break schema mappings.

Practice Questions

1. What fields should every structured log entry include?
2. Why should field names be consistent across services?
3. How do custom serializers improve log quality?
4. What is the cost trade-off of including many fields in log entries?
5. How does log enrichment work with child loggers?

Challenge

Design a JSON log schema for a multi-service e-commerce platform. Define required fields, business context fields, error fields, and request fields. Create a validation script that verifies all services emit logs matching the schema.

FAQ

Mini Project

Create a structured logging library for internal use. Implement: (1) standard log schema, (2) child loggers with inherited context, (3) custom serializers for errors, requests, and users, (4) log enrichment middleware, (5) log size limits and truncation.

What's Next

Continue to Log Levels to learn effective log level strategies.