Grafana Loki LogQL Pipeline Stage Not Applying Fix

In this tutorial, you'll learn about Grafana Loki LogQL Pipeline Stage Not Applying Fix. We cover key concepts, practical examples, and best practices.

Your Grafana Loki LogQL pipeline stage appears to have no effect — the | json parser does not extract fields, or the | label_format rename is not reflected in results. Pipeline stages must be chained in the correct order, and each stage operates on the output of the previous stage.

The Problem

{job="myapp"} | json | label_format level=severity | level = "error"

The label_format renames level to severity, but the filter level = "error" references the old name. The filter runs after the rename — since level no longer exists, the filter matches nothing.

Step-by-Step Fix

1. Order stages correctly

{job="myapp"}
  | json                         # 1. Parse JSON → extracts fields
  | level = "error"              # 2. Filter by original field
  | label_format severity=level  # 3. Rename after filtering

2. Use line_format for output transformation

{job="myapp"}
  | json
  | level = "error"
  | line_format "{{.msg}} ({{.duration_ms}}ms)"

3. Chain multiple parsers

{job="myapp"}
  | logfmt                        # Parse key=value format
  | level = "error"               # Filter by level
  | regex "(?P<path>/\\w+/\\w+)"  # Extract path from message
  | line_format "{{.path}}"       # Display only path

4. Use unwrap in metric queries

// Pipeline stage before unwrap
sum by (level) (
  rate(
    {job="myapp"}
      | json
      | duration_ms > 0
      | unwrap duration_ms [5m]
  )
)

5. Combine drop and keep stages

{job="myapp"}
  | json
  | level = "error"    # Keep only error logs
  | drop level         # Remove level field from output
  | line_format "{{.msg}}"  # Show only message

Expected output:

Before: {"level":"error","msg":"timeout","duration_ms":5000}
After pipeline:
  | json → {level, msg, duration_ms}
  | level = "error" → keeps this line
  | line_format "{{.msg}} ({{.duration_ms}}ms)"
  → "timeout (5000ms)"

Prevention Tips

• Order pipeline stages: parse → filter → transform → format
• Use | line_format to reshape output without affecting filtering
• Test pipeline step by step in Grafana Explore
• Use | json --non-strict to handle malformed JSON lines
• Reference field names after they are created by parsers

Common Mistakes with loki parser

1. Non-exhaustive pattern matches that compile with warnings then crash at runtime
2. Misunderstanding that String is [Char] with poor performance for large text operations
3. Using foldl instead of foldl' causing stack overflow on large lists

These mistakes appear frequently in real-world GRAFANA code. DodaTech's contributors have identified these patterns through analysis of open-source projects and production systems.

Practice Exercise

Write a pure function that safely divides two integers using Maybe, then test it with edge cases like division by zero and negative numbers.

This exercise reinforces the concepts covered in this guide. Try implementing it before checking online solutions.

FAQ