Grafana Panel Query Error Fix

In this tutorial, you'll learn about Grafana Panel Query Error Fix. We cover key concepts, practical examples, and best practices.

Grafana shows Panel error: Data source is producing too many data points or Query error: bad syntax — the query is malformed, returns too many series, or the time range produces too many data points.

The Problem

# WRONG — no aggregation, returns every time series individually
rate(node_cpu_seconds_total[5m])

Panel error: Data source is producing too many data points for this panel.
Try reducing the query resolution by increasing the time range step.

Without aggregation, this query returns one series per CPU core per mode per instance. On a 100-node cluster, that's thousands of series — too many for the panel to render.

Step-by-Step Fix

1. Aggregate with by() or without()

# Aggregate by instance only
avg(rate(node_cpu_seconds_total{mode="user"}[5m])) by(instance)

# Aggregate everything to a single value
avg(rate(node_cpu_seconds_total{mode="user"}[5m]))

2. Reduce the time range or increase step

# Use $__interval to auto-adjust based on time range
avg(rate(node_cpu_seconds_total{mode="user"}[$__interval])) by(instance)

In Grafana panel settings: set Min step to 15s or higher. Grafana auto-calculates the step based on the time range — for 7 days, the step might be too small.

3. Fix PromQL syntax

# WRONG — missing rate and aggregation
node_cpu_seconds_total

# RIGHT — proper rate and aggregation
avg(rate(node_cpu_seconds_total{mode="idle"}[5m])) by(instance) * 100

4. Use template variables in panel queries

avg(rate(node_cpu_seconds_total{mode=~"$mode", instance=~"$instance"}[5m])) by(instance)

5. Handle LokiQL log queries

# Count log lines by level
sum by(level) (count_over_time({app="myapp"} |= "error" [5m]))

# Top 10 sources of errors
topk(10, sum by(source) (count_over_time({app="myapp"} |= "error" [5m])))

Expected output:

Panel: CPU Usage by Instance
Query: avg(rate(node_cpu_seconds_total{mode="user"}[5m])) by(instance)
Series: 3 (web-1, web-2, web-3)
✓ Panel rendered successfully

Prevention Tips

• Always aggregate with by() or without() for node-level metrics
• Use $__interval for auto-adapting range vectors
• Set Min step in Grafana panel > Query options
• Test queries in Explore before adding to panels
• Use topk() to limit high-cardinality queries

Common Mistakes with panel error

1. Mixing let bindings with <- bindings in do notation, producing type errors
2. Overlapping type class instances that cause GHC to reject the program with ambiguous dispatch errors
3. Non-exhaustive pattern matches that compile with warnings then crash at runtime

These mistakes appear frequently in real-world GRAFANA code. DodaTech's contributors have identified these patterns through analysis of open-source projects and production systems.

Practice Exercise

Write a pure function that safely divides two integers using Maybe, then test it with edge cases like division by zero and negative numbers.

This exercise reinforces the concepts covered in this guide. Try implementing it before checking online solutions.

FAQ