How to Fix CoreDNS Prometheus Metrics Not Scraping

In this tutorial, you'll learn about How to Fix CoreDNS Prometheus Metrics Not Scraping. We cover key concepts, practical examples, and best practices.

Prometheus cannot scrape CoreDNS metrics — the /metrics endpoint returns 404 Not Found or Prometheus shows the target as DOWN — the Prometheus plugin is not enabled in CoreDNS.

The Problem

$ curl http://localhost:9153/metrics
404 Not Found

Step-by-Step Fix

Step 1: Enable Prometheus plugin in Corefile

apiVersion: v1
kind: ConfigMap
metadata:
  name: coredns
  namespace: kube-system
data:
  Corefile: |
    .:53 {
        errors
        health
        kubernetes cluster.local
        prometheus :9153
        forward . /etc/resolv.conf
        cache 30
    }

Step 2: Check metrics endpoint

kubectl port-forward -n kube-system pod/coredns-xxxxx 9153:9153 &
curl http://localhost:9153/metrics | head -20

Expected:

# HELP coredns_dns_requests_count Counter of DNS requests made per zone, proto and family.
# TYPE coredns_dns_requests_count counter
coredns_dns_requests_count{proto="udp",family="1",server="dns://:53",zone="."} 123

Step 3: Configure Prometheus ServiceMonitor

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: coredns
  namespace: kube-system
spec:
  selector:
    matchLabels:
      k8s-app: kube-dns
  endpoints:
  - port: metrics
    interval: 15s

Step 4: Verify target in Prometheus UI

# In Prometheus UI, go to Status > Targets
# Look for coredns endpoint

Step 5: Add prometheus.io annotations

apiVersion: v1
kind: Service
metadata:
  annotations:
    prometheus.io/scrape: "true"
    prometheus.io/port: "9153"

Prevention Tips

• Always enable the <a href="/devops/prometheus-grafana/">prometheus</a> plugin in CoreDNS Corefile
• Expose port 9153 in the CoreDNS service
• Use ServiceMonitor for automatic discovery in Prometheus Operator
• Monitor CoreDNS metrics for DNS query volume and errors

Common Mistakes with prometheus

1. Misunderstanding that String is [Char] with poor performance for large text operations
2. Using foldl instead of foldl' causing stack overflow on large lists
3. Forgetting deriving (Show, Eq) on custom data types needed for debugging

These mistakes appear frequently in real-world COREDNS code. DodaTech's contributors have identified these patterns through analysis of open-source projects and production systems.

Practice Exercise

Write a pure function that safely divides two integers using Maybe, then test it with edge cases like division by zero and negative numbers.

This exercise reinforces the concepts covered in this guide. Try implementing it before checking online solutions.

FAQ