ELK Stack on Kubernetes: ECK Operator & Helm

In this tutorial, you'll learn about ELK Stack on Kubernetes: ECK Operator & Helm. We cover key concepts, practical examples, and best practices to help you understand and apply this topic effectively.

The Elastic Cloud on Kubernetes (ECK) operator automates the deployment, management, and scaling of Elasticsearch, Kibana, and other Elastic Stack components on Kubernetes, handling configuration, backups, and self-healing through custom resources.

What You'll Learn

In this tutorial, you will deploy ECK on a Kubernetes cluster, create an Elasticsearch cluster using custom resources, deploy Kibana connected to that cluster, and configure scaling and persistence.

Why It Matters

Running the ELK Stack manually on Kubernetes requires managing StatefulSets, persistent volumes, service discovery, and configuration for each component. ECK abstracts all of this. You declare the desired state in a YAML custom resource, and the operator handles the rest: pod creation, volume mounts, TLS certificate generation, and rolling upgrades.

Real-World Use

DodaZIP runs its ELK Stack on Kubernetes using ECK. The Elasticsearch cluster spans 3 availability zones with 6 data nodes. Kibana is exposed through an ingress with OAuth authentication. When a node fails, the operator automatically replaces the pod and re-attaches the persistent volume. The team manages the entire stack through a few Kubernetes manifests instead of complex Configuration Management.

Step 1: Install the ECK Operator

Install the ECK operator using the official manifest:

kubectl create -f https://download.elastic.co/downloads/eck/2.14.0/crds.yaml
kubectl apply -f https://download.elastic.co/downloads/eck/2.14.0/operator.yaml

Expected output:

elasticsearch.elasticsearch.k8s.elastic.co created
kibana.kibana.k8s.elastic.co created
...

Verify the operator is running:

kubectl get pods -n elastic-system

Step 2: Create an Elasticsearch Cluster

Define an Elasticsearch cluster custom resource:

apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: logs-cluster
spec:
  version: 8.14.0
  nodeSets:
    - name: master
      count: 3
      config:
        node.roles: ["master"]
    - name: data
      count: 3
      config:
        node.roles: ["data", "ingest"]
      volumeClaimTemplates:
        - metadata:
            name: elasticsearch-data
          spec:
            storageClassName: standard
            accessModes: ["ReadWriteOnce"]
            resources:
              requests:
                storage: 500Gi

Apply the resource:

kubectl apply -f elasticsearch.yaml

ECK creates the pods, persistent volumes, and a service for the cluster.

Step 3: Deploy Kibana

Create a Kibana custom resource connected to the Elasticsearch cluster:

apiVersion: kibana.k8s.elastic.co/v1
kind: Kibana
metadata:
  name: logs-kibana
spec:
  version: 8.14.0
  count: 2
  elasticsearchRef:
    name: logs-cluster
  config:
    server.publicBaseUrl: https://kibana.example.com
  http:
    tls:
      selfSignedCertificate:
        disabled: true

ECK automatically configures Kibana to connect to the logs-cluster Elasticsearch cluster and handles TLS.

Step 4: Access Kibana

ECK creates a service for Kibana. Port-forward or configure an ingress:

kubectl port-forward service/logs-kibana-kb-http 5601:5601

Get the elastic user password from the Kubernetes secret:

kubectl get secret logs-cluster-es-elastic-user -o=jsonpath='{.data.elastic}' | base64 --decode

Log in to Kibana at https://localhost:5601 with user elastic and the retrieved password.

Step 5: Scale the Cluster

To scale, update the count in the Elasticsearch custom resource:

spec:
  nodeSets:
    - name: data
      count: 6  # increased from 3

Apply the change. ECK automatically adds new pods and rebalances shards.

Common Mistakes

1. Insufficient Persistent Storage

Log data grows fast. Plan storage based on daily log volume and retention. Use volumeClaimTemplates to request adequate storage.

2. Not Setting Resource Limits

Without resource limits, Elasticsearch pods can consume all node resources, affecting other workloads. Always set requests and limits for CPU and memory.

3. Using Default Storage Class for Production

Default storage classes may not provide the IOPS Elasticsearch needs. Use a storage class with SSDs and sufficient performance.

4. Exposing Kibana Without TLS

ECK generates self-signed TLS certificates by default. For production, use a proper certificate or disable self-signed and configure an ingress with TLS.

5. Ignoring Pod Disruption Budgets

Without PDBs, node maintenance can take down multiple Elasticsearch pods simultaneously. Configure PDBs to ensure at least one replica per shard remains available.

Practice Questions

1. What is the ECK operator? Elastic Cloud on Kubernetes operator that automates deployment, management, and scaling of Elasticsearch, Kibana, and other Elastic Stack components.

2. How do you define an Elasticsearch cluster with ECK? Create a custom resource of kind Elasticsearch specifying version, node sets, roles, and persistent storage.

3. How does Kibana connect to Elasticsearch in ECK? Through the elasticsearchRef field that references the Elasticsearch custom resource by name.

4. How do you scale an Elasticsearch cluster in ECK? Update the count field in the node set specification and reapply the custom resource.

5. Challenge: Deploy a production-ready ELK Stack on Kubernetes with 3 master nodes, 5 data nodes with 1TB of SSD storage each, 2 Kibana instances behind an ingress with TLS, and pod disruption budgets for all components.

What's Next

Optimize your ELK Stack for production with performance tuning, index lifecycle management, and cluster monitoring.

Built by the developers of Doda Browser, DodaZIP, and Durga Antivirus Pro.