Multi-Cluster Kubernetes: Management, Federation & Service Mesh

In this tutorial, you'll learn about Multi. We cover key concepts, practical examples, and best practices to help you understand and apply this topic effectively.

Multi-cluster Kubernetes architectures distribute workloads across multiple clusters for geographic latency reduction, disaster recovery, provider diversity, and isolation of failure domains beyond what a single cluster can achieve.

What You'll Learn

This tutorial covers multi-cluster management strategies including Cluster API for provisioning, Submariner for cross-cluster networking, service mesh federation with Istio multicluster, and workload placement patterns for active-passive and active-active topologies.

Why It Matters

Single-cluster deployments create a single point of failure at the cluster level. Multi-cluster architectures provide region-level availability, reduce latency for global users by placing workloads closer to traffic sources, and enable workload Migration between providers without application changes.

Real-World Use

Spotify runs over 300 Kubernetes clusters managed by Cluster API, with automatic cluster provisioning and lifecycle management. The New York Times runs applications across multiple Cloud Computing regions using Istio multicluster mesh for global traffic routing and failover.

Multi-Cluster Topologies

Choose the right topology for your use case.

graph TD
  subgraph "Hub Cluster"
    A[Management Plane]
  end
  subgraph "Region US-East"
    B[Cluster 1]
  end
  subgraph "Region EU-West"
    C[Cluster 2]
  end
  subgraph "Region AP-Southeast"
    D[Cluster 3]
  end
  A --> B
  A --> C
  A --> D
  B <--> C
  C <--> D

Expected output: a hub-and-spoke diagram showing a management cluster controlling three regional workload clusters with inter-cluster peering.

Cluster API for Declarative Provisioning

Cluster API creates and manages Kubernetes clusters as custom resources.

# Install Cluster API on the management cluster
clusterctl init --infrastructure aws

# Verify Cluster API pods
kubectl -n capa-system get pods

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: prod-us-east
spec:
  clusterNetwork:
    pods:
      cidrBlocks:
      - 192.168.0.0/16
    services:
      cidrBlocks:
      - 172.16.0.0/20
  infrastructureRef:
    apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
    kind: AWSCluster
    name: prod-us-east
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
kind: AWSCluster
metadata:
  name: prod-us-east
spec:
  region: us-east-1
  network:
    vpc:
      cidrBlock: 10.0.0.0/16

# Provision the cluster
kubectl apply -f cluster-us-east.yaml

# Watch provisioning progress
clusterctl describe cluster prod-us-east

# Retrieve kubeconfig
clusterctl get kubeconfig prod-us-east > prod-us-east.kubeconfig

Expected output: Cluster API creates VPC, subnets, security groups, EC2 instances, and bootstraps Kubernetes. The describe command shows the cluster status transitioning from Provisioning to Ready.

Cross-Cluster Networking with Submariner

Submariner connects pods and services across clusters with encrypted tunnels.

# Install Submariner with subctl
subctl deploy-broker --kubeconfig management.kubeconfig

# Join clusters to the broker
subctl join --kubeconfig cluster1.kubeconfig broker-info.subm --clusterid us-east
subctl join --kubeconfig cluster2.kubeconfig broker-info.subm --clusterid eu-west

# Export a service across clusters
apiVersion: submariner.io/v1
kind: ServiceExport
metadata:
  name: api-service
  namespace: production
---
# Import the service on another cluster
apiVersion: submariner.io/v1
kind: ServiceImport
metadata:
  name: api-service
  namespace: production
spec:
  type: SuperClusterIP
  ports:
  - port: 8080

# Verify cross-cluster connectivity
kubectl --kubeconfig cluster1.kubeconfig get pods -l app=api
kubectl --kubeconfig cluster2.kubeconfig run test --image=busybox --rm -it -- \
  wget -qO- http://api-service.production.svc.clusterset.local:8080/health

Expected output: pods in cluster2 can reach the api-service running in cluster1 through the Submariner SuperClusterIP, with traffic encrypted by IPsec tunnels.

Istio Multicluster Mesh

Federate service meshes across cluster boundaries.

# Install Istio on each cluster with shared control plane
istioctl install --set profile=minimal \
  --set values.global.meshID=global-mesh \
  --set values.global.multiCluster.clusterName=us-east \
  --kubeconfig us-east.kubeconfig

istioctl install --set profile=minimal \
  --set values.global.meshID=global-mesh \
  --set values.global.multiCluster.clusterName=eu-west \
  --kubeconfig eu-west.kubeconfig

# Enable cross-cluster service discovery
istioctl x create-remote-secret --kubeconfig us-east.kubeconfig \
  --name=us-east | kubectl apply -f - --kubeconfig eu-west.kubeconfig

# Global service entry for cross-cluster routing
apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
  name: cross-cluster-api
spec:
  hosts:
  - api.global-mesh.svc.cluster.local
  addresses:
  - 240.0.0.1
  ports:
  - number: 8080
    name: http
    protocol: HTTP
  resolution: DNS
  location: MESH_INTERNAL
  endpoints:
  - address: api-service.production.svc.clusterset.local
    ports:
      http: 8080

Workload Placement Strategies

Place workloads across clusters based on latency, cost, or compliance.

# Deploy workloads to specific clusters
apiVersion: scheduling.x-k8s.io/v1alpha1
kind: ClusterSelector
metadata:
  name: eu-only
spec:
  clusters:
  - eu-west
  - eu-central
---
apiVersion: v1
kind: Pod
metadata:
  name: gdpr-app
  annotations:
    scheduler.alpha.kubernetes.io/cluster-selector: eu-only
spec:
  containers:
  - name: app
    image: myapp:1.0

# Create namespace on specific cluster
kubectl --kubeconfig eu-west.kubeconfig create namespace gdpr-workloads

# Deploy with cluster-aware scheduling
kubectl apply --context=us-east -f deployment.yaml

# Verify pod distribution across clusters
kubectl get pods --all-clusters

Practice Questions

1. What is the difference between hub-spoke and peer-to-peer multi-cluster topologies? Hub-spoke uses a central management cluster to provision and manage workload clusters, with limited direct communication between workload clusters. Peer-to-peer connects all clusters in a mesh, enabling direct workload communication across any cluster.

2. How does Submariner enable cross-cluster service discovery? Submariner uses a broker cluster to exchange service metadata and establishes IPsec tunnels between clusters. ServiceImport and ServiceExport resources expose services across clusters with a SuperClusterIP that routes to the appropriate cluster.

3. What is the purpose of mTLS in a multicluster mesh? mTLS ensures that all cross-cluster traffic is encrypted and authenticated, preventing unauthorized access and man-in-the-middle attacks between clusters that may traverse the public internet.

Frequently Asked Questions

Built by the developers of Doda Browser, DodaZIP, and Durga Antivirus Pro.