Kubernetes Networking: CNI Plugins, Calico & Cilium

In this tutorial, you'll learn about Kubernetes Networking: CNI Plugins, Calico & Cilium. We cover key concepts, practical examples, and best practices to help you understand and apply this topic effectively.

Container Network Interface plugins implement Kubernetes networking by managing pod IP assignment, routing, and network policy enforcement across cluster nodes.

What You'll Learn

This tutorial covers CNI plugin architecture, Calico for network policies and BGP routing, Cilium with eBPF for high-performance networking, and comparing plugin features.

Why It Matters

Network performance directly impacts application latency. Choosing the wrong CNI plugin causes throughput bottlenecks, slow pod startup, and limited network policy capabilities.

Real-World Use

Alibaba Cloud runs millions of pods with Calico and Cilium in production. Cilium provides eBPF-based networking that achieves 40 percent better throughput than iptables-based solutions in benchmarks.

CNI Plugin Architecture

When a pod is created, the kubelet calls the CNI plugin to configure the network interface. The plugin assigns an IP address, sets up routing, and reports the result.

# List installed CNI plugins
ls /opt/cni/bin/

# View CNI configuration
cat /etc/cni/net.d/*.conf

Calico: Network Policies and BGP

Calico uses BGP for routing and iptables or eBPF for network policies.

Installing Calico

# Install Calico operator
kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.27/manifests/tigera-operator.yaml

# Configure Calico
kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.27/manifests/custom-resources.yaml

Calico Network Policies

Calico extends Kubernetes NetworkPolicy with additional features.

apiVersion: projectcalico.org/v3
kind: NetworkPolicy
metadata:
  name: allow-api-access
  namespace: production
spec:
  selector: app == "api-server"
  ingress:
  - action: Allow
    protocol: TCP
    source:
      selector: app == "frontend"
    destination:
      ports:
      - 8080
  egress:
  - action: Allow
    protocol: TCP
    destination:
      selector: app == "database"
      ports:
      - 5432

Calico Global Network Policies

Global policies apply across all namespaces.

apiVersion: projectcalico.org/v3
kind: GlobalNetworkPolicy
metadata:
  name: default-deny
spec:
  selector: all()
  types:
  - Ingress
  - Egress

# Apply a Calico policy
kubectl apply -f policy.yaml

# Check Calico status
calicoctl get nodes

# View policy enforcement
calicoctl get networkpolicies -A

Cilium: eBPF-Powered Networking

Cilium uses eBPF for high-performance networking and security.

Installing Cilium

# Install Cilium CLI
curl -L --remote-name-all https://github.com/cilium/cilium-cli/releases/latest/download/cilium-linux-amd64.tar.gz
tar xzvf cilium-linux-amd64.tar.gz
sudo mv cilium /usr/local/bin/

# Install Cilium
cilium install

Cilium Network Policies

Cilium supports L7-aware policies for HTTP, gRPC, and Kafka.

apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: l7-api-policy
  namespace: default
spec:
  endpointSelector:
    matchLabels:
      app: api
  ingress:
  - fromEndpoints:
    - matchLabels:
        app: frontend
    toPorts:
    - ports:
      - port: "8080"
        protocol: TCP
      rules:
        http:
        - method: "GET"
          path: "/api/v1/.*"

Cilium Observability

# Check Cilium status
cilium status

# View network flows in real time
cilium monitor

# Generate Hubble observability dashboard
cilium hubble enable
cilium hubble ui

Comparing CNI Plugins

Calico uses iptables for policy enforcement, supports BGP peering with physical network equipment, and is widely adopted. Cilium uses eBPF for better performance, supports L7 policy enforcement, and provides deep Observability with Hubble.

# Check current CNI plugin
kubectl -n kube-system get pods -l k8s-app=calico-node
kubectl -n kube-system get pods -l k8s-app=cilium

Practice Questions

1. What is the role of a CNI plugin in Kubernetes? It assigns IP addresses to pods and configures network interfaces and routing on each node.

2. How does Calico enforce network policies? Calico uses iptables or eBPF to filter traffic based on policy rules, with Felix as the policy enforcement agent.

3. What advantage does eBPF give Cilium over iptables-based solutions? eBPF provides better throughput, lower latency, and the ability to inspect L7 protocols like HTTP.

4. What is the difference between Calico NetworkPolicy and CiliumNetworkPolicy? Calico policies extend Kubernetes NetworkPolicy with global policies. Cilium policies support L7-aware rules for HTTP and gRPC.

5. How does BGP routing work in Calico? Calico peers BGP with physical routers to distribute pod IP routes across the network fabric.