Cloud Kubernetes Operators — AWS Operators, Azure Operators & GKE Operators Guide

In this tutorial, you'll learn about Cloud Kubernetes Operators. We cover key concepts, practical examples, and best practices to help you understand and apply this topic effectively.

Cloud Kubernetes operators extend the Kubernetes API to manage cloud resources — S3 buckets, RDS instances, Pub/Sub topics — directly through kubectl and Kubernetes manifests instead of cloud consoles or CLI scripts.

What You'll Learn

You'll learn how to install and use ACK, ASO, and Config Connector to provision cloud resources from Kubernetes, manage resource lifecycle with controllers, and implement GitOps for cloud infrastructure.

Why It Matters

Managing cloud resources separately from Kubernetes creates a split-brain problem. Your app config is in YAML, but your database is in Terraform. Operators unify everything under the Kubernetes API. DodaZIP uses ACK to provision S3 buckets and DynamoDB tables alongside application deployments.

Real-World Use

A platform team uses Config Connector to provision a new Cloud Spanner instance and Pub/Sub topic automatically when a new microservice namespace is created in GKE. No Terraform runs, no console access — just kubectl apply.

Operator Architecture

flowchart LR
  A[kubectl apply -f resource.yaml] --> B[Kubernetes API Server]
  B --> C[Operator Controller]
  C --> D[Cloud Provider API]
  D --> E["S3 Bucket / RDS / Pub/Sub"]
  C --> F[Update Resource Status]
  F --> G[Ready for Application]
  style C fill:#48f,color:#fff
  style D fill:#f90,color:#fff

AWS Controllers for Kubernetes (ACK)

ACK lets you manage AWS resources via Kubernetes CRDs.

# Install ACK for S3
helm install --create-namespace --namespace ack-system \
  ack-s3-controller \
  oci://public.ecr.aws/aws-controllers-k8s/s3-chart \
  --version v1.0.0

# Create an S3 bucket using Kubernetes
kubectl apply -f - <<EOF
apiVersion: s3.services.k8s.aws/v1alpha1
kind: Bucket
metadata:
  name: my-app-data
spec:
  name: my-app-data-12345
tags:
    - key: Environment
      value: Production
EOF

# Create a DynamoDB table
kubectl apply -f - <<EOF
apiVersion: dynamodb.services.k8s.aws/v1alpha1
kind: Table
metadata:
  name: user-sessions
spec:
  tableName: user-sessions
  attributeDefinitions:
    - attributeName: sessionId
      attributeType: S
  keySchema:
    - attributeName: sessionId
      keyType: HASH
  billingMode: PAY_PER_REQUEST
EOF

Azure Service Operator (ASO)

ASO manages Azure resources through Kubernetes CRDs.

# Install ASO
helm install aso-v2 \
  oci://mcr.microsoft.com/azure-service-operator/v2/charts/asov2 \
  --create-namespace --namespace azure-service-operator-system

# Create an Azure Storage account
kubectl apply -f - <<EOF
apiVersion: storage.azure.com/v1api20230101
kind: StorageAccount
metadata:
  name: dodatechstorage
  namespace: default
spec:
  location: eastus
  kind: StorageV2
  sku:
    name: Standard_LRS
tags:
    Environment: Production
EOF

# Create a resource group
kubectl apply -f - <<EOF
apiVersion: resources.azure.com/v1api20200601
kind: ResourceGroup
metadata:
  name: prod-rg
spec:
  location: eastus
EOF

GCP Config Connector

Config Connector manages GCP resources via Kubernetes CRDs.

# Install Config Connector
gcloud container clusters update my-cluster \
  --region us-central1 \
  --update-addons ConfigConnector=ENABLED

# Create a Pub/Sub topic
kubectl apply -f - <<EOF
apiVersion: pubsub.cnrm.cloud.google.com/v1beta1
kind: PubSubTopic
metadata:
  name: order-events
  annotations:
    cnrm.cloud.google.com/project-id: my-project
spec:
  messageStoragePolicy:
    allowedPersistenceRegions:
      - us-central1
      - us-east1
EOF

# Create a Cloud SQL instance
kubectl apply -f - <<EOF
apiVersion: sql.cnrm.cloud.google.com/v1beta1
kind: SQLInstance
metadata:
  name: prod-database
  annotations:
    cnrm.cloud.google.com/project-id: my-project
spec:
  databaseVersion: POSTGRES_15
  region: us-central1
  settings:
    tier: db-custom-2-7680
    diskType: SSD
    diskSize: 100
EOF

GitOps with Operators

# kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: production

resources:
  - bucket.yaml
  - database.yaml
  - deployment.yaml
  - service.yaml

# Apply everything together
kubectl apply -k ./production/

Common Errors

1. Operator version mismatch — ACK, ASO, and Config Connector release new CRD versions. An old controller cannot manage resources created with a newer CRD version.
2. IAM permissions too broad — The operator service account needs only the permissions for resources it manages. Follow least-privilege for the operator's IAM role.
3. Not setting resource deletion policy — Deleting a Kubernetes CR deletes the cloud resource by default. Set spec.resourcePolicy: keep to preserve the resource on CR deletion.
4. Operator resource conflicts with Terraform — Managing the same S3 bucket via both ACK and Terraform causes drift. Choose one tool per resource.
5. Namespace-scoped vs cluster-scoped confusion — Some CRDs are namespace-scoped, others cluster-scoped. Check the operator docs before applying.

Practice Questions

1. What is the difference between ACK and Crossplane? ACK is AWS-specific, managed by AWS. Crossplane is provider-agnostic and supports AWS, Azure, GCP, and others with a unified control plane.
2. How does the operator reconcile state? The operator controller watches CR changes, compares desired state with actual cloud resource state, and makes API calls to converge them.
3. What happens when you delete a CRD resource? By default, the operator deletes the cloud resource. Some operators support finalizers that preserve the resource for manual cleanup.
4. How do operators handle credentials? Operators use Kubernetes secrets or workload identity. ACK uses IAM roles for service accounts. ASO uses Azure managed identities.
5. Challenge: Design a GitOps workflow where creating a Kubernetes namespace automatically provisions an S3 bucket, IAM role, and VPC for that namespace. Use ACK controllers and ArgoCD.

Mini Project

Deploy a full-stack app using operators:

• Create an S3 bucket via ACK for static assets
• Create a DynamoDB table via ACK for sessions
• Create a Cloud SQL instance via Config Connector for relational data
• Deploy the app pods referencing these resources
• Verify everything reconciles after simulating a resource drift

FAQ

Built by the developers of Doda Browser, DodaZIP, and Durga Antivirus Pro.