Terraform Remote Backends: S3, Terraform Cloud & More

Terraform remote backends store state files in a shared, durable, and locked location that enables team collaboration and prevents state corruption from concurrent operations.

What You'll Learn

In this tutorial, you will learn how to configure Terraform remote backends including S3 with DynamoDB locking, Terraform Cloud, Azure Storage, and Google Cloud Storage, plus migration between backends.

Why It Matters

Local state files are a single point of failure. They cannot be shared, get lost easily, and have no locking. Remote backends provide durability, team access, state versioning, and encryption for production infrastructure.

Real-World Use

DodaTech uses S3 remote backends with DynamoDB locking for all Terraform configurations. The platform team at Durga Antivirus Pro manages over fifty distinct state files across AWS, Azure, and GCP using a standardized backend configuration template.

S3 Backend

The most popular remote backend uses AWS S3 for storage and DynamoDB for state locking:

terraform {
  backend "s3" {
    bucket         = "dodatech-terraform-state"
    key            = "prod/infrastructure/terraform.tfstate"
    region         = "us-east-1"
    encrypt        = true
    dynamodb_table = "terraform-state-lock"
  }
}

Expected output: Running <a href="/devops/terraform/">terraform</a> init configures the S3 backend. Terraform fetches the existing state from s3://dodatech-<a href="/devops/terraform/">terraform</a>-state/prod/infrastructure/<a href="/devops/terraform/">terraform</a>.tfstate and acquires a DynamoDB lock before each operation.

Creating the S3 Backend

aws s3api create-bucket --bucket dodatech-terraform-state --region us-east-1
aws s3api put-bucket-versioning --bucket dodatech-terraform-state --versioning-configuration Status=Enabled
aws s3api put-bucket-encryption --bucket dodatech-terraform-state --server-side-encryption-configuration '{"Rules":[{"ApplyServerSideEncryptionByDefault":{"SSEAlgorithm":"AES256"}}]}'
aws dynamodb create-table --table-name terraform-state-lock --attribute-definitions AttributeName=LockID,AttributeType=S --key-schema AttributeName=LockID,KeyType=HASH --billing-mode PAY_PER_REQUEST

Expected output: An S3 bucket with versioning and encryption enabled, plus a DynamoDB table for locks. Bucket names must be globally unique.

Terraform Cloud Backend

Terraform Cloud provides managed state storage with built-in locking, version history, and collaboration features:

terraform {
  cloud {
    organization = "dodatech"
    workspaces {
      name = "production-infrastructure"
    }
  }
}

Expected output: <a href="/devops/terraform/">terraform</a> init authenticates to Terraform Cloud. State is stored on Terraform Cloud servers with automatic encryption, locking, and version history accessible through the web UI.

Terraform Cloud with Tags

terraform {
  cloud {
    organization = "dodatech"
    workspaces {
      tags = ["production", "aws"]
    }
  }
}

Expected output: Terraform creates workspaces dynamically based on tag matching. Multiple configurations can share the same workspace set.

Azure Storage Backend

terraform {
  backend "azurerm" {
    resource_group_name  = "terraform-state-rg"
    storage_account_name = "dodatechterraformstate"
    container_name       = "tfstate"
    key                  = "prod.infrastructure.tfstate"
  }
}

Expected output: State stored in Azure Blob Storage with encryption at rest. Authentication uses Azure CLI credentials or managed identity.

GCS Backend

terraform {
  backend "gcs" {
    bucket = "dodatech-terraform-state"
    prefix = "prod/infrastructure"
  }
}

Expected output: State stored in Google Cloud Storage with bucket-level encryption. Object versioning preserves state history.

Migration Between Backends

Local to Remote

terraform init -migrate-state

Expected output: Terraform detects the backend change and prompts to copy the local state file to the remote backend. Confirm to migrate.

Remote to Remote

terraform init -reconfigure

Expected output: Reconfigures the backend without copying state. Use <a href="/devops/terraform/">terraform</a> state pull and <a href="/devops/terraform/">terraform</a> state push for manual migration between backends.

Backend Configuration Patterns

Partial Configuration

Separate static and dynamic backend settings:

# backend.hcl
bucket         = "dodatech-terraform-state"
region         = "us-east-1"
dynamodb_table = "terraform-state-lock"
encrypt        = true

terraform init -backend-config=backend.hcl -backend-config="key=prod/network/terraform.tfstate"

Expected output: The key is provided at init time while static settings come from the config file. This pattern supports multiple environments from one configuration.

Common Mistakes

1. Hardcoding Backend Configuration

Backend configuration containing bucket names and keys makes the configuration non-portable. Use partial configuration with -backend-config.

2. Skipping State Versioning

An S3 bucket without versioning cannot recover from accidental state deletion or corruption. Always enable versioning.

3. Not Encrypting State at Rest

State files may contain sensitive data like resource IDs and plaintext secrets. Always enable encryption on the backend.

4. Sharing Backends Across Unrelated Configurations

Multiple unrelated configurations writing to the same state file causes corruption. Each configuration needs its own state file (different key).

5. Migrating Without Backing Up

Always back up the local state file before migrating to a remote backend. If migration fails, restore from backup.

Practice Questions

1. What are the two components of an S3 backend setup? An S3 bucket for state file storage and a DynamoDB table for state locking.

2. Why should you enable versioning on the S3 state bucket? Versioning enables recovery from accidental state deletion, corruption, or unwanted changes by restoring previous versions.

3. How does Terraform Cloud handle state locking? Terraform Cloud manages locking automatically. Operations are serialized per workspace, preventing concurrent modifications.

4. What is the difference between -migrate-state and -reconfigure? -migrate-state copies existing state to the new backend. -reconfigure reinitializes without copying state.

5. Challenge: Create an S3 bucket with versioning and encryption, a DynamoDB lock table, configure the backend, migrate local state, and verify locking by running concurrent applies.

Mini Project: Remote Backend Setup

Provision an S3 bucket with versioning and encryption, create a DynamoDB lock table, configure the Terraform backend, migrate state from local to remote, and verify locking and state history through the AWS console.

Related Concepts

What's Next

Configure Terraform remote backends for team collaboration, then explore Dynamic Blocks to write flexible, DRY configurations.

Built by the developers of Doda Browser, DodaZIP, and Durga Antivirus Pro.