State Module - Shipyard

Overview

The State module creates AWS resources for Terraform remote state storage with state locking. It provisions a versioned, encrypted S3 bucket and a DynamoDB table for state locking to prevent concurrent modifications.

Features

S3 Bucket

Versioned, encrypted bucket with public access blocked

State Locking

DynamoDB table for concurrent access prevention

Encryption

Server-side encryption with AES256

Versioning

Automatic versioning for state file history

Public Access Blocked

S3 public access block for security

Pay-Per-Request

DynamoDB on-demand billing for cost efficiency

Why Remote State?

Team Collaboration

Remote state allows multiple team members to work on the same infrastructure without conflicts. State locking prevents simultaneous modifications.

Security

Storing state locally exposes sensitive data (passwords, keys). Remote state in S3 provides encryption at rest and access control via IAM.

State History

S3 versioning maintains a history of state files, allowing rollback if needed.

Automation

CI/CD pipelines require shared state storage. Remote state enables automated Terraform runs in GitHub Actions, GitLab CI, etc.

Usage

Step 1: Create State Backend

Deploy the state module first (using local state):

# state-backend.tf

terraform {
  required_version = ">= 1.0"
  
  # Use local state initially
  # After creating the backend, migrate to remote state
}

provider "aws" {
  region = "us-east-1"
}

module "state" {
  source = "[email protected]:opsnorth/terraform-modules.git//state?ref=v1.0.0"

  environment = "production"

  tags = {
    Environment = "production"
    Purpose     = "terraform-state"
  }
}

output "s3_bucket_id" {
  value = module.state.s3_bucket_id
}

output "dynamodb_table_name" {
  value = module.state.dynamodb_table_name
}

output "backend_config" {
  value = <<-EOT
    terraform {
      backend "s3" {
        bucket         = "${module.state.s3_bucket_id}"
        key            = "terraform.tfstate"
        region         = "us-east-1"
        dynamodb_table = "${module.state.dynamodb_table_name}"
        encrypt        = true
      }
    }
  EOT
}

# Deploy the backend
terraform init
terraform apply

# Save the bucket and table names
terraform output -raw backend_config

Step 2: Configure Backend in Main Infrastructure

Update your main Terraform configuration to use remote state:

# main.tf

terraform {
  required_version = ">= 1.0"
  
  backend "s3" {
    bucket         = "terraform-state-production-123456789012"
    key            = "production/terraform.tfstate"
    region         = "us-east-1"
    dynamodb_table = "terraform-state-lock-production"
    encrypt        = true
  }
}

provider "aws" {
  region = "us-east-1"
}

# Your infrastructure modules
module "vpc" {
  source = "[email protected]:opsnorth/terraform-modules.git//vpc?ref=v1.0.0"
  # ...
}

module "eks" {
  source = "[email protected]:opsnorth/terraform-modules.git//eks?ref=v1.0.0"
  # ...
}

# Initialize with the new backend
terraform init

# Terraform will prompt to migrate existing state
# Type 'yes' to copy local state to S3

Step 3: Migrate State Backend to Remote

After creating the backend, you should migrate the state backend configuration itself to use remote state. This creates a circular dependency that needs careful handling.

# state-backend.tf (updated)

terraform {
  required_version = ">= 1.0"
  
  backend "s3" {
    bucket         = "terraform-state-production-123456789012"
    key            = "state-backend/terraform.tfstate"
    region         = "us-east-1"
    dynamodb_table = "terraform-state-lock-production"
    encrypt        = true
  }
}

# ... rest of configuration

# Re-initialize to migrate state backend's own state
terraform init -migrate-state

Environment-Specific Backends

Create separate state backends per environment:

# Development
module "state_dev" {
  source = "[email protected]:opsnorth/terraform-modules.git//state?ref=v1.0.0"
  
  environment = "dev"
}

# Staging
module "state_staging" {
  source = "[email protected]:opsnorth/terraform-modules.git//state?ref=v1.0.0"
  
  environment = "staging"
}

# Production
module "state_production" {
  source = "[email protected]:opsnorth/terraform-modules.git//state?ref=v1.0.0"
  
  environment = "production"
}

State File Organization

Organize state files using different keys:

terraform-state-bucket/
├── production/
│   ├── network/terraform.tfstate       # VPC
│   ├── compute/terraform.tfstate       # EKS
│   ├── data/terraform.tfstate          # RDS
│   └── secrets/terraform.tfstate       # Vault
├── staging/
│   └── terraform.tfstate
└── dev/
    └── terraform.tfstate

Backend configuration per layer:

# Network layer
terraform {
  backend "s3" {
    bucket = "terraform-state-production"
    key    = "production/network/terraform.tfstate"
    # ...
  }
}

# Compute layer
terraform {
  backend "s3" {
    bucket = "terraform-state-production"
    key    = "production/compute/terraform.tfstate"
    # ...
  }
}

Inputs

Name	Description	Type	Default	Required
`environment`	Environment name (e.g., dev, staging, prod)	`string`	n/a	yes
`force_destroy`	Allow S3 bucket destruction with objects	`bool`	`false`	no
`tags`	Tags to apply	`map(string)`	`{}`	no

Outputs

Name	Description
`s3_bucket_id`	S3 bucket name (use in backend config)
`s3_bucket_arn`	S3 bucket ARN
`dynamodb_table_name`	DynamoDB table name (use in backend config)
`dynamodb_table_arn`	DynamoDB table ARN

State Locking

DynamoDB provides automatic state locking:

# Terminal 1: Acquire lock
terraform apply
# Lock acquired: LockID=terraform-state-production/production/terraform.tfstate

# Terminal 2: Try to acquire lock (blocked)
terraform apply
# Error: Error acquiring the state lock
# Lock Info:
#   ID: abc123...
#   Path: terraform-state-production/production/terraform.tfstate
#   Operation: OperationTypeApply
#   Who: user@hostname
#   Created: 2024-01-15 10:30:00

Force Unlock (Use with Caution)

# Only if you're certain no one else is using the state
terraform force-unlock <LOCK_ID>

Force unlocking can cause state corruption if another process is actively using the state. Only use when certain the lock is stale.

State File Security

Sensitive Data in State

Terraform state files contain sensitive data:

Database passwords
API keys
Private keys
Resource IDs

Security measures:

Encryption at Rest

S3 server-side encryption with AES256 (enabled by default)

Encryption in Transit

HTTPS for all S3 API calls (enforced by AWS)

Access Control

Restrict S3 bucket access via IAM policies to authorized users only

Versioning

S3 versioning prevents accidental deletions and allows rollback

IAM Policy for State Access

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket",
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject"
      ],
      "Resource": [
        "arn:aws:s3:::terraform-state-production",
        "arn:aws:s3:::terraform-state-production/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:GetItem",
        "dynamodb:PutItem",
        "dynamodb:DeleteItem"
      ],
      "Resource": "arn:aws:dynamodb:us-east-1:123456789012:table/terraform-state-lock-production"
    }
  ]
}

Backup and Recovery

S3 Versioning

Versioning is enabled by default. To restore a previous version:

# List versions
aws s3api list-object-versions \
  --bucket terraform-state-production \
  --prefix production/terraform.tfstate

# Download specific version
aws s3api get-object \
  --bucket terraform-state-production \
  --key production/terraform.tfstate \
  --version-id <VERSION_ID> \
  terraform.tfstate.backup

# Restore by copying back
aws s3 cp terraform.tfstate.backup \
  s3://terraform-state-production/production/terraform.tfstate

Manual Backup

# Pull current state
terraform state pull > terraform.tfstate.backup

# Push backup (if needed)
terraform state push terraform.tfstate.backup

Regularly backup state files before major infrastructure changes.

Cost Considerations

S3 Costs

Storage: ~$0.023/GB/month (Standard)
State files are typically < 1 MB
Cost: < $0.01/month per state file

DynamoDB Costs

On-demand pricing: $1.25 per million write requests
Typical usage: ~100 lock operations/day
Cost: ~$0.01/month

Total Monthly Cost

~$0.10-0.50/month depending on number of state files and operations.

The state backend is extremely cost-effective compared to the value it provides for team collaboration and state safety.

Troubleshooting

State Lock Timeout

Error:

Error: Error acquiring the state lock

Solutions:

Wait for lock to release (someone else is running Terraform)

Check DynamoDB table:

aws dynamodb scan --table-name terraform-state-lock-production

Force unlock if stale:
```
terraform force-unlock <LOCK_ID>
```

Cannot Access State File

Error:

Error: Failed to get existing workspaces: AccessDenied

Check IAM permissions:

aws s3 ls s3://terraform-state-production/
aws dynamodb describe-table --table-name terraform-state-lock-production

State Drift

If state becomes out of sync with actual resources:

# Refresh state from AWS
terraform refresh

# Or import resources
terraform import <resource_type>.<name> <resource_id>

# Or recreate state
terraform state rm <resource>
terraform import <resource_type>.<name> <resource_id>

Best Practices

One Backend Per Environment

Create separate state backends for dev, staging, and production to prevent accidental changes.

Use State Locking

Always use DynamoDB for state locking to prevent concurrent modifications.

Enable Versioning

S3 versioning provides safety net for accidental state corruption.

Restrict Access

Use IAM policies to limit state file access to authorized users only.

Separate State Files

Split infrastructure into layers (network, compute, data) with separate state files.

Regular Backups

Backup state files before major infrastructure changes.

Usage Guide

State management patterns and best practices

Infrastructure Guide

Complete deployment workflow

VPC Module

Start deploying infrastructure

Terraform Docs

Official Terraform S3 backend documentation

Overview

Modules

​Overview

​Features

S3 Bucket

State Locking

Encryption

Versioning

Public Access Blocked

Pay-Per-Request

​Why Remote State?

​Usage

​Step 1: Create State Backend

​Step 2: Configure Backend in Main Infrastructure

​Step 3: Migrate State Backend to Remote

​Environment-Specific Backends

​State File Organization

​Inputs

​Outputs

​State Locking

​Force Unlock (Use with Caution)

​State File Security

​Sensitive Data in State

Encryption at Rest

Encryption in Transit

Access Control

Versioning

​IAM Policy for State Access

​Backup and Recovery

​S3 Versioning

​Manual Backup

​Cost Considerations

​S3 Costs

​DynamoDB Costs

​Total Monthly Cost

​Troubleshooting

​State Lock Timeout

​Cannot Access State File

​State Drift

​Best Practices

One Backend Per Environment

Use State Locking

Enable Versioning

Restrict Access

Separate State Files

Regular Backups

​Related Documentation

Usage Guide

Infrastructure Guide

VPC Module

Terraform Docs

Build docs developers (and LLMs) love

Overview

Features

Why Remote State?

Usage

Step 1: Create State Backend

Step 2: Configure Backend in Main Infrastructure

Step 3: Migrate State Backend to Remote

Environment-Specific Backends

State File Organization

Inputs

Outputs

State Locking

Force Unlock (Use with Caution)

State File Security

Sensitive Data in State

IAM Policy for State Access

Backup and Recovery

S3 Versioning

Manual Backup

Cost Considerations

S3 Costs

DynamoDB Costs

Total Monthly Cost

Troubleshooting

State Lock Timeout

Cannot Access State File

State Drift

Best Practices

Related Documentation