Overview
State locking is a critical safety mechanism that prevents multiple Terraform processes from modifying state simultaneously. Concurrent state modifications can corrupt state and lead to infrastructure inconsistencies.Why State Locking Matters
The Concurrent Modification Problem
Without locking, two operators running Terraform simultaneously could:- Both read the same initial state
- Both plan changes based on that state
- Both write modified state back
- Second write overwrites first, losing changes
- State becomes inconsistent with reality
Protected Operations
State locking protects these operations:terraform applyterraform destroyterraform importterraform statecommands (mv, rm, etc.)terraform plan(when using-outflag)
State Locking Architecture
Lock Information
Locks contain metadata about who holds the lock:internal/states/statemgr/locker.go
Locker Interface
State managers that support locking implement theLocker interface:
internal/backend/backend.go:86-94
Command-Level Locking
Terraform commands use theclistate.Locker wrapper for user-friendly locking:
internal/command/clistate/state.go:56-68
Creating a Locker
internal/command/state_mv.go:90
Acquiring a Lock
internal/command/state_mv.go:91-99
Lock Timeout
Locks are acquired with a timeout to prevent indefinite waiting:-lock-timeout flag.
Reference: internal/command/clistate/state.go:117-127
Lock Error Messages
Lock Acquisition Error
internal/command/clistate/state.go:24-29
Unlock Error
internal/command/clistate/state.go:31-44
Lock Timeout Threshold
User notification appears after a threshold:internal/command/clistate/state.go:23
Disabling Locks
The -lock Flag
Most state-modifying commands support-lock=false:
Lock-Disabled Wrapper
For testing or special cases, locks can be disabled programmatically:internal/states/statemgr/lock.go:16-48
Backend Locking Support
Backends with Locking
These backends support native locking:- S3: Using DynamoDB table for locks
- Azure: Using blob leases
- GCS: Using object versioning
- Terraform Cloud: Built-in locking
- Consul: Using key locks
- etcd: Using distributed locks
- Kubernetes: Using lease objects
Backends without Locking
These backends do NOT support locking:- Local: Basic filesystem backend
- HTTP: Plain HTTP endpoints
- Artifactory: JFrog Artifactory backend
- PostgreSQL: Direct database backend (deprecated)
For production use, always choose backends that support locking.
S3 Backend Locking Example
Configuration
DynamoDB Table Setup
Required IAM Permissions
Force Unlock
When to Force Unlock
Force unlock should only be used when:- A Terraform process crashed and didn’t release its lock
- A CI/CD job was killed and left a stale lock
- You are absolutely certain no other process is using the state
Force Unlock Command
Confirming Force Unlock
Terraform requires confirmation:Lock Monitoring
Viewing Lock Information
Different backends expose lock information differently:DynamoDB (S3 Backend)
Consul Backend
Lock Expiration
Most backends do not have automatic lock expiration:- Locks persist until explicitly released
- Crashed processes leave stale locks
- Monitor for stale locks in production
Consider implementing monitoring to detect locks held longer than expected operation times.
Locking Best Practices
Always Enable Locking
Always Enable Locking
Use backends with locking support for all shared environments. The local backend without locking is only suitable for individual development.
Set Appropriate Timeouts
Set Appropriate Timeouts
Configure lock timeouts based on expected operation duration:
Handle Lock Failures Gracefully
Handle Lock Failures Gracefully
In CI/CD, implement retry logic with exponential backoff for transient lock failures.
Monitor Lock Duration
Monitor Lock Duration
Track how long operations hold locks to identify potential issues:
- Long-running plans
- Stuck operations
- Resource contention
Document Lock Recovery
Document Lock Recovery
Maintain runbooks for force-unlock scenarios:
- How to verify process is truly dead
- Steps to safely force unlock
- Post-unlock verification
Use Descriptive Lock Operations
Use Descriptive Lock Operations
Lock operation names help identify what’s running:
- “apply”
- “destroy”
- “state-mv”
- “import”
Troubleshooting Lock Issues
Lock Already Held
- Wait for the other operation to complete
- Contact the user holding the lock
- Verify the process is still running
- Force unlock if process is confirmed dead
Lock Timeout
- Increase timeout:
terraform apply -lock-timeout=20m - Check if another operation is stuck
- Verify backend connectivity
Permission Denied on Lock
Stale Lock After Crash
If Terraform crashes, the lock may not be released:Lock-Free Operations
These operations do not require locks:terraform initterraform validateterraform plan(without-out)terraform showterraform outputterraform state list(read-only)terraform state show(read-only)
Related Resources
- State Overview - Core state concepts
- State Backends - Backend configuration and locking support
- State Manipulation - State commands and locking behavior
Source Code References
- CLI Locking:
internal/command/clistate/state.go - State Manager Locking:
internal/states/statemgr/locker.go - Lock Disabled:
internal/states/statemgr/lock.go - Lock with Context:
internal/states/statemgr/lock.go - State Move Locking:
internal/command/state_mv.go:89-99 - State Remove Locking:
internal/command/state_rm.go:50-61