What Terraform Manages
Our Terraform configuration manages the complete Penn Labs infrastructure stack:- EKS Cluster: Production Kubernetes cluster with node groups and autoscaling
- Networking: VPC, subnets, security groups, and load balancers
- Database: RDS PostgreSQL cluster with database roles and grants
- DNS: Route53 hosted zones for all product domains
- IAM: Roles and policies for services, GitHub Actions, and team members
- Vault: HashiCorp Vault for secrets management
- Monitoring: Grafana, Datadog, and observability infrastructure
- CI/CD: GitHub Actions secrets and AWS credentials
Main Terraform Files
Core Infrastructure
main.tf- Defines locals for products, SRE members, cluster configuration, and domainsprovider.tf- Configures providers (AWS, Kubernetes, Helm, PostgreSQL, Vault) and S3 backendvpc.tf- Creates VPC using the VPC Moduleeks.tf- Creates EKS cluster and installs aws-node-termination-handler
Database & Storage
rds.tf- Creates RDS PostgreSQL cluster with databases and roles for each productdb-backup.tf- Grants db-backup IAM role access to sql.pennlabs.org S3 bucket
Networking & DNS
route53.tf- Creates Route53 hosted zones using the Domain Module
IAM & Security
iam.tf- Creates IAM roles for products that can be assumed by Service Accountssre.tf- Creates IAM users for platform members with kubectl accessgh-actions.tf- Creates IAM user for GitHub Actions with kubectl role accessbastion.tf- Configures bastion host for team leads to exec into pods
Secrets & CI/CD
vault.tf- Provisions HashiCorp Vault with TLS, KMS, and EC2 instancegithub.tf- Creates organization-level GitHub Actions secrets
Cluster Configuration
production-cluster.tf- Uses Base Cluster Module and installs team-sync, Grafana, Bitwarden, Renovate, and db-backup
Environment Variables
The following environment variables must be exported before running Terraform:| Variable | Description |
|---|---|
GH_PERSONAL_TOKEN | GitHub Personal Access Token for Penn Labs Admin account |
GF_GH_CLIENT_ID | Client ID for Grafana Penn Labs OAuth2 application on GitHub |
GF_GH_CLIENT_SECRET | Client Secret for Grafana Penn Labs OAuth2 application on GitHub |
GF_SLACK_URL | Slack notification URL for Grafana alerts |
Key Configuration
Our Terraform setup defines several important configurations inmain.tf:1-64:
- Products: common-funding-application, hub-at-penn, ocwp, office-hours-queue, penn-clubs, penn-courses, platform, platform-dev, penn-mobile
- Cluster: Name
productionwith 5 nodes - Domains: ohq.io, pennbasics.com, penncfa.com, pennclubs.com, penncoursealert.com, penncourseplan.com, penncoursereview.com, penndegreeplan.com, penncourses.org, pennlabs.org, pennmobile.org
- VPC CIDR: 10.0.0.0/16
Prerequisites
Before working with Terraform:- Install Terraform - Download from terraform.io
- Install psql - Required for database operations during bootstrapping
- AWS Credentials - IAM user with AdministratorAccess policy
- Environment Variables - All variables listed above must be exported
- Access to Vault - Root token for secrets management
Remote State
Terraform state is stored in an S3 backend configured inprovider.tf. This ensures:
- State is shared across team members
- State is versioned and backed up
- State locking prevents concurrent modifications
- Sensitive values are encrypted at rest