Skip to main content
This guide shows two common ways to run KoreShield on AWS: a simple EC2 Docker host or a managed ECS service.

​
Use Cases

  • Centralized LLM security proxy for multiple apps and teams
  • Compliance workloads that require audit trails and private networking
  • High-throughput production traffic with autoscaling and managed ingress

​
Architecture Choices

  • EC2 + Docker for low-cost, low-ops environments
  • ECS Fargate for managed compute and simpler scaling
  • Optional: ALB in front of the service for TLS termination and health checks

​
Option A: EC2 + Docker

1

Provision an EC2 Instance

  • Choose Amazon Linux 2023 or Ubuntu 22.04
  • Open inbound ports for your app and KoreShield (default 8000)
  • Prefer placing the instance in a private subnet with an ALB in front
2

Install Docker

# Amazon Linux 2023
sudo dnf update -y
sudo dnf install -y docker
sudo systemctl enable --now docker
sudo usermod -aG docker ec2-user
3

Configure KoreShield

# On the EC2 host
git clone https://github.com/koreshield/koreshield.git
cd koreshield/koreshield
cp config/config.example.yaml config/config.yaml
Set provider API keys:
export OPENAI_API_KEY=your-api-key
# or other provider keys
4

Run with Docker

docker build -t koreshield .

docker run -d \
  -p 8000:8000 \
  -e OPENAI_API_KEY=$OPENAI_API_KEY \
  -v $(pwd)/config/config.yaml:/app/config/config.yaml \
  koreshield
5

Verify Health

curl http://localhost:8000/health

​
Option B: ECS (Fargate)

1

Build and Push Image to ECR

aws ecr create-repository --repository-name koreshield

# Authenticate and push
aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin <account-id>.dkr.ecr.us-east-1.amazonaws.com

docker build -t koreshield .
docker tag koreshield:latest <account-id>.dkr.ecr.us-east-1.amazonaws.com/koreshield:latest

docker push <account-id>.dkr.ecr.us-east-1.amazonaws.com/koreshield:latest
2

Create an ECS Task Definition

  • Container port: 8000
  • Environment variables: provider keys
  • Mount your config or bake it into the image
  • Attach an IAM role for Secrets Manager or SSM access
3

Create a Service

  • Use an ALB or NLB
  • Configure health checks at /health
  • Enable autoscaling on CPU or request count

​
Secrets and Config

Store provider keys in AWS Secrets Manager or SSM Parameter Store for enhanced security.
  • Store provider keys in AWS Secrets Manager or SSM Parameter Store
  • Pass secrets to the task as environment variables
  • Use CONFIG_FILE if you mount a custom config path

​
Networking and TLS

  • Terminate TLS at the ALB
  • Restrict inbound access to known CIDR ranges or VPC endpoints
  • Use a WAF if the endpoint is internet-facing

​
Observability

  • Enable json_logs: true and ship logs to CloudWatch
  • Scrape /metrics with Prometheus or use a sidecar exporter
  • Create CloudWatch alarms for high error rates and latency

​
Security Notes

Always use secure secret management for API keys and sensitive configuration.
  • Store API keys in AWS Secrets Manager or SSM Parameter Store
  • Use json_logs: true in production
  • Restrict inbound access to the proxy endpoint

​
Troubleshooting

502/504 from ALB: confirm target health checks hit `/health`

​
Next Steps

Build docs developers (and LLMs) love

Get started for freeTalk to us