Skip to main content

​
Quick Start

The fastest way to deploy LiteLLM is using Docker Compose with the official image.
1

Pull the Docker Image

docker pull ghcr.io/berriai/litellm:main-latest
Stable releases use the -stable tag (undergoes 12-hour load tests):
docker pull ghcr.io/berriai/litellm:main-stable
2

Create Configuration File

Create a config.yaml file with your model configuration:
config.yaml
model_list:
  - model_name: gpt-4o
    litellm_params:
      model: gpt-4o
      api_key: os.environ/OPENAI_API_KEY
  
  - model_name: claude-sonnet-4
    litellm_params:
      model: anthropic/claude-sonnet-4-20250514
      api_key: os.environ/ANTHROPIC_API_KEY

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
  database_url: os.environ/DATABASE_URL
3

Run with Docker Compose

Use the provided docker-compose.yml for a complete setup with PostgreSQL and Prometheus:
docker-compose.yml
services:
  litellm:
    image: ghcr.io/berriai/litellm:main-stable
    ports:
      - "4000:4000"
    environment:
      DATABASE_URL: "postgresql://llmproxy:dbpassword9090@db:5432/litellm"
      STORE_MODEL_IN_DB: "True" # Allows adding models via UI
    volumes:
      - ./config.yaml:/app/config.yaml
    command:
      - "--config=/app/config.yaml"
    depends_on:
      - db
    healthcheck:
      test:
        - CMD-SHELL
        - python3 -c "import urllib.request; urllib.request.urlopen('http://localhost:4000/health/liveliness')"
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s

  db:
    image: postgres:16
    restart: always
    environment:
      POSTGRES_DB: litellm
      POSTGRES_USER: llmproxy
      POSTGRES_PASSWORD: dbpassword9090
    ports:
      - "5432:5432"
    volumes:
      - postgres_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -d litellm -U llmproxy"]
      interval: 1s
      timeout: 5s
      retries: 10

  prometheus:
    image: prom/prometheus
    volumes:
      - prometheus_data:/prometheus
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
    ports:
      - "9090:9090"
    command:
      - "--config.file=/etc/prometheus/prometheus.yml"
      - "--storage.tsdb.path=/prometheus"
      - "--storage.tsdb.retention.time=15d"
    restart: always

volumes:
  prometheus_data:
    driver: local
  postgres_data:
    name: litellm_postgres_data
Start all services:
docker-compose up -d
4

Verify Deployment

Check health status:
curl http://localhost:4000/health/liveliness
Access the admin UI at http://localhost:4000/ui

​
Docker Image Details

​
Multi-Stage Build

The Dockerfile uses a multi-stage build for security and size optimization: 
# Base images from Chainguard (minimal, security-focused)
ARG LITELLM_BUILD_IMAGE=cgr.dev/chainguard/wolfi-base
ARG LITELLM_RUNTIME_IMAGE=cgr.dev/chainguard/wolfi-base

# Builder stage
FROM $LITELLM_BUILD_IMAGE AS builder
WORKDIR /app

# Install build dependencies
RUN apk add --no-cache bash gcc py3-pip python3 python3-dev openssl openssl-dev
RUN python -m pip install build

# Copy source and build
COPY . .
RUN ./docker/build_admin_ui.sh
RUN rm -rf dist/* && python -m build
RUN pip install dist/*.whl
RUN pip wheel --no-cache-dir --wheel-dir=/wheels/ -r requirements.txt

# Runtime stage
FROM $LITELLM_RUNTIME_IMAGE AS runtime
WORKDIR /app

# Install runtime dependencies (includes Node.js for Prisma)
RUN apk add --no-cache bash openssl tzdata nodejs npm python3 py3-pip libsndfile

# Copy built artifacts
COPY --from=builder /app/dist/*.whl .
COPY --from=builder /wheels/ /wheels/
RUN pip install *.whl /wheels/* --no-index --find-links=/wheels/

# Generate Prisma client
RUN prisma generate --schema=./litellm/proxy/schema.prisma

EXPOSE 4000/tcp
ENTRYPOINT ["docker/prod_entrypoint.sh"]
CMD ["--port", "4000"]

​
Environment Variables

Security Best Practice: Use environment variables for sensitive data instead of hardcoding in config files.
VariableDescriptionRequired
DATABASE_URLPostgreSQL connection stringYes (for production)
LITELLM_MASTER_KEYMaster key for proxy authenticationRecommended
STORE_MODEL_IN_DBEnable model management via UINo
OPENAI_API_KEYOpenAI API keyIf using OpenAI
ANTHROPIC_API_KEYAnthropic API keyIf using Anthropic
USE_DDTRACEEnable Datadog tracingNo
SEPARATE_HEALTH_APPRun health checks on separate portNo

​
Standalone Docker Run

For simple deployments without Docker Compose: 
# Without database (in-memory)
docker run -d \
  --name litellm \
  -p 4000:4000 \
  -e LITELLM_MASTER_KEY=sk-1234 \
  -e OPENAI_API_KEY=your-key \
  ghcr.io/berriai/litellm:main-stable \
  --model gpt-4o

# With external database
docker run -d \
  --name litellm \
  -p 4000:4000 \
  -e DATABASE_URL=postgresql://user:pass@host:5432/litellm \
  -e LITELLM_MASTER_KEY=sk-1234 \
  -v $(pwd)/config.yaml:/app/config.yaml \
  ghcr.io/berriai/litellm:main-stable \
  --config /app/config.yaml

​
Database Setup

LiteLLM uses Prisma with PostgreSQL for persistence.

​
Database Schema

The schema includes tables for:
  • API Keys (LiteLLM_VerificationToken) - Virtual keys with budgets and rate limits
  • Teams (LiteLLM_TeamTable) - Team management and spend tracking
  • Users (LiteLLM_UserTable) - User authentication and authorization
  • Models (LiteLLM_ProxyModelTable) - Model configurations
  • Spend Logs (LiteLLM_SpendLogs) - Request logs and cost tracking
  • Organizations (LiteLLM_OrganizationTable) - Multi-tenant support

​
Migrations

Migrations run automatically on container startup. The proxy takes 15-20 seconds to fully initialize.
The entrypoint script (docker/prod_entrypoint.sh) handles migrations: 
#!/bin/sh
if [ "$SEPARATE_HEALTH_APP" = "1" ]; then
    export LITELLM_ARGS="$@"
    exec supervisord -c /etc/supervisord.conf
fi

if [ "$USE_DDTRACE" = "true" ]; then
    export DD_TRACE_OPENAI_ENABLED="False"
    exec ddtrace-run litellm "$@"
else
    exec litellm "$@"
fi
Prisma generates the client and applies migrations during the build process.

​
Prometheus Configuration

Create prometheus.yml for metrics collection:
prometheus.yml
global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'litellm'
    static_configs:
      - targets: ['litellm:4000']
LiteLLM exposes metrics at /metrics endpoint.

​
Volume Persistence

Always use named volumes for production databases to prevent data loss.
volumes:
  postgres_data:
    name: litellm_postgres_data
    driver: local
  prometheus_data:
    driver: local

​
Health Checks

LiteLLM provides multiple health check endpoints:
  • /health/liveliness - Basic service health
  • /health/readiness - Ready to accept traffic
  • /health - Combined health status
Health check configuration: 
healthcheck:
  test: ["CMD-SHELL", "python3 -c \"import urllib.request; urllib.request.urlopen('http://localhost:4000/health/liveliness')\""]
  interval: 30s
  timeout: 10s
  retries: 3
  start_period: 40s  # Allow time for migrations

​
Upgrading

# Pull latest stable image
docker-compose pull

# Restart services (migrations run automatically)
docker-compose up -d

# Check logs
docker-compose logs -f litellm

​
Troubleshooting

​
Container won’t start

# Check logs
docker logs litellm

# Common issues:
# 1. Database not ready - increase start_period in healthcheck
# 2. Invalid config.yaml - validate YAML syntax
# 3. Missing environment variables - check .env file

​
Database connection errors

# Test database connectivity
docker exec -it litellm_db psql -U llmproxy -d litellm

# Check DATABASE_URL format:
# postgresql://username:password@hostname:5432/database

​
Performance issues

# Check resource usage
docker stats litellm

# Increase container resources
docker run --memory=2g --cpus=2 ...

​
Next Steps

Kubernetes

Scale with Kubernetes and Helm charts

Monitoring

Set up observability and alerting

Security

Secure your deployment

High Availability

Deploy for production at scale

Build docs developers (and LLMs) love

Get started for freeTalk to us