Deploy to Railway

Quick Deploy

Deploy LiteLLM to Railway with one click:

Click Deploy Button

Click the “Deploy on Railway” button above to open the template.

Configure Template

Railway will prompt you to set:

Project name
Environment variables (API keys)
Region preference

Deploy

Railway automatically:

Creates a new project
Provisions PostgreSQL database
Builds and deploys LiteLLM
Generates a public URL

Access Service

Once deployed, Railway provides:

Public URL: https://litellm-production.up.railway.app
Custom domain: Add your own domain in settings

Manual Railway Deployment

From GitHub Repository

Create New Project

Go to Railway Dashboard
Click New Project
Select Deploy from GitHub repo
Choose BerriAI/litellm or your fork

Configure Service

Railway auto-detects Docker configuration from the repository.Detected settings:

Runtime: Docker
Build: Uses /Dockerfile
Start command: Automatic from ENTRYPOINT

Add PostgreSQL

Click New Service in your project
Select Database → PostgreSQL
Railway provisions database and injects DATABASE_URL

Set Environment Variables

Add required variables (see configuration section)

Deploy

Railway automatically builds and deploys on push to main branch.

From Docker Image

Create New Project

Click New Project
Select Deploy a Docker Image

Specify Image

ghcr.io/berriai/litellm:main-stable

Configure Command

Add custom start command:

--port $PORT --config /app/config.yaml

Database Setup

Railway PostgreSQL

Railway automatically sets DATABASE_URL when you add PostgreSQL to your project.

Add Database

In your project, click New
Select Database → PostgreSQL
Railway creates database with:
- 1GB storage (Hobby plan)
- Automatic backups
- Private networking

Connection String

Railway automatically injects:

DATABASE_URL=postgresql://postgres:password@host:5432/railway

No manual configuration needed!

Enable Model Storage

Add to your service:

STORE_MODEL_IN_DB=True

Database Migrations

Migrations run automatically on container start using Prisma.

First deployment takes 15-20 seconds while Prisma runs migrations and generates the client.

Environment Configuration

Required Variables

In Railway service settings, add:

# Authentication
LITELLM_MASTER_KEY=sk-1234  # Generate a secure key

# Database (auto-injected if using Railway PostgreSQL)
# DATABASE_URL=postgresql://...  

# Enable UI model management
STORE_MODEL_IN_DB=True

Provider API Keys

OpenAI
Anthropic
Google Gemini
Azure OpenAI

OPENAI_API_KEY=sk-proj-...

ANTHROPIC_API_KEY=sk-ant-...

GEMINI_API_KEY=AIza...

AZURE_API_KEY=your-key
AZURE_API_BASE=https://your-resource.openai.azure.com
AZURE_API_VERSION=2024-02-15-preview

Optional Configuration

# Redis for caching
REDIS_HOST=redis.railway.internal
REDIS_PORT=6379
REDIS_PASSWORD=your-password

# Observability
LANGFUSE_PUBLIC_KEY=pk-...
LANGFUSE_SECRET_KEY=sk-...

# Debug mode
LITELLM_LOG=DEBUG

# Custom port (Railway sets this automatically)
# PORT=4000

Don’t set PORT manually - Railway injects this automatically and maps to your service.

Configuration with config.yaml

Mount Configuration File

Create Config in Repo

Add config.yaml to your repository:

config.yaml

model_list:
  - model_name: gpt-4o
    litellm_params:
      model: gpt-4o
      api_key: os.environ/OPENAI_API_KEY
  
  - model_name: claude-sonnet-4
    litellm_params:
      model: anthropic/claude-sonnet-4-20250514
      api_key: os.environ/ANTHROPIC_API_KEY

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
  database_url: os.environ/DATABASE_URL

Update Start Command

In Railway service settings:

litellm --config /app/config.yaml --port $PORT

Redeploy

Push to GitHub or trigger manual deploy in Railway dashboard.

Use Railway Variables

Create Variable Groups

Organize variables by environment:

production - Live environment
staging - Testing environment
development - Local development

Reference in Config

model_list:
  - model_name: gpt-4o
    litellm_params:
      model: gpt-4o
      api_key: os.environ/OPENAI_API_KEY
      api_base: os.environ/OPENAI_API_BASE  # Optional override

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
  database_url: os.environ/DATABASE_URL

Custom Domain

Generate Railway Domain

Railway automatically provides:

https://litellm-production-xxxx.up.railway.app

Add Custom Domain

In service settings, go to Settings → Networking
Click Add Custom Domain
Enter your domain: api.yourdomain.com

Configure DNS

Add a CNAME record:

Type:  CNAME
Name:  api
Value: litellm-production-xxxx.up.railway.app
TTL:   Auto

Wait for SSL

Railway automatically provisions SSL certificate (2-5 minutes).

Railway provides automatic SSL for all domains using Let’s Encrypt.

Redis for Caching

Add Redis Service

Add Redis to Project

Click New in your project
Select Database → Redis
Railway provisions Redis with:
- 100MB storage (Hobby plan)
- Private networking
- Auto-injected connection details

Configure LiteLLM

Railway auto-injects Redis variables:

REDIS_URL=redis://default:[email protected]:6379

Or use individual variables:

REDIS_HOST=redis.railway.internal
REDIS_PORT=6379
REDIS_PASSWORD=auto-generated

Enable Caching

Update config.yaml:

general_settings:
  cache: true
  redis_host: os.environ/REDIS_HOST
  redis_port: os.environ/REDIS_PORT
  redis_password: os.environ/REDIS_PASSWORD

Private Networking

Railway provides free private networking between services in the same project.

Services can communicate using:

# Private DNS (recommended)
postgresql://user:[email protected]:5432/db
redis://redis.railway.internal:6379

# Public URLs (avoid for internal communication)
https://service-name.up.railway.app

Deployment Strategies

Continuous Deployment

Railway automatically deploys on Git push:

git add .
git commit -m "Update LiteLLM configuration"
git push origin main

Railway will:

Detect the push via webhook
Build Docker image
Run health checks
Deploy with zero downtime

Manual Deployment

Go to service in Railway dashboard
Click Deploy → Redeploy
Or Trigger Deploy for specific commit

Rollback

View Deployments

Click Deployments tab in service dashboard

Select Previous Version

Click on successful deployment

Rollback

Click Redeploy this version

Monitoring and Observability

Railway Observability

Built-in monitoring includes:

Logs: Real-time log streaming
Metrics: CPU, Memory, Network, Disk
Deployments: Build and deploy history
Events: Service lifecycle events

View Logs

# In Railway dashboard
1. Click on your service
2. Go to "Logs" tab
3. Filter by:
   - Time range
   - Log level (ERROR, INFO, DEBUG)
   - Search text

Metrics Dashboard

Access in Metrics tab:

CPU usage: % utilization over time
Memory usage: MB used / available
Network: Inbound/outbound traffic
Request count: Requests per second

External Monitoring

Add monitoring services:

Langfuse
Datadog
Sentry

LANGFUSE_PUBLIC_KEY=pk-...
LANGFUSE_SECRET_KEY=sk-...
LANGFUSE_HOST=https://cloud.langfuse.com

Add to config.yaml:

litellm_settings:
  success_callback: ["langfuse"]

USE_DDTRACE=true
DD_API_KEY=your-key
DD_SITE=datadoghq.com
DD_SERVICE=litellm-proxy
DD_ENV=production

SENTRY_DSN=https://[email protected]/...
SENTRY_ENVIRONMENT=production

Scaling

Vertical Scaling

Upgrade service resources:

Go to Settings → Resources
Select plan:
- Hobby: 512MB RAM, 0.5 vCPU ($5/month)
- Pro: 8GB RAM, 8 vCPU ($20/month)

Horizontal Scaling

Railway Hobby plan supports 1 instance only. Upgrade to Pro for multiple instances.

Pro plan features:

Multiple service instances
Load balancing
Zero-downtime deployments
Vertical and horizontal scaling

Cost Management

Pricing Overview

Hobby Plan ($5/month):
  - 512MB RAM, 0.5 vCPU per service
  - $5 free credit (covers small deployments)
  - 100GB bandwidth
  - Unlimited projects

Pro Plan (Usage-based):
  - $20/month base
  - Pay for resources used
  - 8GB RAM, 8 vCPU available
  - Priority support

Cost Optimization Tips

Use Shared Database

Share PostgreSQL between staging/development environments.

Remove Unused Services

Delete stopped or unnecessary services to avoid charges.

Set Resource Limits

Configure memory and CPU limits to control costs:

# In Railway service settings
Memory Limit: 1GB
CPU Limit: 1 vCPU

Use Sleep Mode

Enable auto-sleep for development environments (Hobby plan only).

Troubleshooting

Build Failures

# Check build logs in Deployments tab

# Common issues:

# 1. Out of memory during build
Error: Docker build failed - OOM
Solution: Upgrade to Pro plan or optimize Dockerfile

# 2. npm/pip installation timeout
Error: Command timed out after 15 minutes
Solution: Use Docker layer caching, pre-build dependencies

# 3. File not found
Error: COPY failed: file not found: config.yaml
Solution: Ensure file is committed to Git

Service Won’t Start

# Check logs for startup errors

# Missing DATABASE_URL
Error: DATABASE_URL not set
Solution: Add PostgreSQL service to project

# Port binding error
Error: Port 4000 already in use
Solution: Use $PORT variable, don't hardcode port

# Config file error
Error: Failed to load config.yaml
Solution: Validate YAML syntax, check file path

Database Connection Issues

# Use private networking
# ✅ Correct:
DATABASE_URL=postgresql://user:[email protected]:5432/db

# ❌ Wrong (public URL, slower and less secure):
DATABASE_URL=postgresql://user:[email protected]:5432/db

# Test connection
railway run psql $DATABASE_URL

High Memory Usage

# Check metrics in Railway dashboard
# If memory usage > 80%:

# 1. Upgrade to larger instance
# 2. Enable Redis caching to reduce memory load
# 3. Optimize model configurations
# 4. Reduce concurrent requests

Railway CLI

Installation

# Install Railway CLI
npm install -g @railway/cli

# Login
railway login

# Link to project
railway link

Common Commands

# View logs
railway logs

# Open service in browser
railway open

# Run command with Railway environment
railway run litellm --config config.yaml

# Deploy manually
railway up

# View environment variables
railway variables

# Set environment variable
railway variables set OPENAI_API_KEY=sk-...

# SSH into service
railway shell

Comparison: Railway vs Render

Feature	Railway	Render
Free tier	$5 credit	90-day PostgreSQL trial
Pricing	Usage-based	Fixed tiers
Auto-scaling	Pro plan	Yes
Private networking	✅ Free	❌
CLI	✅	❌
Build speed	Fast (layer caching)	Medium
Zero-downtime deploys	✅	✅

Next Steps

Fly.io Deployment

Deploy closer to users with edge compute

Monitoring

Set up observability and alerts

Security

Secure your deployment

Performance

Optimize for production workloads

Deploy

Production

​Quick Deploy

​Manual Railway Deployment

​From GitHub Repository

​From Docker Image

​Database Setup

​Railway PostgreSQL

​Database Migrations

​Environment Configuration

​Required Variables

​Provider API Keys

​Optional Configuration

​Configuration with config.yaml

​Mount Configuration File

​Use Railway Variables

​Custom Domain

​Redis for Caching

​Add Redis Service

​Private Networking

​Deployment Strategies

​Continuous Deployment

​Manual Deployment

​Rollback

​Monitoring and Observability

​Railway Observability

​View Logs

​Metrics Dashboard

​External Monitoring

​Scaling

​Vertical Scaling

​Horizontal Scaling

​Cost Management

​Pricing Overview

​Cost Optimization Tips

​Troubleshooting

​Build Failures

​Service Won’t Start

​Database Connection Issues

​High Memory Usage

​Railway CLI

​Installation

​Common Commands

​Comparison: Railway vs Render

​Next Steps

Fly.io Deployment

Monitoring

Security

Performance

Build docs developers (and LLMs) love

Quick Deploy

Manual Railway Deployment

From GitHub Repository

From Docker Image

Database Setup

Railway PostgreSQL

Database Migrations

Environment Configuration

Required Variables

Provider API Keys

Optional Configuration

Configuration with config.yaml

Mount Configuration File

Use Railway Variables

Custom Domain

Redis for Caching

Add Redis Service

Private Networking

Deployment Strategies

Continuous Deployment

Manual Deployment

Rollback

Monitoring and Observability

Railway Observability

View Logs

Metrics Dashboard

External Monitoring

Scaling

Vertical Scaling

Horizontal Scaling

Cost Management

Pricing Overview

Cost Optimization Tips

Troubleshooting

Build Failures

Service Won’t Start

Database Connection Issues

High Memory Usage

Railway CLI

Installation

Common Commands

Comparison: Railway vs Render

Next Steps