Skip to main content

​
Deployment Options

The AI Gateway can be deployed in multiple ways to suit your infrastructure needs:

NPM/Bun

Quick local development

Docker

Containerized deployment

Node.js Server

Self-hosted production server

Cloudflare Workers

Edge deployment

Kubernetes

Scalable orchestration

Cloud Providers

AWS, Azure, GCP
For a fully managed solution without infrastructure concerns, consider Portkey’s hosted gateway which processes billions of tokens daily in production.

​
Quick Start (npx)

The fastest way to get started:
1

Run with npx

No installation required - just run:
npx @portkey-ai/gateway
Or with Bun:
bunx @portkey-ai/gateway
2

Verify it's running

The gateway starts on port 8787:
  • API: http://localhost:8787/v1
  • Console: http://localhost:8787/public/
Test it:
curl http://localhost:8787
You should see: AI Gateway says hey!
3

Make your first request

Now follow the quickstart guide to make your first API call.
This method is perfect for local development and testing. For production, use Docker or a Node.js server.

​
Docker

Deploy using Docker for containerized production environments.
1

Run with Docker

Pull and run the latest image from Docker Hub:
docker run --rm -p 8787:8787 portkeyai/gateway:latest
The gateway is now running at http://localhost:8787
2

Run with environment variables

Pass configuration via environment variables:
docker run --rm -p 8787:8787 \
  -e REDIS_CONNECTION_STRING="redis://localhost:6379" \
  portkeyai/gateway:latest
3

Build from source (optional)

Clone the repository and build your own image:
git clone https://github.com/portkey-ai/gateway
cd gateway
docker build -t portkey-gateway .
docker run --rm -p 8787:8787 portkey-gateway

​
Docker Compose

For multi-container setups with Redis caching:
1

Download docker-compose.yaml

wget "https://raw.githubusercontent.com/Portkey-AI/gateway/main/docker-compose.yaml"
2

Start the services

docker compose up -d
This starts:
  • AI Gateway on port 8787
  • Redis for caching (if configured)
3

View logs

docker compose logs -f gateway
4

Stop the services

docker compose down
The Docker image is built from the Dockerfile which uses a multi-stage build for optimal size.

​
Node.js Server

Run the gateway as a standalone Node.js application.
1

Clone the repository

git clone https://github.com/portkey-ai/gateway
cd gateway
2

Install dependencies

npm install
Or with Bun:
bun install
3

Build the project

npm run build
This compiles the TypeScript code and prepares the production bundle.
4

Start the server

node build/start-server.js
Or use npm script:
npm run start:node
5

Configure (optional)

Create a conf.json file to customize settings. See conf_sample.json for available options.You can also use environment variables:
export REDIS_CONNECTION_STRING="redis://localhost:6379"
node build/start-server.js
For development, use npm run dev:node which includes hot-reload.

​
Running as a Service

Create a systemd service file for production:
/etc/systemd/system/portkey-gateway.service
[Unit]
Description=Portkey AI Gateway
After=network.target

[Service]
Type=simple
User=portkey
WorkingDirectory=/opt/portkey-gateway
ExecStart=/usr/bin/node /opt/portkey-gateway/build/start-server.js
Restart=always
Environment=NODE_ENV=production

[Install]
WantedBy=multi-user.target
Enable and start: 
sudo systemctl enable portkey-gateway
sudo systemctl start portkey-gateway
sudo systemctl status portkey-gateway

​
Cloudflare Workers

Deploy to Cloudflare’s edge network for low-latency global distribution.
1

Clone and setup

git clone https://github.com/portkey-ai/gateway
cd gateway
npm install
2

Configure Wrangler

Make sure you have the Wrangler CLI installed and authenticated:
npm install -g wrangler
wrangler login
3

Deploy to Cloudflare

npm run deploy
This builds and deploys the gateway to Cloudflare Workers.
4

Get your worker URL

Wrangler will output your worker URL:
https://your-worker.your-subdomain.workers.dev
Cloudflare Workers have request limits on the free tier. Consider upgrading for production use.

​
Kubernetes

Deploy to Kubernetes for production-grade orchestration.
1

Create deployment manifest

Create deployment.yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
  name: portkey-gateway
spec:
  replicas: 3
  selector:
    matchLabels:
      app: portkey-gateway
  template:
    metadata:
      labels:
        app: portkey-gateway
    spec:
      containers:
      - name: gateway
        image: portkeyai/gateway:latest
        ports:
        - containerPort: 8787
        env:
        - name: NODE_ENV
          value: "production"
---
apiVersion: v1
kind: Service
metadata:
  name: portkey-gateway
spec:
  type: LoadBalancer
  ports:
  - port: 80
    targetPort: 8787
  selector:
    app: portkey-gateway
2

Apply to cluster

kubectl apply -f deployment.yaml
3

Verify deployment

kubectl get deployments
kubectl get services
kubectl get pods
4

Access the gateway

Get the external IP:
kubectl get service portkey-gateway
For production, consider adding:
  • Horizontal Pod Autoscaling (HPA)
  • Resource limits and requests
  • Ingress controller for HTTPS
  • ConfigMaps for configuration
  • Secrets for API keys

​
AWS EC2

Quick deployment to AWS EC2 using CloudFormation.
1

Use CloudFormation template

The repository includes a CloudFormation template for one-click deployment:Deploy to AWS EC2
2

Configure parameters

Set:
  • VPC ID
  • Subnet ID
  • Instance Type (t2.micro for testing, t3.small for production)
3

Launch stack

CloudFormation will:
  • Launch an EC2 instance
  • Install Docker
  • Run the gateway container
  • Configure security groups (port 8787)
4

Access your gateway

Get the public DNS from CloudFormation outputs:
http://<instance-public-dns>:8787

​
Other Platforms

​
Replit

Deploy with one click: Deploy on Replit

​
Zeabur

Use the template: Deploy on Zeabur

​
Azure, GCP, OpenShift

For enterprise deployments on:
  • Azure
  • Google Cloud Platform
  • Red Hat OpenShift
  • Other cloud providers
See the enterprise deployment guide or contact the team.

​
Configuration

​
Environment Variables

Common configuration options:
VariableDescriptionDefault
PORTServer port8787
REDIS_CONNECTION_STRINGRedis URL for caching-
NODE_ENVEnvironment modedevelopment

​
Configuration File

Create conf.json for advanced settings: 
{
  "port": 8787,
  "cache": {
    "type": "redis",
    "connection": "redis://localhost:6379"
  }
}
See conf_sample.json in the repository for all available options.

​
Verification

After installation, verify your deployment:
1

Health check

curl http://localhost:8787
Should return: AI Gateway says hey!
2

Test API endpoint

curl http://localhost:8787/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-provider: openai" \
  -H "Authorization: Bearer sk-***" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": "test"}]
  }'
3

Check the console

Open http://localhost:8787/public/ to view the web console.

​
Next Steps

Make Your First Request

Follow the quickstart to make your first API call

Configure Routing

Learn about routing, fallbacks, and load balancing

Add Guardrails

Protect your AI apps with input/output validation

Production Deployment

Best practices for production deployments
Need help? Join our Discord community or check the GitHub repository.

Build docs developers (and LLMs) love

Get started for freeTalk to us