Skip to main content
Portkey AI Gateway Demo

​
What is Portkey AI Gateway?

The AI Gateway is an open-source, lightweight solution designed for fast, reliable, and secure routing to 1600+ language, vision, audio, and image models. Built with Hono framework for TypeScript/JavaScript, it processes over 10 billion tokens daily in production environments.

Blazing Fast

Sub-1ms latency with a tiny 122kb footprint

Battle Tested

Processing 10B+ tokens daily in production

Enterprise Ready

Enhanced security, scale, and custom deployments

​
Why Choose AI Gateway?

​
Universal LLM Integration

Integrate with any LLM in under 2 minutes. The gateway provides a unified OpenAI-compatible API for 250+ models across 45+ providers including:
  • OpenAI, Azure OpenAI, Anthropic Claude
  • Google Gemini, AWS Bedrock, Cohere
  • Together AI, Groq, Perplexity, Mistral
  • Ollama, Hugging Face, and many more

​
Production-Grade Reliability

Automatic Retries

Retry failed requests up to 5 times with exponential backoff

Fallbacks

Automatically switch to backup providers on failures

Load Balancing

Distribute requests across multiple API keys or providers

Request Timeouts

Set granular timeouts to manage latencies

​
Advanced Features

  • Guardrails: Verify LLM inputs and outputs with 40+ pre-built guardrails
  • Multi-modal Support: Text, vision, audio, image generation, and real-time APIs
  • Smart Caching: Reduce costs and improve latency with response caching
  • Conditional Routing: Route requests based on custom logic and conditions
  • MCP Gateway: Centralized control plane for Model Context Protocol servers
Get started with these essential resources:

Quickstart Guide

Make your first API call in under 5 minutes

Installation Guide

Deploy the gateway with npx, Docker, or Node.js

Core Concepts

Learn about routing, configs, and providers

API Reference

Explore the complete API documentation

​
Key Capabilities

​
Routing & Load Balancing

Distribute requests intelligently across multiple providers and API keys: 
config = {
  "strategy": {
    "mode": "loadbalance",
  },
  "targets": [
    {"provider": "openai", "api_key": "sk-***", "weight": 0.7},
    {"provider": "anthropic", "api_key": "sk-ant-***", "weight": 0.3}
  ]
}

client = client.with_options(config=config)

​
Guardrails

Protect your AI deployments with input/output validation: 
config = {
  "retry": {"attempts": 5},
  "output_guardrails": [{
    "default.contains": {"operator": "none", "words": ["Apple"]},
    "deny": True
  }]
}

client = client.with_options(config=config)
The retry config ensures requests are retried up to 5 times, while the guardrail denies any response containing “Apple”.

​
Multi-Provider Fallbacks

Automatically failover to backup providers: 
config = {
  "strategy": {"mode": "fallback"},
  "targets": [
    {"provider": "openai", "api_key": "sk-***"},
    {"provider": "anthropic", "api_key": "sk-ant-***"}
  ]
}

​
Architecture

The AI Gateway is built with:
  • Hono Framework: Fast, lightweight web framework supporting multiple runtimes
  • Provider System: Modular provider implementations with standardized interfaces
  • Middleware Pipeline: Request validation, caching, logging, and routing
  • Plugin System: Extensible guardrails for content filtering and validation
The gateway runs on multiple environments: Node.js, Cloudflare Workers, Docker, Kubernetes, and more.

​
Community & Support

GitHub

Star the repository and contribute

Discord

Join our community for help

Twitter

Follow for updates and tips

​
Next Steps

1

Install the Gateway

Follow the quickstart guide to get the gateway running locally in 2 minutes.
2

Make Your First Request

Send your first API call to any LLM provider through the gateway.
3

Explore Features

Learn about routing, guardrails, and caching.
4

Deploy to Production

Choose your deployment method for production use.
Need Enterprise Features? Check out our enterprise version with advanced security, governance, and compliance features.

Build docs developers (and LLMs) love

Get started for freeTalk to us