Supported Providers

Overview

Portkey AI Gateway supports 250+ LLMs from 78+ providers, giving you access to virtually every major AI model through a single, unified API.

All Supported Providers

Here’s the complete list of providers integrated with Portkey:

Major LLM Providers

Provider	Models	Features	Documentation
OpenAI	GPT-4, GPT-3.5, o1, o3, DALL-E, Whisper	Chat, Completions, Embeddings, Images, Audio, Realtime	OpenAI →
Anthropic	Claude 3.5, Claude 3, Claude 2	Chat, Vision, Function Calling	Anthropic →
Azure OpenAI	GPT-4, GPT-3.5, Embeddings	All OpenAI features via Azure	Azure OpenAI →
Google Gemini	Gemini 2.0, Gemini 1.5 Pro/Flash	Chat, Vision, Embeddings, Function Calling	Google Gemini →
AWS Bedrock	Claude, Llama, Mistral, Titan	Chat, Embeddings, Converse API	AWS Bedrock →
Cohere	Command, Command R, Command R+	Chat, Embeddings, Rerank	Cohere →
Mistral AI	Mistral Large, Medium, Small	Chat, Embeddings, Function Calling	Mistral →

Specialized Providers

Provider	Models	Specialty	Documentation
Together AI	100+ open models	Open-source models, Fast inference	Together AI →
Anyscale	Llama, Mistral, Mixtral	Open models with Endpoints	Anyscale →
Groq	Llama, Mixtral, Gemma	Ultra-fast inference (500+ tokens/s)	Groq →
DeepInfra	100+ models	Cost-effective inference	DeepInfra →
Perplexity	Sonar models	Search-augmented generation	Perplexity →
Ollama	Any local model	Local/self-hosted models	Ollama →
Fireworks AI	80+ models	Fast inference, fine-tuning	Fireworks AI
Replicate	Thousands of models	Community models, image gen	Replicate

Cloud AI Platforms

Provider	Description
Google Vertex AI	Google Cloud AI platform with Gemini, PaLM
Azure AI Inference	Microsoft’s unified AI inference service
Sagemaker	AWS machine learning platform
Workers AI	Cloudflare’s edge AI platform

Additional Providers (A-Z)

A-C Providers

302.AI - AI model aggregation platform
AI21 - Jamba models
AIBadgr - Educational AI platform
Anyscale - Ray-based inference
Cerebras - Ultra-fast inference
CometAPI - API marketplace
Cohere - Enterprise NLP
Cortex - Snowflake AI

D-I Providers

DashScope - Alibaba AI platform
DeepBricks - AI infrastructure
DeepInfra - Cost-effective inference
DeepSeek - Chinese AI models
Featherless AI - Lightweight models
Fireworks AI - Fast inference platform
HuggingFace - 100,000+ models
Hyperbolic - Decentralized AI
Inference.net - Distributed inference
IO Intelligence - Enterprise AI

J-O Providers

Jina - Embeddings and search
Kluster AI - Cluster computing
Krutrim - Indian AI models
Lambda - GPU cloud
LemonfoxAI - AI infrastructure
Lepton - Simplified AI deployment
LingYi - Chinese AI models
MatterAI - Scientific AI
Meshy - 3D generation
Milvus - Vector database
Modal - Serverless AI
MonsterAPI - Cost-effective inference
Moonshot - Chinese AI platform
NCompass - Enterprise AI
Nebius - Cloud AI platform
NextBit - AI infrastructure
Nomic - Embeddings (Nomic Embed)
Novita AI - Multi-modal AI
NScale - Scalable inference
Ollama - Local models
OpenRouter - Model router
Oracle - Oracle Cloud AI
OVHcloud - European cloud AI

P-Z Providers

PaLM - Google’s legacy models
Perplexity AI - Search-augmented LLMs
Predibase - Fine-tuning platform
Qdrant - Vector search
Recraft AI - Image generation
Reka AI - Multimodal models
Replicate - Community model hosting
SambaNova - AI hardware acceleration
Segmind - Image generation
SiliconFlow - Chinese AI platform
Stability AI - Stable Diffusion
Together AI - Open-source models
Triton - NVIDIA Triton
Tripo3D - 3D generation
Upstage - Korean AI models
Voyage - Embeddings
Workers AI - Cloudflare edge AI
X.AI - Grok models
Z.AI - AI infrastructure
Zhipu - Chinese AI (ChatGLM)

Provider Identifier Reference

When making requests, use these provider identifiers:

# Syntax
client = Portkey(
    provider="<provider-identifier>",
    Authorization="<api-key>"
)

Common Provider Identifiers

Provider Name	Identifier	Example
OpenAI	`openai`	`provider="openai"`
Anthropic	`anthropic`	`provider="anthropic"`
Azure OpenAI	`azure-openai`	`provider="azure-openai"`
Google Gemini	`google`	`provider="google"`
AWS Bedrock	`bedrock`	`provider="bedrock"`
Cohere	`cohere`	`provider="cohere"`
Mistral AI	`mistral-ai`	`provider="mistral-ai"`
Together AI	`together-ai`	`provider="together-ai"`
Anyscale	`anyscale`	`provider="anyscale"`
Groq	`groq`	`provider="groq"`
Perplexity	`perplexity-ai`	`provider="perplexity-ai"`
DeepInfra	`deepinfra`	`provider="deepinfra"`
Ollama	`ollama`	`provider="ollama"`
Fireworks AI	`fireworks-ai`	`provider="fireworks-ai"`
Replicate	`replicate`	`provider="replicate"`

Feature Support Matrix

Core Features

Provider	Chat	Streaming	Embeddings	Function Calling	Vision
OpenAI	✅	✅	✅	✅	✅
Anthropic	✅	✅	❌	✅	✅
Azure OpenAI	✅	✅	✅	✅	✅
Google Gemini	✅	✅	✅	✅	✅
AWS Bedrock	✅	✅	✅	✅	✅
Cohere	✅	✅	✅	✅	❌
Mistral	✅	✅	✅	✅	❌
Together AI	✅	✅	✅	✅	✅
Anyscale	✅	✅	✅	✅	❌
Groq	✅	✅	❌	✅	✅
DeepInfra	✅	✅	❌	✅	✅
Perplexity	✅	✅	❌	❌	❌
Ollama	✅	✅	✅	❌	✅

Special Features

Provider	Audio (TTS)	Audio (STT)	Image Generation	Batch API	Fine-tuning
OpenAI	✅	✅	✅	✅	✅
Anthropic	❌	❌	❌	✅	❌
Azure OpenAI	✅	✅	✅	✅	✅
AWS Bedrock	❌	❌	✅	✅	✅
Stability AI	❌	❌	✅	❌	❌
Fireworks AI	❌	❌	✅	❌	✅

Request Examples

Basic Provider Switching

from portkey_ai import Portkey

# OpenAI
openai_client = Portkey(provider="openai", Authorization="sk-***")
response = openai_client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello"}]
)

# Switch to Anthropic - same code structure!
anthropic_client = Portkey(provider="anthropic", Authorization="sk-ant-***")
response = anthropic_client.chat.completions.create(
    model="claude-3-5-sonnet-20241022",
    messages=[{"role": "user", "content": "Hello"}]
)

Multi-Provider Fallback

config = {
    "strategy": {"mode": "fallback"},
    "targets": [
        {"provider": "openai", "api_key": "sk-***"},
        {"provider": "anthropic", "api_key": "sk-ant-***"},
        {"provider": "google", "api_key": "***"},
        {"provider": "groq", "api_key": "gsk-***"}
    ]
}

client = Portkey().with_options(config=config)

Adding New Providers

Portkey regularly adds new providers. To request a provider integration:

Check the GitHub issues for existing requests
Open a feature request with provider details
Contribute a provider implementation

Contribute a Provider

Help add new providers to the gateway

Provider Pricing

For detailed pricing information across all providers, visit:

Portkey Models

Browse pricing for 2,300+ models across 40+ providers

Next Steps

Provider Overview

Learn how provider routing works

OpenAI

OpenAI integration guide

Fallbacks

Set up automatic fallbacks

Load Balancing

Distribute across providers

Overview

Major Providers

Specialized Providers

Overview

All Supported Providers

Major LLM Providers

Specialized Providers

Cloud AI Platforms

Additional Providers (A-Z)

Provider Identifier Reference

Common Provider Identifiers

Feature Support Matrix

Core Features

Special Features

Request Examples

Basic Provider Switching

Multi-Provider Fallback

Adding New Providers

Contribute a Provider

Provider Pricing

Portkey Models

Next Steps

Provider Overview

OpenAI

Fallbacks

Load Balancing

Build docs developers (and LLMs) love

Overview

Major Providers

Specialized Providers

​Overview

​All Supported Providers

​Major LLM Providers

​Specialized Providers

​Cloud AI Platforms

​Additional Providers (A-Z)

​Provider Identifier Reference

​Common Provider Identifiers

​Feature Support Matrix

​Core Features

​Special Features

​Request Examples

​Basic Provider Switching

​Multi-Provider Fallback

​Adding New Providers

Contribute a Provider

​Provider Pricing

Portkey Models

​Next Steps

Provider Overview

OpenAI

Fallbacks

Load Balancing

Build docs developers (and LLMs) love

Overview

All Supported Providers

Major LLM Providers

Specialized Providers

Cloud AI Platforms

Additional Providers (A-Z)

Provider Identifier Reference

Common Provider Identifiers

Feature Support Matrix

Core Features

Special Features

Request Examples

Basic Provider Switching

Multi-Provider Fallback

Adding New Providers

Provider Pricing

Next Steps