Skip to main content
OpenFang supports 27 LLM providers spanning cloud APIs, self-hosted solutions, and specialized coding assistants. Each provider is configured via environment variables for API keys and optional TOML settings for custom endpoints.

​
Provider List

OpenFang supports these providers out of the box:

Anthropic

Claude Opus/Sonnet/Haiku

OpenAI

GPT-4o, GPT-4, o1, o3

Google

Gemini 2.0 Flash/Pro

Groq

Ultra-fast inference

DeepSeek

Coding specialists

xAI

Grok models

OpenRouter

Multi-provider routing

Mistral

European open models

Together AI

Open source models

Fireworks

Fast open models

Cohere

Command R+

Perplexity

Search-augmented LLMs

AI21 Labs

Jamba models

Cerebras

Fastest inference

SambaNova

Enterprise inference

Hugging Face

70k+ models

Replicate

Cloud model hosting

Ollama

Local models

vLLM

Self-hosted inference

LM Studio

Desktop inference

Qwen

Alibaba models

MiniMax

Chinese frontier

Zhipu AI

GLM models

Moonshot

Kimi long context

Qianfan

Baidu ERNIE

Bedrock

AWS hosted models

GitHub Copilot

Copilot API

​
Quick Setup

​
1. Anthropic (Claude)

export ANTHROPIC_API_KEY="sk-ant-api03-..."
Available Models: claude-opus-4-20250514, claude-sonnet-4-20250514, claude-haiku-4.5-20251001, claude-opus-4.6, claude-sonnet-4.6 API Key: Get from console.anthropic.com

​
2. OpenAI

export OPENAI_API_KEY="sk-proj-..."
Available Models: gpt-4o, gpt-4o-mini, gpt-4-turbo, gpt-4, o1, o1-mini, o3-mini, gpt-3.5-turbo API Key: Get from platform.openai.com

​
3. Google Gemini

export GEMINI_API_KEY="AIza..."
# or
export GOOGLE_API_KEY="AIza..."
Available Models: gemini-2.0-flash-exp, gemini-2.0-flash-thinking-exp, gemini-1.5-pro, gemini-1.5-flash, gemini-exp-1206 API Key: Get from aistudio.google.com

​
4. Groq

export GROQ_API_KEY="gsk_..."
Available Models: llama-3.3-70b-versatile, llama-3.1-70b-versatile, llama-3.2-90b-vision-preview, mixtral-8x7b-32768 API Key: Get from console.groq.com Free Tier: 30 requests/minute, 7,000 requests/day

​
5. DeepSeek

export DEEPSEEK_API_KEY="sk-..."
Available Models: deepseek-chat, deepseek-reasoner, deepseek-coder API Key: Get from platform.deepseek.com

​
6. Ollama (Local)

# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh

# Pull a model
ollama pull llama3.2
No API Key Required: Ollama runs locally on your machine. Available Models: Any model from ollama.com/library - llama3.2, mistral, codellama, qwen2.5, etc.

​
7. OpenRouter

export OPENROUTER_API_KEY="sk-or-..."
API Key: Get from openrouter.ai/keys Benefits: Access 100+ models through a single API, automatic failover, cost optimization

​
8. xAI (Grok)

export XAI_API_KEY="xai-..."
Available Models: grok-beta, grok-vision-beta API Key: Get from console.x.ai

​
Chinese Providers

​
Qwen (Alibaba)

export DASHSCOPE_API_KEY="sk-..."
Models: qwen-max, qwen-plus, qwen-turbo, qwen3-235b-a22b, qwen-vl-plus, qwen-coder-plus

​
Zhipu AI (GLM)

export ZHIPU_API_KEY="..."
Models: glm-4-plus, glm-4-flash, glm-5-20250605, glm-4v-plus, codegeex-4

​
MiniMax

export MINIMAX_API_KEY="..."
Models: MiniMax-M2.5, MiniMax-M2.1, minimax-text-01, abab6.5-chat Context: Up to 1M tokens

​
Moonshot (Kimi)

export MOONSHOT_API_KEY="sk-..."
Models: moonshot-v1-128k, moonshot-v1-32k, moonshot-v1-8k

​
Baidu Qianfan (ERNIE)

export QIANFAN_API_KEY="..."
Models: ernie-4.5-8k, ernie-4.0-turbo-8k, ernie-speed-128k

​
Self-Hosted & Local

​
vLLM

python -m vllm.entrypoints.openai.api_server \
  --model meta-llama/Llama-3.3-70B-Instruct \
  --port 8000
No API Key Required for local deployment.

​
LM Studio

[default_model]
provider = "lmstudio"
model = "llama-3.3-70b"

[provider_urls]
lmstudio = "http://localhost:1234/v1"
  1. Download LM Studio
  2. Load a model (e.g., Llama 3.3 70B)
  3. Start local server (default port 1234)
  4. Configure OpenFang to use it

​
Enterprise & Cloud

​
AWS Bedrock

export AWS_ACCESS_KEY_ID="AKIA..."
export AWS_SECRET_ACCESS_KEY="..."
export AWS_REGION="us-east-1"
Available Models: bedrock/anthropic.claude-opus-4-6, bedrock/anthropic.claude-sonnet-4-6, bedrock/amazon.nova-pro-v1:0, bedrock/meta.llama3-3-70b-instruct-v1:0

​
GitHub Copilot

export GITHUB_TOKEN="ghp_..."
Requires: Active GitHub Copilot subscription

​
Additional Providers

​
Mistral AI

export MISTRAL_API_KEY="..."
Models: mistral-large, mistral-medium, mistral-small, codestral

​
Together AI

export TOGETHER_API_KEY="..."
Models: Open source models like Llama, Mixtral, Qwen via fast inference

​
Fireworks AI

export FIREWORKS_API_KEY="..."
Models: Ultra-fast inference for Llama, Mixtral, DeepSeek

​
Cohere

export COHERE_API_KEY="..."
Models: command-r-plus, command-r, command

​
Perplexity

export PERPLEXITY_API_KEY="pplx-..."
Models: sonar, sonar-pro - search-augmented responses

​
AI21 Labs

export AI21_API_KEY="..."
Models: jamba-1.5-large, jamba-1.5-mini

​
Cerebras

export CEREBRAS_API_KEY="..."
Models: World’s fastest inference (2,000+ tokens/sec)

​
SambaNova

export SAMBANOVA_API_KEY="..."
Models: Enterprise-grade inference platform

​
Hugging Face

export HF_API_KEY="hf_..."
Models: 70,000+ models via Inference API

​
Replicate

export REPLICATE_API_TOKEN="r8_..."
Models: Cloud-hosted open source models

​
Custom Provider URLs

Override any provider’s base URL: 
[provider_urls]
openai = "https://api.openai.com/v1"           # Default
anthropic = "https://my-proxy.com/anthropic"   # Custom proxy
ollama = "http://192.168.1.100:11434"          # Remote Ollama
vllm = "http://gpu-cluster.internal:8000/v1"   # Internal vLLM

​
Provider Status Check

Check which providers are configured: 
# List all providers and their status
openfang providers status

# Test a specific provider
openfang providers test anthropic

# List models from a provider
openfang models list --provider anthropic

​
Multi-Provider Strategy

[default_model]
provider = "groq"  # Free tier
model = "llama-3.3-70b-versatile"
api_key_env = "GROQ_API_KEY"

[[fallback_providers]]
provider = "ollama"  # Local fallback
model = "llama3.2:latest"

​
Troubleshooting

​
Authentication Errors

# Verify API key is set
echo $ANTHROPIC_API_KEY

# Test provider connection
openfang providers test anthropic

​
Rate Limits

Configure fallback providers to handle rate limits: 
[[fallback_providers]]
provider = "groq"
model = "llama-3.3-70b-versatile"

​
Model Not Found

# List all models from provider
openfang models list --provider anthropic

# Check if provider is configured
openfang providers status

​
Next Steps

Model Configuration

Configure model routing and fallbacks

Channel Setup

Connect messaging platforms