Custom LLM Providers

Manifest’s smart routing requires API keys for the LLM providers you want to use. This guide shows how to configure providers for intelligent model selection.

Supported Providers

Manifest supports the following providers:

Provider	Models	Authentication
OpenAI	GPT-4o, GPT-4o-mini, o1, o1-mini, o3-mini	API key (starts with `sk-`)
Anthropic	Claude 3.5 Sonnet, Claude 3.5 Haiku, Claude 3 Opus	API key (starts with `sk-ant-`)
Google	Gemini 2.0 Flash, Gemini 1.5 Pro, Gemini 1.5 Flash	API key
DeepSeek	DeepSeek V3, DeepSeek Chat	API key
OpenRouter	All OpenRouter models	API key (starts with `sk-or-`)
Ollama	Local models (Llama, Mistral, etc.)	None (local)

Adding a Provider

Navigate to Routing settings

In the Manifest dashboard:

Cloud mode: Sign in at app.manifest.build → Select agent → Routing
Local mode: Go to http://127.0.0.1:2099 → Routing

Add a provider

Click Providers → + Add ProviderSelect the provider from the dropdown:

OpenAI
Anthropic
Google
DeepSeek
OpenRouter
Ollama

Enter your API key

Paste your API key in the input field:

sk-proj-abcd1234...

The key is encrypted using AES-256-GCM before storage.

Save

Click Save. The provider is now active and available for routing.The UI shows:

Provider name
Key prefix (first 8 characters, e.g., sk-proj-...)
Status (Active/Inactive)
Connected models

Provider-Specific Setup

OpenAI

Get an API key

Go to platform.openai.com/api-keys and create a new key.Key format: sk-proj-... (newer format) or sk-... (legacy format)

Add to Manifest

Providers → + Add Provider → OpenAI
API Key: sk-proj-abcd1234...

Available models

After adding OpenAI, these models become available:

gpt-4o — Latest GPT-4 Optimized
gpt-4o-mini — Faster, cheaper GPT-4 variant
o1 — Reasoning model (high latency, high cost)
o1-mini — Smaller reasoning model
o3-mini — Newest reasoning model
gpt-3.5-turbo — Legacy fast model

Anthropic

Get an API key

Go to console.anthropic.com/settings/keys and create a new key.Key format: sk-ant-api03-...

Add to Manifest

Providers → + Add Provider → Anthropic
API Key: sk-ant-api03-abcd1234...

Available models

After adding Anthropic:

claude-3-5-sonnet-20241022 — Latest Sonnet (best balance)
claude-3-5-haiku-20241022 — Fastest Claude model
claude-3-opus-20240229 — Highest quality (expensive)

Google

Get an API key

Go to aistudio.google.com/apikey and create a new key.

Add to Manifest

Providers → + Add Provider → Google
API Key: AIzaSy...

Available models

After adding Google:

gemini-2.0-flash-exp — Latest experimental Flash model
gemini-1.5-pro — High-quality, multimodal
gemini-1.5-flash — Fast, cost-effective

DeepSeek

Get an API key

Add to Manifest

Providers → + Add Provider → DeepSeek
API Key: sk-...

Available models

After adding DeepSeek:

deepseek-chat — Latest chat model
deepseek-reasoner — Reasoning-focused model (similar to o1)

OpenRouter

Get an API key

Go to openrouter.ai/keys and create a new key.Key format: sk-or-v1-...

Add to Manifest

Providers → + Add Provider → OpenRouter
API Key: sk-or-v1-abcd1234...

Available models

OpenRouter provides access to 100+ models from multiple providers. Manifest automatically syncs supported models from OpenRouter’s catalog.Popular OpenRouter models:

anthropic/claude-3.5-sonnet
openai/gpt-4o
google/gemini-pro
meta-llama/llama-3.1-405b

Ollama (Local)

Install Ollama

Download Ollama from ollama.com and install it.

Pull models

ollama pull llama3.1
ollama pull mistral

Start Ollama server

ollama serve

Default endpoint: http://127.0.0.1:11434

Add to Manifest

Providers → + Add Provider → Ollama
API Key: (leave blank)

No API key is required for Ollama.

Available models

Manifest detects locally-installed Ollama models:

llama3.1 — Meta’s Llama 3.1
mistral — Mistral 7B
codellama — Code-specialized Llama
Any other pulled models

Automatic Model Discovery

When you add a provider, Manifest automatically:

Fetches the model list from the provider’s API
Syncs pricing data from the model_pricing table
Enables models for routing based on tier compatibility
Recalculates tier assignments to optimize cost and quality

You don’t need to manually configure which models are available.

Tier Auto-Assignment

Manifest assigns models to tiers automatically based on:

Cost — Cheaper models preferred when quality is sufficient
Capability — Complex tasks get more powerful models
Provider availability — Only uses models from active providers

Example tier assignments after adding OpenAI + Anthropic:

Tier	Auto-Assigned Model	Reason
Simple	`gpt-4o-mini`	Cheapest, fast, good for simple queries
Standard	`claude-3-5-haiku-20241022`	Balanced cost and quality
Complex	`gpt-4o`	High capability, reasonable cost
Reasoning	`o1-mini`	Best reasoning quality

See LLM Routing for details on how the scorer assigns tiers.

Removing a Provider

Navigate to Providers

Dashboard → Routing → Providers

Remove the provider

Click the X or Remove button next to the provider.

Confirm

A warning will appear if any tier overrides use models from this provider:

gpt-4o-mini is no longer available.
Simple tier is back to automatic mode (claude-3-5-haiku-20241022).

Overrides are cleared automatically, and the tier falls back to auto-assignment.

Recalculation

Manifest recalculates tier assignments using the remaining providers.

Provider Aliases

Manifest recognizes common provider aliases:

Canonical Name	Aliases
OpenAI	`openai`, `OpenAI`
Anthropic	`anthropic`, `Anthropic`
Google	`google`, `Google`, `gemini`, `Gemini`
DeepSeek	`deepseek`, `DeepSeek`
OpenRouter	`openrouter`, `OpenRouter`
Ollama	`ollama`, `Ollama`

Aliases are case-insensitive and expanded internally by the expandProviderNames function (packages/backend/src/routing/provider-aliases.ts:7).

API Key Security

Encryption

API keys are encrypted before storage using AES-256-GCM:

// packages/backend/src/common/utils/crypto.util.ts
encrypt(apiKey, getEncryptionSecret())

The encryption secret is derived from the BETTER_AUTH_SECRET environment variable. Keys are never stored in plaintext.

Key Prefix Display

The UI shows only the first 8 characters of your key (e.g., sk-proj-...) for verification. The full key is never displayed after initial entry.

Key Rotation

To rotate an API key:

Get a new key from the provider

Update the provider in Manifest:

Providers → OpenAI → Edit → Enter new key → Save

Delete the old key from the provider’s dashboard

Manifest uses the new key immediately for all requests.

Rate Limiting

Manifest respects provider rate limits:

429 Too Many Requests — Manifest backs off exponentially
503 Service Unavailable — Manifest retries with jitter

If a provider is rate-limited, Manifest may fall back to another provider in the same tier.

Cost Optimization

To minimize costs:

Add multiple providers — Manifest chooses the cheapest model for each tier
Use tier overrides sparingly — Auto-assignment optimizes cost automatically
Monitor usage — Set up notification rules for cost thresholds
Remove expensive providers — If you don’t need reasoning models, skip OpenAI o1

Troubleshooting

Provider not appearing after adding

Check the browser console for errors. Common causes:

Invalid API key format
Network error communicating with Manifest backend
Provider API is down

Remove and re-add the provider:

Providers → Remove → + Add Provider

Models not available for routing

Verify the provider is active:

Providers → OpenAI → Status: Active

If inactive, the provider’s API key may be invalid. Update the key:

Providers → OpenAI → Edit → Enter new key → Save

Routing not using my preferred model

Manifest automatically selects models based on cost and capability. To force a specific model:

Routing → Tiers → Complex → Override Model → gpt-4o → Save

This locks the Complex tier to gpt-4o regardless of cost.

API key validation errors

If you see “Auth invalid” in the logs:

Verify the key is correct (copy from provider dashboard)
Check the key hasn’t expired (some providers expire keys after inactivity)

Test the key manually:

curl https://api.openai.com/v1/models \
  -H "Authorization: Bearer sk-proj-..."

If the manual test fails, regenerate the key from the provider.

Next Steps

Routing

Learn how Manifest routes requests across models

Costs

Track and optimize your LLM spending

Get Started

Core Concepts

Features

Guides

Custom LLM Providers

Supported Providers

Adding a Provider

Provider-Specific Setup

OpenAI

Anthropic

Google

DeepSeek

OpenRouter

Ollama (Local)

Automatic Model Discovery

Tier Auto-Assignment

Removing a Provider

Provider Aliases

API Key Security

Encryption

Key Prefix Display

Key Rotation

Rate Limiting

Cost Optimization

Troubleshooting

Provider not appearing after adding

Models not available for routing

Routing not using my preferred model

API key validation errors

Next Steps

Routing

Costs

Build docs developers (and LLMs) love

Get Started

Core Concepts

Features

Guides

​Supported Providers

​Adding a Provider

​Provider-Specific Setup

​OpenAI

​Anthropic

​Google

​DeepSeek

​OpenRouter

​Ollama (Local)

​Automatic Model Discovery

​Tier Auto-Assignment

​Removing a Provider

​Provider Aliases

​API Key Security

​Encryption

​Key Prefix Display

​Key Rotation

​Rate Limiting

​Cost Optimization

​Troubleshooting

​Provider not appearing after adding

​Models not available for routing

​Routing not using my preferred model

​API key validation errors

​Next Steps

Routing

Costs

Build docs developers (and LLMs) love

Supported Providers

Adding a Provider

Provider-Specific Setup

OpenAI

Anthropic

Google

DeepSeek

OpenRouter

Ollama (Local)

Automatic Model Discovery

Tier Auto-Assignment

Removing a Provider

Provider Aliases

API Key Security

Encryption

Key Prefix Display

Key Rotation

Rate Limiting

Cost Optimization

Troubleshooting

Provider not appearing after adding

Models not available for routing

Routing not using my preferred model

API key validation errors

Next Steps