Managing API Keys

Overview

vLLora acts as a unified gateway to multiple AI providers. Each provider requires its own API credentials. This guide covers how to configure and manage API keys for different providers.

Supported Providers

vLLora supports the following AI providers out of the box:

Tier 1 Providers (Built-in)

OpenAI

GPT-4, GPT-4o, GPT-4o-mini, and GPT-3.5 models

Anthropic

Claude 3.5 Sonnet, Claude 3 Opus, and Haiku models

Google Gemini

Gemini Pro and Gemini Flash models

Amazon Bedrock

Access to multiple foundation models via AWS

Additional Providers

DeepSeek - Advanced AI models
xAI (Grok) - X.ai models
DeepInfra - Cloud-hosted open models
Fireworks AI - Fast inference for compound AI
Together AI - Open-source model platform
Azure OpenAI - Microsoft Azure AI services
Groq - LPU-powered ultra-fast inference
Mistral AI - European AI leader
OpenRouter - Multi-model routing
Parasail - Cost-efficient AI deployment
Vertex AI - Google Cloud AI platform
Z.AI (Zhipu) - Chinese LLM provider

Configuring API Keys

Via Web UI (Recommended)

Start vLLora:
```
vllora
```
Open the UI at http://localhost:9091
Navigate to Providers or Settings
Select a provider and click Configure
Enter your API key and save

API keys are stored securely in the local SQLite database and are scoped to your current project.

Via REST API

You can also manage API keys programmatically:

List Providers

curl http://localhost:9090/providers

Response:

[
  {
    "id": "cbe839fa-87f5-4644-9f0c-df45a67237a6",
    "name": "openai",
    "description": "OpenAI develops advanced language models...",
    "endpoint": null,
    "has_credentials": false,
    "is_custom": false
  },
  ...
]

Add/Update Provider Credentials

curl -X PUT http://localhost:9090/providers/openai \
  -H "Content-Type: application/json" \
  -d '{
    "credentials": {
      "api_key": "sk-..."
    }
  }'

Response:

{
  "provider": {
    "id": "cbe839fa-87f5-4644-9f0c-df45a67237a6",
    "name": "openai",
    "has_credentials": true
  }
}

Delete Provider Credentials

curl -X DELETE http://localhost:9090/providers/openai

Provider-Specific Configuration

OpenAI

Get an API key: platform.openai.com/api-keys

{
  "credentials": {
    "api_key": "sk-proj-..."
  }
}

Available models:

gpt-4o - Latest GPT-4 Omni model
gpt-4o-mini - Faster, cheaper GPT-4o
gpt-4-turbo - Previous generation GPT-4
gpt-3.5-turbo - Fast and cost-effective

Anthropic

Get an API key: console.anthropic.com

{
  "credentials": {
    "api_key": "sk-ant-..."
  }
}

Available models:

claude-3-5-sonnet-20241022 - Most capable Claude model
claude-3-opus-20240229 - Highest intelligence
claude-3-sonnet-20240229 - Balanced performance
claude-3-haiku-20240307 - Fastest responses

Google Gemini

Get an API key: aistudio.google.com/apikey

{
  "credentials": {
    "api_key": "AIza..."
  }
}

Available models:

gemini-2.0-flash-exp - Latest experimental flash
gemini-1.5-pro - Powerful and versatile
gemini-1.5-flash - Fast and efficient

Amazon Bedrock

AWS credentials required:

{
  "credentials": {
    "aws_access_key_id": "AKIA...",
    "aws_secret_access_key": "...",
    "aws_region": "us-east-1"
  }
}

Available models:

anthropic.claude-3-5-sonnet-20241022-v2:0
anthropic.claude-3-opus-20240229-v1:0
meta.llama3-70b-instruct-v1:0
And many more via AWS Bedrock

Azure OpenAI

Azure-specific configuration:

{
  "credentials": {
    "api_key": "...",
    "api_base": "https://YOUR_RESOURCE.openai.azure.com/",
    "api_version": "2024-02-15-preview",
    "deployment_name": "gpt-4"
  }
}

Vertex AI (Google Cloud)

Service account required:

{
  "credentials": {
    "project_id": "your-project-id",
    "credentials_json": "{...}",  // Service account JSON
    "location": "us-central1"
  }
}

OpenAI-Compatible Providers

Many providers (DeepSeek, Groq, Fireworks, etc.) use the OpenAI API format:

{
  "credentials": {
    "api_key": "..."
  }
}

The endpoint is pre-configured for each provider.

Custom Providers

You can add custom OpenAI-compatible endpoints:

Create Custom Provider

curl -X POST http://localhost:9090/providers \
  -H "Content-Type: application/json" \
  -d '{
    "provider_name": "my-custom-llm",
    "description": "My self-hosted LLM",
    "endpoint": "https://llm.mycompany.com/v1",
    "custom_inference_api_type": "openai",
    "priority": 50
  }'

Supported API types:

openai - OpenAI-compatible API
anthropic - Anthropic-compatible API
gemini - Google Gemini API
bedrock - AWS Bedrock API

Add Credentials to Custom Provider

curl -X PUT http://localhost:9090/providers/my-custom-llm \
  -H "Content-Type: application/json" \
  -d '{
    "credentials": {
      "api_key": "..."
    }
  }'

Delete Custom Provider

curl -X DELETE http://localhost:9090/providers/definitions/<provider_id>

Only custom providers (created via API) can be deleted. Built-in providers cannot be removed.

Multi-Project Support

vLLora supports multiple projects, each with its own API keys:

# List projects
curl http://localhost:9090/projects

# Create new project
curl -X POST http://localhost:9090/projects \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Production",
    "description": "Production environment"
  }'

API keys are scoped per project. Configure different keys for development, staging, and production.

Security Best Practices

Storage

API keys are stored in the local SQLite database at ~/.vllora/vllora.db
Keys are associated with projects and stored as encrypted credentials
The database is only accessible to the user running vLLora

Access Control

vLLora runs locally and does not expose credentials over the network
API keys are never logged or included in traces
Use project-based separation for different environments

Key Rotation

Generate a new API key from your provider
Update the key in vLLora (via UI or API)
Test that requests work with the new key
Revoke the old key from the provider’s dashboard

Environment Separation

For production deployments:

# Development
vllora --config dev-config.yaml

# Production (different database location)
export HOME=/var/lib/vllora-prod
vllora --config prod-config.yaml

This ensures complete separation of credentials between environments.

Syncing Providers

vLLora can sync provider definitions from the LangDB API:

vllora sync --providers

This updates:

Provider metadata (descriptions, endpoints, etc.)
New providers added to LangDB
Deactivates providers removed from LangDB

Syncing providers does not affect your configured API keys. It only updates provider definitions.

Troubleshooting

Invalid API Key

{
  "error": "invalid_api_key",
  "message": "The API key provided is invalid"
}

Solution: Verify the API key is correct and has not expired.

Provider Not Found

{
  "error": "Provider not found"
}

Solution: Run vllora sync --providers to update provider definitions.

Missing Credentials

{
  "error": "No credentials configured for provider: openai"
}

Solution: Configure API key via UI or API before making requests.

Rate Limiting

vLLora respects provider rate limits. If you hit rate limits:

Configure multiple providers with fallback
Use vLLora’s built-in rate limiting middleware
Upgrade your provider plan for higher limits

Next Steps

Models

Learn about model configuration

Chat Completions

Make your first API request

Monitoring

Monitor API usage and costs

Projects

Manage multiple projects

Get Started

Core Concepts

Features

Guides

​Overview

​Supported Providers

​Tier 1 Providers (Built-in)

OpenAI

Anthropic

Google Gemini

Amazon Bedrock

​Additional Providers

​Configuring API Keys

​Via Web UI (Recommended)

​Via REST API

​List Providers

​Add/Update Provider Credentials

​Delete Provider Credentials

​Provider-Specific Configuration

​OpenAI

​Anthropic

​Google Gemini

​Amazon Bedrock

​Azure OpenAI

​Vertex AI (Google Cloud)

​OpenAI-Compatible Providers

​Custom Providers

​Create Custom Provider

​Add Credentials to Custom Provider

​Delete Custom Provider

​Multi-Project Support

​Security Best Practices

​Storage

​Access Control

​Key Rotation

​Environment Separation

​Syncing Providers

​Troubleshooting

​Invalid API Key

​Provider Not Found

​Missing Credentials

​Rate Limiting

​Next Steps

Models

Chat Completions

Monitoring

Projects

Build docs developers (and LLMs) love

Overview

Supported Providers

Tier 1 Providers (Built-in)

Additional Providers

Configuring API Keys

Via Web UI (Recommended)

Via REST API

List Providers

Add/Update Provider Credentials

Delete Provider Credentials

Provider-Specific Configuration

OpenAI

Anthropic

Google Gemini

Amazon Bedrock

Azure OpenAI

Vertex AI (Google Cloud)

OpenAI-Compatible Providers

Custom Providers

Create Custom Provider

Add Credentials to Custom Provider

Delete Custom Provider

Multi-Project Support

Security Best Practices

Storage

Access Control

Key Rotation

Environment Separation

Syncing Providers

Troubleshooting

Invalid API Key

Provider Not Found

Missing Credentials

Rate Limiting

Next Steps