LLM Providers API - Support Bot

The LLM Providers API enables administrators to configure and manage connections to various Large Language Model providers. All endpoints require admin permissions and support multiple provider types including Anthropic, OpenAI, Google, and custom endpoints.

Authentication

All LLM provider endpoints require authentication and specific permissions:

llm_provider.view - View provider configurations
llm_provider.create - Create new providers
llm_provider.edit - Update provider settings
llm_provider.delete - Delete providers
llm_provider.test - Test provider connections

Provider Types

Supported provider types:

anthropic - Anthropic Claude models
openai - OpenAI GPT models
google - Google Gemini models
custom - Custom OpenAI-compatible endpoints

Custom Authentication Types

For custom providers, the following authentication methods are supported:

none - No authentication required
bearer - Bearer token authentication
basic - Basic HTTP authentication
api_key_header - Custom API key header

LLM Provider Object

string

required

Unique identifier for the provider (UUID)

name

string

required

Display name for the provider (1-100 characters)

provider_type

string

required

Type of provider: anthropic, openai, google, or custom

base_url

string

Base URL for API calls. Optional for standard providers, required for custom endpoints.Default URLs:

Anthropic: https://api.anthropic.com
OpenAI: https://api.openai.com
Google: https://generativelanguage.googleapis.com

has_api_key

boolean

required

Indicates if an API key is configured. The actual key is never returned in responses.

config

object

required

Provider-specific configuration object. May include custom headers, timeouts, or other settings.

models

array

required

Array of available model identifiers (e.g., ["claude-3-5-sonnet-20241022", "gpt-4o"])

is_active

boolean

required

Whether the provider is currently enabled and available for use

is_default

boolean

required

Whether this is the default provider for new conversations

last_health_check_at

string

ISO 8601 timestamp of the last health check

last_health_check_status

string

Status of the last health check: success or error

last_health_check_error

string

Error message from the last health check (if failed)

created_at

string

required

ISO 8601 timestamp when the provider was created

updated_at

string

required

ISO 8601 timestamp of the last update

List All Providers

Returns a list of all LLM providers. Use the active_only query parameter to filter for only enabled providers. Required Permission: llm_provider.view

Query Parameters

active_only

boolean

default:"false"

If true, only returns providers where is_active is true

Response

providers

array

required

Array of LLM provider objects

total

number

required

Total count of providers returned

Example Request

curl -X GET "https://api.example.com/llm-providers/?active_only=true" \
  -H "Authorization: Bearer YOUR_TOKEN"

Example Response

{
  "providers": [
    {
      "id": "550e8400-e29b-41d4-a716-446655440000",
      "name": "Production Claude",
      "provider_type": "anthropic",
      "base_url": null,
      "has_api_key": true,
      "config": {},
      "models": [
        "claude-opus-4-5-20251101",
        "claude-sonnet-4-20250514",
        "claude-3-5-sonnet-20241022"
      ],
      "is_active": true,
      "is_default": true,
      "last_health_check_at": "2026-03-01T10:30:00Z",
      "last_health_check_status": "success",
      "last_health_check_error": null,
      "created_at": "2026-02-15T08:00:00Z",
      "updated_at": "2026-03-01T10:30:00Z"
    },
    {
      "id": "660e8400-e29b-41d4-a716-446655440001",
      "name": "OpenAI GPT",
      "provider_type": "openai",
      "base_url": null,
      "has_api_key": true,
      "config": {},
      "models": ["gpt-4o", "gpt-4o-mini"],
      "is_active": true,
      "is_default": false,
      "last_health_check_at": "2026-03-01T09:15:00Z",
      "last_health_check_status": "success",
      "last_health_check_error": null,
      "created_at": "2026-02-20T10:00:00Z",
      "updated_at": "2026-03-01T09:15:00Z"
    }
  ],
  "total": 2
}

Create Provider

Creates a new LLM provider. API keys are automatically encrypted before storage. If the models array is empty, default models for the provider type will be used. Required Permission: llm_provider.create

Request Body

name

string

required

Display name for the provider (1-100 characters)

provider_type

string

required

Provider type: anthropic, openai, google, or custom

api_key

string

API key for authentication. Will be encrypted before storage.

base_url

string

Custom base URL. Optional for standard providers, required for custom type.

config

object

default:"{}"

Provider-specific configuration (custom headers, timeouts, etc.)

models

array

default:"[]"

Array of model identifiers. If empty, defaults are used:Anthropic defaults:

claude-opus-4-5-20251101
claude-sonnet-4-20250514
claude-3-5-sonnet-20241022
claude-3-5-haiku-20241022
claude-3-opus-20240229
claude-3-sonnet-20240229
claude-3-haiku-20240307

OpenAI defaults:

gpt-4o
gpt-4o-mini
gpt-4-turbo
gpt-4
gpt-3.5-turbo
o1-preview
o1-mini

Google defaults:

gemini-2.5-pro
gemini-2.5-flash
gemini-2.0-flash
gemini-2.0-flash-lite-001
gemini-1.5-pro
gemini-1.5-flash

is_active

boolean

default:"true"

Enable the provider immediately after creation

is_default

boolean

default:"false"

Set as the default provider for new conversations

Response

Returns the created LLM provider object with status code 201 Created.

Example Request

curl -X POST https://api.example.com/llm-providers/ \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Production Claude",
    "provider_type": "anthropic",
    "api_key": "sk-ant-api03-...",
    "models": [
      "claude-3-5-sonnet-20241022",
      "claude-3-5-haiku-20241022"
    ],
    "is_active": true,
    "is_default": true
  }'

Example Response

{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "name": "Production Claude",
  "provider_type": "anthropic",
  "base_url": null,
  "has_api_key": true,
  "config": {},
  "models": [
    "claude-3-5-sonnet-20241022",
    "claude-3-5-haiku-20241022"
  ],
  "is_active": true,
  "is_default": true,
  "last_health_check_at": null,
  "last_health_check_status": null,
  "last_health_check_error": null,
  "created_at": "2026-03-01T12:00:00Z",
  "updated_at": "2026-03-01T12:00:00Z"
}

Get Available Models

Aggregates models from all active providers into a single list. Each model includes its provider information for UI selection. Required Permission: llm_provider.view

Response

models

array

required

Array of available model objects

Model Object

provider_id

string

required

UUID of the provider

provider_name

string

required

Display name of the provider

provider_type

string

required

Type of provider: anthropic, openai, google, or custom

model_id

string

required

Model identifier (e.g., “claude-3-5-sonnet-20241022”)

display_name

string

required

Human-readable model name

is_default_provider

boolean

required

Whether this model’s provider is the default

Example Request

curl -X GET https://api.example.com/llm-providers/models/available \
  -H "Authorization: Bearer YOUR_TOKEN"

Example Response

{
  "models": [
    {
      "provider_id": "550e8400-e29b-41d4-a716-446655440000",
      "provider_name": "Production Claude",
      "provider_type": "anthropic",
      "model_id": "claude-3-5-sonnet-20241022",
      "display_name": "Claude 3.5 Sonnet",
      "is_default_provider": true
    },
    {
      "provider_id": "660e8400-e29b-41d4-a716-446655440001",
      "provider_name": "OpenAI GPT",
      "provider_type": "openai",
      "model_id": "gpt-4o",
      "display_name": "GPT-4o",
      "is_default_provider": false
    }
  ]
}

Get Provider by ID

Fetches detailed information about a single LLM provider. API keys are never returned in responses. Required Permission: llm_provider.view

Path Parameters

provider_id

string

required

UUID of the provider to retrieve

Response

Returns the LLM provider object or 404 Not Found if the provider doesn’t exist.

Example Request

curl -X GET https://api.example.com/llm-providers/550e8400-e29b-41d4-a716-446655440000 \
  -H "Authorization: Bearer YOUR_TOKEN"

Example Response

{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "name": "Production Claude",
  "provider_type": "anthropic",
  "base_url": null,
  "has_api_key": true,
  "config": {},
  "models": [
    "claude-3-5-sonnet-20241022",
    "claude-3-5-haiku-20241022"
  ],
  "is_active": true,
  "is_default": true,
  "last_health_check_at": "2026-03-01T10:30:00Z",
  "last_health_check_status": "success",
  "last_health_check_error": null,
  "created_at": "2026-02-15T08:00:00Z",
  "updated_at": "2026-03-01T10:30:00Z"
}

Error Response

{
  "detail": "Provider not found"
}

Update Provider

Updates a provider configuration. Only provided fields are updated; omitted fields remain unchanged. To update the API key, include it in the request. Required Permission: llm_provider.edit

Path Parameters

provider_id

string

required

UUID of the provider to update

Request Body

All fields are optional. Only include fields you want to update.

name

string

New display name (1-100 characters)

base_url

string

Updated base URL

api_key

string

New API key (will be encrypted). Omit to keep existing key.

config

object

Updated configuration object

models

array

Updated list of model identifiers

is_active

boolean

Enable or disable the provider

is_default

boolean

Set or unset as default provider

Response

Returns the updated provider object or 404 Not Found if the provider doesn’t exist.

Example Request

curl -X PUT https://api.example.com/llm-providers/550e8400-e29b-41d4-a716-446655440000 \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Production Claude (Updated)",
    "models": [
      "claude-opus-4-5-20251101",
      "claude-3-5-sonnet-20241022"
    ],
    "is_active": true
  }'

Example Response

{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "name": "Production Claude (Updated)",
  "provider_type": "anthropic",
  "base_url": null,
  "has_api_key": true,
  "config": {},
  "models": [
    "claude-opus-4-5-20251101",
    "claude-3-5-sonnet-20241022"
  ],
  "is_active": true,
  "is_default": true,
  "last_health_check_at": "2026-03-01T10:30:00Z",
  "last_health_check_status": "success",
  "last_health_check_error": null,
  "created_at": "2026-02-15T08:00:00Z",
  "updated_at": "2026-03-01T13:15:00Z"
}

Delete Provider

Permanently removes a provider and its configuration. This action cannot be undone. Required Permission: llm_provider.delete

Path Parameters

provider_id

string

required

UUID of the provider to delete

Response

Returns 204 No Content on success or 404 Not Found if the provider doesn’t exist.

Example Request

curl -X DELETE https://api.example.com/llm-providers/550e8400-e29b-41d4-a716-446655440000 \
  -H "Authorization: Bearer YOUR_TOKEN"

Success Response

Status: 204 No Content (no body)

Error Response

{
  "detail": "Provider not found"
}

Test Provider Connection

Performs a health check by attempting to connect to the provider’s API. Updates the provider’s health check status in the database. Required Permission: llm_provider.test

Path Parameters

provider_id

string

required

UUID of the provider to test

Response

success

boolean

required

Whether the health check succeeded

provider_id

string

required

UUID of the tested provider

status

string

required

Status of the health check: success or error

message

string

Error message if the test failed

response_time_ms

number

Response time in milliseconds

Example Request

curl -X POST https://api.example.com/llm-providers/550e8400-e29b-41d4-a716-446655440000/test \
  -H "Authorization: Bearer YOUR_TOKEN"

Success Response

{
  "success": true,
  "provider_id": "550e8400-e29b-41d4-a716-446655440000",
  "status": "success",
  "message": null,
  "response_time_ms": 342.5
}

Error Response

{
  "success": false,
  "provider_id": "550e8400-e29b-41d4-a716-446655440000",
  "status": "error",
  "message": "Invalid API key",
  "response_time_ms": 156.2
}

Discover Models from Provider

Fetches the list of available models directly from the provider’s API endpoint. Requires valid credentials to be configured. Required Permission: llm_provider.view

Path Parameters

provider_id

string

required

UUID of the provider to query for models

Response

success

boolean

required

Whether the discovery succeeded

models

array

Array of model identifier strings discovered from the provider

message

string

Status or error message

Example Request

curl -X POST https://api.example.com/llm-providers/550e8400-e29b-41d4-a716-446655440000/discover-models \
  -H "Authorization: Bearer YOUR_TOKEN"

Success Response

{
  "success": true,
  "models": [
    "claude-opus-4-5-20251101",
    "claude-sonnet-4-20250514",
    "claude-3-5-sonnet-20241022",
    "claude-3-5-haiku-20241022",
    "claude-3-opus-20240229",
    "claude-3-sonnet-20240229",
    "claude-3-haiku-20240307"
  ],
  "message": "Successfully discovered 7 models"
}

Error Response

{
  "success": false,
  "models": [],
  "message": "Failed to connect to provider API: Invalid authentication credentials"
}

Discover Models from New Config

Use this endpoint to test provider credentials and discover available models before creating a provider configuration. Accepts the same payload as provider creation but only performs discovery. Required Permission: llm_provider.create

Request Body

Accepts the same fields as the Create Provider endpoint:

provider_type

string

required

Provider type: anthropic, openai, google, or custom

api_key

string

API key for authentication (not saved)

base_url

string

Custom base URL (for custom providers)

config

object

default:"{}"

Provider-specific configuration

name

string

required

Provider name (required but not used for discovery)

models

array

default:"[]"

Not used for discovery

is_active

boolean

default:"true"

Not used for discovery

is_default

boolean

default:"false"

Not used for discovery

Response

success

boolean

required

Whether the discovery succeeded

models

array

Array of model identifier strings discovered from the provider

message

string

Status or error message

Example Request

curl -X POST https://api.example.com/llm-providers/discover-models \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Test Provider",
    "provider_type": "openai",
    "api_key": "sk-...",
    "base_url": "https://api.openai.com"
  }'

Success Response

{
  "success": true,
  "models": [
    "gpt-4o",
    "gpt-4o-mini",
    "gpt-4-turbo",
    "gpt-4",
    "gpt-3.5-turbo",
    "o1-preview",
    "o1-mini"
  ],
  "message": "Successfully discovered 7 models"
}

Error Response

{
  "success": false,
  "models": [],
  "message": "Authentication failed: Invalid API key provided"
}

Use Case

This endpoint is particularly useful for:

Pre-validation - Test credentials before creating a provider
UI workflows - Populate model selection dropdowns dynamically
Custom providers - Verify OpenAI-compatible endpoints return valid model lists
Troubleshooting - Diagnose authentication or connectivity issues

Example Workflow

# Step 1: Discover models from new credentials
curl -X POST https://api.example.com/llm-providers/discover-models \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "My Custom Provider",
    "provider_type": "custom",
    "api_key": "sk-custom-123",
    "base_url": "https://custom-llm.example.com/v1"
  }'

# Step 2: If successful, create the provider with discovered models
curl -X POST https://api.example.com/llm-providers/ \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "My Custom Provider",
    "provider_type": "custom",
    "api_key": "sk-custom-123",
    "base_url": "https://custom-llm.example.com/v1",
    "models": ["model-1", "model-2"],
    "is_active": true
  }'

Authentication

Chat

Knowledge Base

Feedback

Administration

​Authentication

​Provider Types

​Custom Authentication Types

​LLM Provider Object

​List All Providers

​Query Parameters

​Response

​Example Request

​Example Response

​Create Provider

​Request Body

​Response

​Example Request

​Example Response

​Get Available Models

​Response

​Model Object

​Example Request

​Example Response

​Get Provider by ID

​Path Parameters

​Response

​Example Request

​Example Response

​Error Response

​Update Provider

​Path Parameters

​Request Body

​Response

​Example Request

​Example Response

​Delete Provider

​Path Parameters

​Response

​Example Request

​Success Response

​Error Response

​Test Provider Connection

​Path Parameters

​Response

​Example Request

​Success Response

​Error Response

​Discover Models from Provider

​Path Parameters

​Response

​Example Request

​Success Response

​Error Response

​Discover Models from New Config

​Request Body

​Response

​Example Request

​Success Response

​Error Response

​Use Case

​Example Workflow

Build docs developers (and LLMs) love

Authentication

Provider Types

Custom Authentication Types

LLM Provider Object

List All Providers

Query Parameters

Response

Example Request

Example Response

Create Provider

Request Body

Response

Example Request

Example Response

Get Available Models

Response

Model Object

Example Request

Example Response

Get Provider by ID

Path Parameters

Response

Example Request

Example Response

Error Response

Update Provider

Path Parameters

Request Body

Response

Example Request

Example Response

Delete Provider

Path Parameters

Response

Example Request

Success Response

Error Response

Test Provider Connection

Path Parameters

Response

Example Request

Success Response

Error Response

Discover Models from Provider

Path Parameters

Response

Example Request

Success Response

Error Response

Discover Models from New Config

Request Body

Response

Example Request

Success Response

Error Response

Use Case

Example Workflow