Skip to main content
The LLM Gateway API provides a unified interface for interacting with multiple LLM providers through OpenAI-compatible endpoints. The gateway handles intelligent routing, caching, cost optimization, and provider fallback.

​
Base URL

The Gateway API is available at:
  • Production: https://api.llmgateway.io
  • Local Development: http://localhost:4001

​
Key Features

​
Multi-Provider Support

Route requests to multiple providers including:
  • OpenAI
  • Anthropic
  • Google Vertex AI
  • AWS Bedrock
  • Azure OpenAI
  • And more

​
Intelligent Routing

The gateway automatically:
  • Selects the best provider based on cost, latency, and uptime
  • Falls back to alternative providers on failure
  • Routes to model-specific providers when requested
  • Handles provider-specific request/response transformations

​
Cost Optimization

  • Automatic selection of cheapest available provider
  • Response caching with Redis to reduce redundant requests
  • Usage tracking and cost analytics
  • Support for free models

​
Enterprise Features

  • API key management
  • Organization and project isolation
  • IAM-based access control
  • Rate limiting and usage quotas
  • Data retention controls
  • Guardrails and content filtering

​
API Endpoints

The Gateway API exposes the following endpoints:
EndpointDescription
POST /v1/chat/completionsCreate chat completions using OpenAI format
POST /v1/images/generationsGenerate images from text prompts
POST /v1/images/editsEdit images with text prompts
GET /v1/modelsList all available models
POST /v1/messagesCreate messages using Anthropic format
GET /Health check endpoint
GET /metricsPrometheus metrics

​
Content Type

All POST requests must include the Content-Type: application/json header. The gateway validates this and returns a 415 Unsupported Media Type error if missing. Exceptions:
  • /mcp endpoint (handles its own content type validation)
  • /oauth endpoints (accept form-urlencoded or JSON)
  • /v1/images/edits endpoint (accepts multipart/form-data for file uploads)

​
Error Handling

The gateway returns standardized error responses: 
{
  "error": true,
  "status": 400,
  "message": "Error description"
}
Common HTTP status codes:
  • 400 - Bad Request (invalid parameters)
  • 401 - Unauthorized (missing or invalid API key)
  • 402 - Payment Required (insufficient credits)
  • 403 - Forbidden (IAM policy violation)
  • 410 - Gone (project archived)
  • 415 - Unsupported Media Type (missing Content-Type header)
  • 429 - Too Many Requests (rate limit exceeded)
  • 499 - Client Closed Request (client disconnected)
  • 500 - Internal Server Error
  • 503 - Service Unavailable (health check failed)
  • 504 - Gateway Timeout (upstream provider timeout)

​
Health Check

The root endpoint / provides health status: 
curl https://api.llmgateway.io/

{
  "message": "LLM Gateway is running",
  "version": "1.0.0",
  "health": {
    "status": "healthy",
    "redis": {
      "connected": true
    },
    "database": {
      "connected": true
    }
  }
}
Query parameters:
  • skip - Comma-separated list of health checks to skip (e.g., redis,database)

​
Next Steps

Authentication

Learn how to authenticate with the Gateway API

Chat Completions

Create chat completions with multiple providers

Images

Generate and edit images

Models

List available models and capabilities

Build docs developers (and LLMs) love

Get started for freeTalk to us