Skip to main content

Overview

LLM Gateway provides detailed usage analytics for every API request, including token usage, costs, latency, errors, and routing metadata. All data is stored and can be queried via the Analytics API.

What’s Tracked

Every request generates a comprehensive log entry:
apps/api/src/routes/logs.ts
interface LogEntry {
  // Identifiers
  id: string;
  requestId: string;
  organizationId: string;
  projectId: string;
  apiKeyId: string;
  
  // Timestamps and Performance
  createdAt: Date;
  duration: number;  // milliseconds
  
  // Model Information
  requestedModel: string;
  requestedProvider: string | null;
  usedModel: string;
  usedProvider: string;
  
  // Request/Response Data
  messages: any;  // Full conversation history
  content: string | null;  // Response content
  reasoningContent: string | null;  // Extended thinking
  responseSize: number;  // bytes
  
  // Token Usage
  promptTokens: string | null;
  completionTokens: string | null;
  totalTokens: string | null;
  reasoningTokens: string | null;
  imageInputTokens: string | null;
  imageOutputTokens: string | null;
  
  // Costs (USD)
  cost: number | null;
  inputCost: number | null;
  outputCost: number | null;
  requestCost: number | null;
  imageInputCost: number | null;
  imageOutputCost: number | null;
  estimatedCost: boolean | null;
  
  // Request Parameters
  temperature: number | null;
  maxTokens: number | null;
  topP: number | null;
  frequencyPenalty: number | null;
  presencePenalty: number | null;
  reasoningEffort: string | null;
  reasoningMaxTokens: number | null;
  responseFormat: any | null;
  tools: any | null;
  toolChoice: any | null;
  toolResults: any | null;
  
  // Status and Errors
  finishReason: string | null;
  unifiedFinishReason: string | null;
  hasError: boolean | null;
  errorDetails: any | null;
  canceled: boolean | null;
  
  // Flags
  streamed: boolean | null;
  cached: boolean | null;
  
  // Metadata
  customHeaders: any | null;
  mode: "api-keys" | "credits" | "hybrid";
  usedMode: "api-keys" | "credits";
  source: string | null;
  routingMetadata: any | null;
}

Querying Logs

Retrieve logs via the REST API:
curl https://api.llmgateway.io/logs \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -G \
  --data-urlencode "projectId=proj_abc123" \
  --data-urlencode "startDate=2024-01-01" \
  --data-urlencode "limit=50"

Query Parameters

Filter logs using these parameters:
apps/api/src/routes/logs.ts
const querySchema = z.object({
  // Filtering
  apiKeyId: z.string().optional(),
  providerKeyId: z.string().optional(),
  projectId: z.string().optional(),
  orgId: z.string().optional(),
  
  // Date Range
  startDate: z.string().optional(),  // ISO string
  endDate: z.string().optional(),
  
  // Model/Provider
  model: z.string().optional(),  // e.g., "gpt-4o"
  provider: z.string().optional(),  // e.g., "openai"
  
  // Status
  finishReason: z.string().optional(),
  unifiedFinishReason: z.string().optional(),
  
  // Source Tracking
  source: z.string().optional(),
  
  // Custom Headers
  customHeaderKey: z.string().optional(),
  customHeaderValue: z.string().optional(),
  
  // Pagination
  cursor: z.string().optional(),
  limit: z.number().int().min(1).max(100).optional(),  // default: 50
  orderBy: z.enum(["createdAt_asc", "createdAt_desc"]).optional()  // default: desc
});

Example Filters

By Model

?model=gpt-4o

By Provider

?provider=anthropic

Date Range

?startDate=2024-01-01&endDate=2024-01-31

Custom Header

?customHeaderKey=user-id&customHeaderValue=123

Pagination

Logs use cursor-based pagination:
{
  "logs": [...],
  "pagination": {
    "nextCursor": "log_xyz789",
    "hasMore": true,
    "limit": 50
  }
}
Fetch the next page:
curl "https://api.llmgateway.io/logs?cursor=log_xyz789&limit=50"

Cost Tracking

LLM Gateway calculates costs for all requests:
apps/gateway/src/lib/costs.ts
export function calculateCosts(params: {
  promptTokens: number;
  completionTokens: number;
  reasoningTokens?: number;
  cachedTokens?: number;
  imageInputCount?: number;
  imageOutputCount?: number;
  model: ModelDefinition;
  provider: ProviderModelMapping;
}) {
  const inputCost = (params.promptTokens / 1_000_000) * (provider.inputPrice ?? 0);
  const outputCost = (params.completionTokens / 1_000_000) * (provider.outputPrice ?? 0);
  const cachedInputCost = (params.cachedTokens ?? 0) / 1_000_000 * (provider.cachedInputPrice ?? 0);
  const requestCost = provider.requestPrice ?? 0;
  
  return {
    inputCost,
    outputCost,
    cachedInputCost,
    requestCost,
    totalCost: inputCost + outputCost + cachedInputCost + requestCost
  };
}

Cost Breakdown

{
  "usage": {
    "prompt_tokens": 150,
    "completion_tokens": 50,
    "total_tokens": 200,
    "reasoning_tokens": 1200,
    "cost_usd_total": 0.00045,
    "cost_usd_input": 0.00015,
    "cost_usd_output": 0.0003,
    "cost_usd_cached_input": 0.0,
    "cost_usd_request": 0.0
  }
}
Costs are calculated using real-time pricing from each provider’s documentation.

Source Tracking

Track where requests originate:
curl https://api.llmgateway.io/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "x-source: my-app/v1.0" \
  -d '{...}'
Query by source:
curl "https://api.llmgateway.io/logs?source=my-app/v1.0"

Custom Headers

Add custom metadata to requests:
curl https://api.llmgateway.io/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "x-llmgateway-user-id: user_123" \
  -H "x-llmgateway-session-id: sess_456" \
  -d '{...}'
Query by custom headers:
curl "https://api.llmgateway.io/logs?customHeaderKey=user-id&customHeaderValue=user_123"

Unique Models & Providers

Get a list of all models and providers you’ve used:
curl "https://api.llmgateway.io/logs/unique-models?projectId=proj_abc123"
Response:
{
  "models": [
    "gpt-4o",
    "gpt-4o-mini",
    "claude-3-5-sonnet-20241022",
    "gemini-2.0-flash-001"
  ],
  "providers": [
    "openai",
    "anthropic",
    "google-ai-studio"
  ]
}

Data Retention

Log retention depends on your plan:
  • Free Plan: 3 days
  • Pro Plan: 90 days
  • Enterprise Plan: Custom retention
Data storage costs $0.01 per 1M tokens. This is automatically deducted from your credits when retention is enabled.

Privacy and Security

Control what data is stored:
// Set retention level in organization settings
organization.retentionLevel = "none" | "metadata" | "retain";
  • none - Don’t store request/response data
  • metadata - Store metadata only (no message content)
  • retain - Store full request/response data

Fetch Individual Logs

Get a specific log by ID:
curl "https://api.llmgateway.io/logs/log_abc123" \
  -H "Authorization: Bearer YOUR_API_KEY"
Response:
{
  "log": {
    "id": "log_abc123",
    "requestId": "req_xyz789",
    "usedModel": "openai/gpt-4o",
    "usedProvider": "openai",
    "totalTokens": "200",
    "cost": 0.00045,
    "duration": 1234,
    "messages": [...],
    "content": "The capital of France is Paris.",
    "routingMetadata": {...}
  }
}

Dashboard Analytics

View analytics in the web dashboard:
  • Usage over time - Token and request trends
  • Cost breakdown - By model, provider, and project
  • Latency metrics - P50, P95, P99 latencies
  • Error rates - Track failed requests
  • Top models - Most used models
  • Top users - By API key
Analytics data is updated in real-time. Refresh the dashboard to see the latest metrics.

Export Analytics

Export logs to CSV or JSON:
# Fetch all logs and save to file
curl "https://api.llmgateway.io/logs?projectId=proj_abc123&limit=100" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  > logs.json

Build docs developers (and LLMs) love