Overview
The LLM Gateway provides a comprehensive type system for building harnesses, handling messages, managing tools, and controlling event flow. All types follow OpenAI’s messaging patterns with dedicated tool roles.
Message Types
ContentPart
Content parts for multipart messages supporting text, images, and documents.
type TextContentPart = { type: "text"; text: string };
type ImageContentPart = { type: "image"; mediaType: string; data: string };
type DocumentContentPart = { type: "document"; mediaType: string; data: string };
type ContentPart = TextContentPart | ImageContentPart | DocumentContentPart;
The content type: "text", "image", or "document"
Text content (only for type: "text")
MIME type for images and documents (e.g., "image/png", "application/pdf")
Base64-encoded content for images and documents
Message
Message structure following OpenAI’s pattern with dedicated tool role.
type Message =
| { role: "system"; content: string }
| { role: "user"; content: string | ContentPart[] }
| { role: "assistant"; content: string | null; tool_calls?: ToolCall[] }
| { role: "tool"; tool_call_id: string; content: string | ContentPart[] };
System Message
System instructions or context
User Message
content
string | ContentPart[]
required
User message as plain text or multipart content
Assistant Message
Assistant role identifier
Assistant response text (null when only tool calls present)
Tool invocations requested by the assistant
ID of the tool call this message responds to
content
string | ContentPart[]
required
Tool execution result
interface ToolDefinition<TSchema extends z.ZodTypeAny = z.ZodTypeAny, TResult = unknown> {
name: string;
description: string;
schema: TSchema;
execute?: (input: z.infer<TSchema>, ctx: ToolContext) => Promise<ToolExecutionResult<TResult>>;
derivePermission?: (params: Record<string, unknown>) => ToolPermission;
}
Natural language description shown to the model
Zod schema defining the tool’s input parameters
Optional execution handler. Receives validated input and context, returns result.
Optional function to derive permission requirements from parameters
Example
import { z } from "zod";
const readFileTool: ToolDefinition = {
name: "read_file",
description: "Read contents of a file",
schema: z.object({
path: z.string(),
}),
execute: async (input, ctx) => {
const contents = await fs.readFile(input.path, "utf-8");
return { context: contents };
},
};
interface ToolCall<TSchema extends z.ZodTypeAny = z.ZodTypeAny> {
id: string;
name: string;
arguments?: z.infer<TSchema>;
}
Unique identifier for this tool call
Name of the tool to invoke
Parsed tool arguments matching the tool’s schema
interface ToolExecutionResult<T = unknown> {
context?: string; // Injected into agent context window
result?: T; // Available to harness/application
}
Text injected into the agent’s context window (shown to the model)
Structured result available to the harness or application
ToolContext
Context passed to tool execute functions.
interface ToolContext {
parentId?: string;
spawn?: (task: string) => Promise<string>;
fileTime?: FileTime;
}
The tool_call ID that spawned this execution context
Function to spawn subagent tasks
File timestamp tracking for deterministic execution
Permission Types
Defines permission rules for tool execution control.
interface ToolPermission {
tool: string;
params?: Record<string, string>; // param name → glob pattern
}
Tool name this permission applies to
Parameter constraints using glob patterns (e.g., {"path": "src/**"})
Permissions
Complete permission configuration for harness invocation.
interface Permissions {
allowlist?: ToolPermission[];
allowOnce?: ToolPermission[];
deny?: Array<{ toolCallId: string; reason?: string }>;
}
Tools always permitted without relay events
Tools permitted once, then require relay approval
deny
Array<{ toolCallId: string; reason?: string }>
Explicitly denied tool calls with optional reasons
Event Types
HarnessEvent
Discriminated union of all events a harness can yield.
type HarnessEvent =
| { type: "harness_start"; runId: string; parentId?: string; depth?: number; maxIterations?: number }
| { type: "harness_end"; runId: string; parentId?: string; reason?: "final" | "max_iterations"; iterations?: number; totalUsage?: { inputTokens: number; outputTokens: number } }
| { type: "reasoning"; runId: string; id: string; parentId?: string; content: string }
| { type: "text"; runId: string; id: string; parentId?: string; content: string }
| { type: "tool_call"; runId: string; id: string; parentId?: string; name: string; input: unknown }
| { type: "tool_result"; runId: string; id: string; parentId?: string; name: string; output: unknown }
| { type: "tool_progress"; runId: string; id: string; parentId?: string; toolCallId: string; name: string; content: unknown }
| { type: "repl_input"; runId: string; id: string; parentId?: string; code: string; iteration?: number }
| { type: "repl_progress"; runId: string; id: string; parentId?: string; chunk: string; stream: "stdout" | "stderr" }
| { type: "repl_output"; runId: string; id: string; parentId?: string; stdout: string; error?: string; done: boolean; iteration?: number; durationMs?: number; truncated?: boolean }
| { type: "usage"; runId: string; parentId?: string; inputTokens: number; outputTokens: number; cacheReadTokens?: number; cacheCreationTokens?: number }
| { type: "error"; runId: string; parentId?: string; error: Error }
| RelayEvent;
RelayEvent
Event requesting permission for tool execution.
type RelayEvent = {
type: "relay";
kind: "permission";
runId: string;
id: string;
parentId?: string;
toolCallId: string;
tool: string;
params: Record<string, unknown>;
respond: (response: PermissionResponse) => void;
};
Relay kind (currently only permission)
Tool call awaiting permission
params
Record<string, unknown>
required
Tool parameters
Callback to approve or deny the tool call
PermissionResponse
Response to a permission relay event.
type PermissionResponse = { approved: boolean; reason?: string };
Whether the tool call is approved
Optional explanation for approval or denial
Harness Interface
HarnessModule
Callback-based harness interface.
interface HarnessModule {
invoke(params: InvokeParams): Promise<void>;
supportedModels(): Promise<string[]>;
}
GeneratorHarnessModule
Generator-based harness interface (preferred).
interface GeneratorHarnessModule {
invoke(params: GeneratorInvokeParams): AsyncIterable<HarnessEvent>;
supportedModels(): Promise<string[]>;
}
InvokeParams
Parameters for callback-based harness invocation.
interface InvokeParams {
model?: string;
messages: Message[];
tools?: ToolDefinition[];
emit: (event: HarnessEvent) => void;
context?: { parentId?: string };
permissions?: Permissions;
}
Model identifier (e.g., "claude-3-5-sonnet-20241022")
Execution context with optional parent ID
GeneratorInvokeParams
Parameters for generator-based harness invocation.
interface GeneratorInvokeParams {
model?: string;
messages: Message[];
context?: string;
tools?: ToolDefinition[];
env?: {
parentId?: string;
spawn?: (task: string, parentId: string) => Promise<string>;
fileTime?: FileTime;
};
permissions?: Permissions;
}
Additional context string injected into the conversation
Execution environment with spawn capability and file time tracking
Harness
Harness metadata.
interface Harness {
id: string;
name: string;
description?: string;
modulePath?: string;
supportedModels?: string[];
registeredAt?: number;
}
Unique harness identifier
Human-readable harness name
File system path to the harness module
List of supported model identifiers
Unix timestamp of registration
LoadedHarness
A loaded harness with its module.
interface LoadedHarness {
harness: Harness;
module: HarnessModule;
}
The loaded harness module implementation
