Skip to main content

​
What is a harness?

A harness is an async generator function that yields events during LLM invocations. It’s the core abstraction in LLM Gateway — everything from a single API call to a multi-agent orchestration is built by composing harnesses. 
interface GeneratorHarnessModule {
  invoke(params: GeneratorInvokeParams): AsyncIterable<HarnessEvent>;
  supportedModels(): Promise<string[]>;
}
Harnesses implement a simple contract:
  • invoke() takes messages, tools, and configuration, returns an async iterable of events
  • supportedModels() returns the list of model IDs the harness can handle

​
Two types of harnesses

​
Provider harnesses

Provider harnesses make single LLM API calls and stream the results. They handle one request-response cycle and yield events like text, reasoning, tool_call, and usage. 
import { createGeneratorHarness } from "./packages/ai/harness/providers/zen";

const harness = createGeneratorHarness();

for await (const event of harness.invoke({
  model: "glm-4.7",
  messages: [{ role: "user", content: "What is 2+2?" }],
})) {
  if (event.type === "text") {
    process.stdout.write(event.content);
  }
}
Available provider harnesses:

Zen

OpenAI-compatible API with support for reasoning content. Default provider.

Anthropic

Claude models via the Anthropic Messages API.

OpenAI

GPT models via OpenAI Chat Completions API.

OpenRouter

Access 100+ models through OpenRouter aggregator.

​
Agent harness

The agent harness wraps a provider harness to add agentic behavior: tool execution, permission checking, and an iterative loop that continues until the model has no more tool calls.
Agent harness wrapping Zen provider
import { createAgentHarness } from "./packages/ai/harness/agent";
import { createGeneratorHarness } from "./packages/ai/harness/providers/zen";
import { bashTool } from "./packages/ai/tools";

const agent = createAgentHarness({ 
  harness: createGeneratorHarness(),
  maxIterations: 10 
});

for await (const event of agent.invoke({
  model: "glm-4.7",
  messages: [{ role: "user", content: "List files in this directory" }],
  tools: [bashTool],
  permissions: { allowlist: [{ tool: "bash" }] }
})) {
  if (event.type === "tool_call") {
    console.log(`[calling ${event.name}]`);
  }
  if (event.type === "tool_result") {
    console.log(`[result]`, event.output);
  }
  if (event.type === "text") {
    process.stdout.write(event.content);
  }
}
The agent harness adds:
  • Agentic loop: Continues calling the LLM until no tool calls remain or maxIterations is reached
  • Permission handling: Checks allowlists, yields relay events, waits for approval
  • Tool execution: Executes approved tools with proper context and error handling
  • Message history: Builds up the conversation with assistant responses and tool results

​
Event flow

A typical agentic conversation produces this event sequence:

​
Lifecycle events

Agent harnesses emit lifecycle events to mark the boundaries of their execution:
​
harness_start
event
Marks the beginning of an agent run. Contains runId and optional depth and maxIterations for nested invocations.
​
harness_end
event
Marks the end of an agent run. Contains runId, optional reason (“final” or “max_iterations”), and usage totals for RLM harness.
Lifecycle tracking
let isRunning = false;

for await (const event of agent.invoke(params)) {
  if (event.type === "harness_start") {
    isRunning = true;
    console.log(`Agent ${event.runId} started`);
  }
  if (event.type === "harness_end") {
    isRunning = false;
    console.log(`Agent ${event.runId} finished`);
  }
}

​
Harness parameters

The invoke() method accepts these parameters:
​
model
string
required
Model ID to use for LLM calls (e.g., “glm-4.7”, “claude-3.5-sonnet”).
​
messages
Message[]
required
Conversation history. Each message has role (“system”, “user”, “assistant”, “tool”) and content.
​
tools
ToolDefinition[]
Tools available to the agent. Each tool has a name, description, Zod schema, and optional execute() function.
​
permissions
Permissions
Permission rules. Contains allowlist, allowOnce, and deny arrays for controlling tool access.
​
env
object
Environment context:
  • parentId: Links this run to a parent (for subagents)
  • spawn: Function to spawn subagents
  • fileTime: File timestamp tracking utility

​
Creating custom harnesses

You can create custom harnesses for specialized behavior: 
async function* retryHarness(
  baseHarness: GeneratorHarnessModule,
  maxRetries = 3
) {
  return {
    async *invoke(params: GeneratorInvokeParams) {
      let attempt = 0;
      while (attempt < maxRetries) {
        try {
          for await (const event of baseHarness.invoke(params)) {
            if (event.type === "error") {
              attempt++;
              if (attempt >= maxRetries) yield event;
              break;
            }
            yield event;
          }
          return; // Success
        } catch (error) {
          attempt++;
          if (attempt >= maxRetries) {
            yield { type: "error", runId: "", error };
          }
        }
      }
    },
    supportedModels: () => baseHarness.supportedModels()
  };
}
See Composition to learn how to layer harness behavior, and Custom Harnesses for detailed implementation patterns.

​
Key characteristics

Harnesses wrap other harnesses. The agent harness wraps a provider harness. You can add retries, logging, rate limiting, or caching by wrapping harnesses around each other.
Events arrive as they’re produced. Text streams token-by-token. Tool calls stream as the model emits them. No buffering unless you choose to collect events.
Events are a discriminated union type. Tools use Zod schemas for runtime validation. TypeScript provides autocomplete for event fields.
The same agent code works with any provider. Swap Zen for Anthropic or OpenAI by changing one line. Model-specific features (like reasoning) surface as events when available.

​
Next steps

Events

Learn about the event types that flow through harnesses

Composition

Understand how to layer harness behavior

Agent API

Full API reference for the agent harness

Custom Provider

Build a custom provider harness

Build docs developers (and LLMs) love

Get started for freeTalk to us