Skip to main content

Build AI agents with composable harnesses

LLM Gateway is an agent framework built on three simple ideas: a harness yields events, harnesses compose, and events form a graph.

Get StartedLearn Core Concepts

Key Features

Everything you need to build production-ready AI agents

Composable Harnesses

Wrap harnesses around harnesses to layer behavior without changing the interface

Multi-Agent Orchestration

Run multiple agents concurrently with built-in multiplexing and event coordination

Event-Driven Architecture

Stream fine-grained events that form an immutable conversation graph

Multiple LLM Providers

Built-in support for Zen, Anthropic, OpenAI, and OpenRouter with unified interface

Human-in-the-Loop

Permission system with glob-pattern matching and relay-based approval flow

Recursive Processing

RLM harness gives models a REPL to explore and process arbitrarily long inputs

Quick Start

Get up and running in minutes

1

Install dependencies

LLM Gateway uses Bun as its runtime and package manager.
bun install
Copy the environment example file and configure your API keys:
cp .env.example .env
2

Stream your first LLM call

A provider harness is all you need to stream an LLM response.
import { createGeneratorHarness } from "./packages/ai/harness/providers/zen";

const harness = createGeneratorHarness();

for await (const event of harness.invoke({
  model: "glm-4.7",
  messages: [{ role: "user", content: "What is 2+2?" }],
})) {
  if (event.type === "text") process.stdout.write(event.content);
}
3

Add tool calling

Wrap a provider with the agent harness to enable agentic tool-calling loops.
import { createAgentHarness } from "./packages/ai/harness/agent";
import { bashTool } from "./packages/ai/tools";

const agent = createAgentHarness({ harness: createGeneratorHarness() });

for await (const event of agent.invoke({
  model: "glm-4.7",
  messages: [{ role: "user", content: "List the files in this directory" }],
  tools: [bashTool],
  permissions: { allowlist: [{ tool: "bash" }] }
})) {
  if (event.type === "tool_call") console.log(`[calling ${event.name}]`);
  if (event.type === "text") process.stdout.write(event.content);
}
4

Explore the architecture

Dive deeper into the core concepts:

Explore by Use Case

Pick the components you need for your use case

→ Stream an LLM call

Use a provider harness to stream responses from any LLM with a unified interface.

Read guide

→ Single agent with tools

Wrap a provider with the agent harness to get an agentic loop with built-in tool execution.

Read guide

→ Multi-agent orchestration

Use the orchestrator to manage concurrent agents and coordinate permission relays.

Read guide

→ Client-side rendering

Use graph projections to render threaded conversations with nested subagent branches.

Read guide

Ready to build?

Start with the quickstart guide or dive into the API reference to explore all available harnesses, tools, and utilities.

Quickstart GuideAPI Reference