Plugin System

The ADK plugin system allows you to extend agent behavior by hooking into key lifecycle events. Plugins can intercept and modify requests, responses, tool calls, and more.

Overview

Plugins implement the BasePlugin class and provide callback methods for various lifecycle events:

User messages - Intercept and modify incoming user messages
Agent execution - Before/after agent runs
Model calls - Before/after LLM requests and on errors
Tool execution - Before/after tool calls and on errors
Events - Process agent events as they occur

BasePlugin

All plugins extend the BasePlugin class:

import { BasePlugin } from "@iqai/adk";

class MyPlugin extends BasePlugin {
  constructor() {
    super("my-plugin"); // Unique plugin name
  }

  async beforeModelCallback(params) {
    // Intercept LLM requests
    console.log("About to call model:", params.llmRequest.model);
    return undefined; // Return undefined to continue normally
  }

  async afterModelCallback(params) {
    // Process LLM responses
    console.log("Model responded:", params.llmResponse.text);
    return undefined;
  }
}

Available Callbacks

All callbacks are optional. Return undefined to continue normal execution, or return a value to short-circuit:

onUserMessageCallback

async function

Called when a user message is received. Can modify or replace the message.Parameters:

invocationContext - Current invocation context
userMessage - The user’s Content message

Returns: Modified Content or undefined

beforeRunCallback

async function

Called before agent execution starts.Parameters:

invocationContext - Current invocation context

Returns: Event or undefined

afterRunCallback

async function

Called after agent execution completes.Parameters:

invocationContext - Current invocation context
result - The execution result

onEventCallback

async function

Called for each event during execution.Parameters:

invocationContext - Current invocation context
event - The event being processed

Returns: Modified Event or undefined

beforeAgentCallback

async function

Called before an agent (including sub-agents) starts.Parameters:

agent - The agent about to execute
callbackContext - Callback context with invocation details

Returns: Modified Content or undefined

afterAgentCallback

async function

Called after an agent completes.Parameters:

agent - The agent that executed
callbackContext - Callback context
result - Agent result

Returns: Modified Content or undefined

beforeModelCallback

async function

Called before each LLM request. Can short-circuit with a cached response.Parameters:

callbackContext - Callback context
llmRequest - The LLM request about to be sent

Returns: LlmResponse to skip the call, or undefined

afterModelCallback

async function

Called after each LLM response.Parameters:

callbackContext - Callback context
llmResponse - The LLM response received
llmRequest - The original request

Returns: Modified LlmResponse or undefined

onModelErrorCallback

async function

Called when an LLM request fails. Can provide a fallback response.Parameters:

callbackContext - Callback context
llmRequest - The failed request
error - The error that occurred

Returns: Fallback LlmResponse or undefined to propagate error

beforeToolCallback

async function

Called before tool execution. Can modify tool arguments.Parameters:

tool - The tool about to execute
toolArgs - Tool arguments
toolContext - Tool execution context

Returns: Modified arguments or undefined

afterToolCallback

async function

Called after tool execution. Can modify tool results.Parameters:

tool - The tool that executed
toolArgs - Tool arguments used
toolContext - Tool execution context
result - Tool execution result

Returns: Modified result or undefined

onToolErrorCallback

async function

Called when a tool fails. Can provide fallback result.Parameters:

tool - The tool that failed
toolArgs - Tool arguments used
toolContext - Tool execution context
error - The error that occurred

Returns: Fallback result or undefined to propagate error

async function

Called when the plugin manager shuts down. Clean up resources here.

PluginManager

The PluginManager manages plugin registration and execution:

import { PluginManager } from "@iqai/adk";

const pluginManager = new PluginManager({
  plugins: [new MyPlugin()],
  closeTimeout: 5000, // Timeout for plugin cleanup
});

// Register additional plugins
pluginManager.registerPlugin(new AnotherPlugin());

// Get a specific plugin
const plugin = pluginManager.getPlugin("my-plugin");

// Get all plugins
const allPlugins = pluginManager.getPlugins();

// Clean up
await pluginManager.close();

Plugins are executed in registration order. If a plugin returns a non-undefined value from a callback, execution stops and that value is used.

Built-in Plugins

LangfusePlugin

Integrates with Langfuse for observability and tracing:

import { LangfusePlugin } from "@iqai/adk";

const langfusePlugin = new LangfusePlugin({
  publicKey: process.env.LANGFUSE_PUBLIC_KEY!,
  secretKey: process.env.LANGFUSE_SECRET_KEY!,
  baseUrl: "https://us.cloud.langfuse.com", // Optional
  release: "v1.0.0", // Optional
  flushAt: 1, // Flush after N events
  flushInterval: 1000, // Flush interval in ms
});

const agent = new AgentBuilder()
  .withName("traced-agent")
  .withModel("gpt-4")
  .withPluginManager(new PluginManager({ plugins: [langfusePlugin] }))
  .buildLlm();

Features:

Traces entire agent execution hierarchies
Tracks token usage and costs
Records tool calls and model generations
Supports multi-agent tracing
Automatic span creation for sub-agents

LangfusePlugin requires the langfuse package: npm install langfuse

ModelFallbackPlugin

Automatically retries failed LLM requests with fallback models:

import { ModelFallbackPlugin } from "@iqai/adk";

const fallbackPlugin = new ModelFallbackPlugin(
  ["gpt-4", "gpt-3.5-turbo", "claude-3-sonnet"], // Fallback chain
  3, // Max retries per model
  1000 // Retry delay in ms
);

const agent = new AgentBuilder()
  .withModel("gpt-4-turbo") // Primary model
  .withPluginManager(new PluginManager({ plugins: [fallbackPlugin] }))
  .buildLlm();

Behavior:

Only handles rate limit errors (429)
Retries same model up to maxRetries times
Falls back to next model in the chain
Preserves all request parameters during fallback

ReflectAndRetryToolPlugin

Automatically retries failed tool calls with reflection guidance:

import { 
  ReflectAndRetryToolPlugin, 
  TrackingScope 
} from "@iqai/adk";

const retryPlugin = new ReflectAndRetryToolPlugin({
  name: "reflect-retry",
  maxRetries: 3,
  throwExceptionIfRetryExceeded: true,
  trackingScope: TrackingScope.INVOCATION, // or TrackingScope.GLOBAL
});

const agent = new AgentBuilder()
  .withTools([myTool])
  .withPluginManager(new PluginManager({ plugins: [retryPlugin] }))
  .buildLlm();

Features:

Catches tool execution errors
Provides structured reflection guidance to the agent
Tracks retry counts per tool
Suggests alternative approaches after failures
Configurable tracking scope (per-invocation or global)

Tracking Scopes:

INVOCATION - Count retries per agent invocation
GLOBAL - Count retries globally across all invocations

ToolOutputFilterPlugin

Automatically filters large tool outputs using LLM-generated JQ filters:

import { ToolOutputFilterPlugin } from "@iqai/adk";
import { LLMRegistry } from "@iqai/adk";

const filterPlugin = new ToolOutputFilterPlugin({
  filterModel: LLMRegistry.newLLM("claude-3-haiku"), // Fast model for filtering
  enabledTools: ["search", "database_query"], // Only filter these tools
  config: {
    sizeThreshold: 8000, // Trigger at 8K chars
    targetSize: 4000, // Target 4K chars
    maxIterations: 3, // Max filtering rounds
    debug: true,
  },
});

const agent = new AgentBuilder()
  .withTools([searchTool, dbTool])
  .withPluginManager(new PluginManager({ plugins: [filterPlugin] }))
  .buildLlm();

Features:

Automatically detects oversized tool outputs
Uses LLM to generate smart JQ filters
Iteratively reduces output size
Preserves relevant data for the query
Adds metadata about filtering

Configuration:

sizeThreshold

number

default:8000

Trigger filtering when output exceeds this many characters

keyThreshold

number

default:50

Trigger filtering when output has more than this many keys

targetSize

number

default:4000

Target size after filtering

maxIterations

number

default:3

Maximum number of filtering iterations

maxSchemaDepth

number

default:4

Maximum depth for schema extraction

ToolOutputFilterPlugin requires the jq command-line tool to be installed on your system.

Creating Custom Plugins

Extend BasePlugin

Create a class that extends BasePlugin:

import { BasePlugin } from "@iqai/adk";

export class LoggingPlugin extends BasePlugin {
  constructor() {
    super("logging-plugin");
  }
}

Implement callbacks

Override the callbacks you need:

async beforeModelCallback(params) {
  console.log(`[${params.callbackContext.agentName}] Calling model:`, {
    model: params.llmRequest.model,
    messageCount: params.llmRequest.contents?.length,
  });
  return undefined;
}

async afterModelCallback(params) {
  console.log(`[${params.callbackContext.agentName}] Model response:`, {
    text: params.llmResponse.text?.slice(0, 100),
    tokens: params.llmResponse.usageMetadata?.totalTokenCount,
  });
  return undefined;
}

Handle cleanup

Implement resource cleanup:

async close() {
  console.log("Logging plugin shutting down");
  // Clean up resources (close files, connections, etc.)
}

const agent = new AgentBuilder()
  .withPluginManager(new PluginManager({
    plugins: [new LoggingPlugin()],
  }))
  .buildLlm();

Advanced Examples

Caching Plugin

Cache LLM responses to save costs:

class CachingPlugin extends BasePlugin {
  private cache = new Map<string, LlmResponse>();

  constructor() {
    super("caching-plugin");
  }

  private getCacheKey(request: LlmRequest): string {
    return JSON.stringify({
      model: request.model,
      contents: request.contents,
      tools: Object.keys(request.toolsDict),
    });
  }

  async beforeModelCallback(params) {
    const key = this.getCacheKey(params.llmRequest);
    const cached = this.cache.get(key);
    
    if (cached) {
      console.log("Cache hit!");
      return cached; // Skip LLM call
    }
    
    return undefined;
  }

  async afterModelCallback(params) {
    const key = this.getCacheKey(params.llmRequest!);
    this.cache.set(key, params.llmResponse);
    return undefined;
  }
}

Metrics Plugin

Track execution metrics:

class MetricsPlugin extends BasePlugin {
  private metrics = {
    invocations: 0,
    modelCalls: 0,
    toolCalls: 0,
    totalTokens: 0,
    errors: 0,
  };

  constructor() {
    super("metrics-plugin");
  }

  async beforeRunCallback() {
    this.metrics.invocations++;
    return undefined;
  }

  async beforeModelCallback() {
    this.metrics.modelCalls++;
    return undefined;
  }

  async afterModelCallback(params) {
    const tokens = params.llmResponse.usageMetadata?.totalTokenCount || 0;
    this.metrics.totalTokens += tokens;
    return undefined;
  }

  async beforeToolCallback() {
    this.metrics.toolCalls++;
    return undefined;
  }

  async onModelErrorCallback() {
    this.metrics.errors++;
    return undefined;
  }

  async onToolErrorCallback() {
    this.metrics.errors++;
    return undefined;
  }

  getMetrics() {
    return { ...this.metrics };
  }
}

Best Practices

Keep plugins focused - Each plugin should have a single responsibility (logging, caching, metrics, etc.)

Handle errors gracefully - Plugin errors can break agent execution. Always use try-catch in callbacks.

Return undefined by default - Only return values when you want to short-circuit execution.

Clean up resources - Implement the close() method to properly clean up connections, files, etc.

Consider performance - Plugins run on every callback. Keep them fast or risk slowing down agents.

Getting Started

Core Concepts

Agents

Models & Providers

Tools

Memory & State

Advanced Features

CLI Tool

Examples

Overview

BasePlugin

Available Callbacks

PluginManager

Built-in Plugins

LangfusePlugin

ModelFallbackPlugin

ReflectAndRetryToolPlugin

ToolOutputFilterPlugin

Creating Custom Plugins

Advanced Examples

Caching Plugin

Metrics Plugin

Best Practices

See Also

Build docs developers (and LLMs) love

Getting Started

Core Concepts

Agents

Models & Providers

Tools

Memory & State

Advanced Features

CLI Tool

Examples

​Overview

​BasePlugin

​Available Callbacks

​PluginManager

​Built-in Plugins

​LangfusePlugin

​ModelFallbackPlugin

​ReflectAndRetryToolPlugin

​ToolOutputFilterPlugin

​Creating Custom Plugins

​Advanced Examples

​Caching Plugin

​Metrics Plugin

​Best Practices

​See Also

Build docs developers (and LLMs) love

Overview

BasePlugin

Available Callbacks

PluginManager

Built-in Plugins

LangfusePlugin

ModelFallbackPlugin

ReflectAndRetryToolPlugin

ToolOutputFilterPlugin

Creating Custom Plugins

Advanced Examples

Caching Plugin

Metrics Plugin

Best Practices

See Also