Middleware System

Middleware provides a composable way to intercept and modify agent execution at key points. Use middleware for logging, authentication, rate limiting, error handling, and more.

Overview

Middleware follows a Koa-style async pipeline pattern. Each middleware receives a context and a next() function, allowing it to run code before and after the next middleware in the chain.

import { createAgent, Middleware } from '@agentlib/core'

const loggingMiddleware: Middleware = {
  name: 'logger',
  async run(mCtx, next) {
    console.log('Before execution')
    await next()  // Continue to next middleware
    console.log('After execution')
  },
}

const agent = createAgent({ name: 'agent' })
  .provider(model)
  .use(loggingMiddleware)

Middleware Scopes

Middleware can target specific execution points:

type MiddlewareScope =
  | 'run:before'    // Before agent run starts
  | 'run:after'     // After agent run completes
  | 'step:before'   // Before each reasoning step
  | 'step:after'    // After each reasoning step
  | 'tool:before'   // Before each tool call
  | 'tool:after'    // After each tool call

Scoped Middleware Example

const toolLogger: Middleware = {
  name: 'tool-logger',
  scope: ['tool:before', 'tool:after'],  // Only run for tool events
  async run(mCtx, next) {
    if (mCtx.scope === 'tool:before') {
      console.log(`[TOOL] Calling ${mCtx.tool?.name}`)
    }
    
    await next()
    
    if (mCtx.scope === 'tool:after') {
      console.log(`[TOOL] ${mCtx.tool?.name} completed`)
    }
  },
}

const agent = createAgent({ name: 'agent' })
  .provider(model)
  .tool(searchTool)
  .use(toolLogger)

If scope is not specified, middleware runs for all scopes.

Middleware Context

The middleware context provides access to execution state:

interface MiddlewareContext<TData = unknown> {
  scope: MiddlewareScope
  ctx: ExecutionContext<TData>  // Full execution context
  tool?: {                       // Present when scope is tool:*
    name: string
    args: Record<string, unknown>
    result?: unknown
  }
}

Accessing Agent Data

interface AppData {
  userId: string
  plan: 'free' | 'pro'
}

const planGuard: Middleware<AppData> = {
  name: 'plan-guard',
  scope: 'run:before',
  async run(mCtx, next) {
    const { data, input } = mCtx.ctx
    
    if (data.plan === 'free' && input.length > 500) {
      throw new Error('Input too long for free plan')
    }
    
    await next()
  },
}

const agent = createAgent<AppData>({
  name: 'agent',
  data: { userId: 'default', plan: 'free' },
})
  .provider(model)
  .use(planGuard)

Common Middleware Patterns

Logging Middleware

const requestLogger: Middleware = {
  name: 'request-logger',
  scope: 'run:before',
  async run(mCtx, next) {
    console.log(`[${new Date().toISOString()}] Input: ${mCtx.ctx.input}`)
    await next()
  },
}

const responseLogger: Middleware = {
  name: 'response-logger',
  scope: 'run:after',
  async run(mCtx, next) {
    await next()
    console.log(`[${new Date().toISOString()}] Completed in ${mCtx.ctx.state.finishedAt - mCtx.ctx.state.startedAt}ms`)
  },
}

Timing Middleware

const timingMiddleware: Middleware = {
  name: 'timing',
  async run(mCtx, next) {
    const start = Date.now()
    await next()
    const duration = Date.now() - start
    console.log(`[${mCtx.scope}] took ${duration}ms`)
  },
}

Rate Limiting

const rateLimits = new Map<string, number>()

const rateLimiter: Middleware<{ userId: string }> = {
  name: 'rate-limiter',
  scope: 'run:before',
  async run(mCtx, next) {
    const userId = mCtx.ctx.data.userId
    const lastCall = rateLimits.get(userId) || 0
    const now = Date.now()
    
    if (now - lastCall < 1000) {  // 1 request per second
      throw new Error('Rate limit exceeded')
    }
    
    rateLimits.set(userId, now)
    await next()
  },
}

const agent = createAgent<{ userId: string }>({
  name: 'rate-limited-agent',
  data: { userId: 'user-123' },
})
  .provider(model)
  .use(rateLimiter)

Authentication

interface AuthData {
  apiKey: string
  authenticated: boolean
}

const authMiddleware: Middleware<AuthData> = {
  name: 'auth',
  scope: 'run:before',
  async run(mCtx, next) {
    const { apiKey } = mCtx.ctx.data
    
    // Validate API key
    const isValid = await validateApiKey(apiKey)
    
    if (!isValid) {
      throw new Error('Invalid API key')
    }
    
    // Mark as authenticated
    mCtx.ctx.data.authenticated = true
    
    await next()
  },
}

const agent = createAgent<AuthData>({
  name: 'secure-agent',
  data: { apiKey: '', authenticated: false },
})
  .provider(model)
  .use(authMiddleware)

Error Handling

const errorHandler: Middleware = {
  name: 'error-handler',
  async run(mCtx, next) {
    try {
      await next()
    } catch (error) {
      console.error(`[ERROR] ${mCtx.scope}:`, error)
      
      // Log to external service
      await logToSentry(error, {
        scope: mCtx.scope,
        sessionId: mCtx.ctx.sessionId,
        input: mCtx.ctx.input,
      })
      
      // Re-throw or handle gracefully
      throw error
    }
  },
}

Tool Validation

const toolValidator: Middleware = {
  name: 'tool-validator',
  scope: 'tool:before',
  async run(mCtx, next) {
    const { name, args } = mCtx.tool!
    
    // Validate tool arguments
    if (name === 'delete_file') {
      const path = args.path as string
      if (path.startsWith('/system')) {
        throw new Error('Cannot delete system files')
      }
    }
    
    await next()
  },
}

Token Budget Tracking

interface TokenData {
  userId: string
  tokenBudget: number
  tokensUsed: number
}

const tokenTracker: Middleware<TokenData> = {
  name: 'token-tracker',
  scope: 'run:after',
  async run(mCtx, next) {
    await next()
    
    const usage = mCtx.ctx.state.usage
    const totalTokens = usage?.totalTokens || 0
    
    mCtx.ctx.data.tokensUsed += totalTokens
    
    if (mCtx.ctx.data.tokensUsed > mCtx.ctx.data.tokenBudget) {
      console.warn('Token budget exceeded!')
    }
    
    console.log(`Tokens used: ${mCtx.ctx.data.tokensUsed} / ${mCtx.ctx.data.tokenBudget}`)
  },
}

const agent = createAgent<TokenData>({
  name: 'token-tracked-agent',
  data: { userId: 'user-123', tokenBudget: 10_000, tokensUsed: 0 },
})
  .provider(model)
  .use(tokenTracker)

Modifying Tool Results

Middleware can intercept and modify tool results:

const toolResultFilter: Middleware = {
  name: 'tool-result-filter',
  scope: 'tool:after',
  async run(mCtx, next) {
    await next()
    
    // Filter sensitive data from tool results
    if (mCtx.tool?.name === 'database_query') {
      const result = mCtx.tool.result as any
      if (result.rows) {
        result.rows = result.rows.map((row: any) => ({
          ...row,
          password: '[REDACTED]',
          apiKey: '[REDACTED]',
        }))
      }
    }
  },
}

Modifying mCtx.tool.result affects what the agent sees. Be careful not to corrupt the data structure.

Using Built-in Middleware

AgentLIB provides pre-built middleware:

Logger Middleware

import { createLogger } from '@agentlib/logger'

const agent = createAgent({ name: 'agent' })
  .provider(model)
  .use(createLogger({
    level: 'debug',      // 'debug' | 'info' | 'warn' | 'error'
    timing: true,        // Log execution times
    prefix: '[agent]',   // Log prefix
  }))

Middleware Execution Order

Middleware runs in registration order:

const middleware1: Middleware = {
  name: 'first',
  async run(mCtx, next) {
    console.log('1: before')
    await next()
    console.log('1: after')
  },
}

const middleware2: Middleware = {
  name: 'second',
  async run(mCtx, next) {
    console.log('2: before')
    await next()
    console.log('2: after')
  },
}

const agent = createAgent({ name: 'agent' })
  .provider(model)
  .use(middleware1)  // Runs first
  .use(middleware2)  // Runs second

// Output:
// 1: before
// 2: before
// 2: after
// 1: after

Place error handlers and loggers early in the chain so they wrap all subsequent middleware.

Complete Example

import 'dotenv/config'
import { createAgent, defineTool, Middleware } from '@agentlib/core'
import { openai } from '@agentlib/openai'
import { BufferMemory } from '@agentlib/memory'
import { createLogger } from '@agentlib/logger'

interface AppData {
  userId: string
  plan: 'free' | 'pro'
  requestCount: number
}

const getWeatherTool = defineTool({
  schema: {
    name: 'get_weather',
    description: 'Get the current weather',
    parameters: {
      type: 'object',
      properties: {
        location: { type: 'string' },
      },
      required: ['location'],
    },
  },
  async execute({ location }) {
    return { location, temperature: 22, condition: 'sunny' }
  },
})

// Custom middleware
const planGuard: Middleware<AppData> = {
  name: 'plan-guard',
  scope: 'run:before',
  async run(mCtx, next) {
    if (mCtx.ctx.data.plan === 'free' && mCtx.ctx.input.length > 500) {
      throw new Error('Input too long for free plan')
    }
    await next()
  },
}

const requestCounter: Middleware<AppData> = {
  name: 'request-counter',
  scope: 'run:before',
  async run(mCtx, next) {
    mCtx.ctx.data.requestCount += 1
    console.log(`Request #${mCtx.ctx.data.requestCount} from ${mCtx.ctx.data.userId}`)
    await next()
  },
}

const timingMiddleware: Middleware = {
  name: 'timing',
  scope: ['run:before', 'run:after'],
  async run(mCtx, next) {
    if (mCtx.scope === 'run:before') {
      console.log(`[START] ${new Date().toISOString()}`)
    }
    
    await next()
    
    if (mCtx.scope === 'run:after') {
      console.log(`[END] ${new Date().toISOString()}`)
    }
  },
}

const agent = createAgent<AppData>({
  name: 'assistant',
  systemPrompt: 'You are a helpful assistant.',
  data: { userId: 'default', plan: 'free', requestCount: 0 },
  policy: { maxSteps: 10, tokenBudget: 10_000 },
})
  .provider(openai({
    apiKey: process.env.OPENAI_API_KEY,
    model: 'gpt-4o',
  }))
  .memory(new BufferMemory({ maxMessages: 20 }))
  .tool(getWeatherTool)
  .use(createLogger({ level: 'debug', timing: true, prefix: '[weather-agent]' }))
  .use(timingMiddleware)
  .use(requestCounter)
  .use(planGuard)

const result = await agent.run({
  input: 'What is the weather in Buenos Aires and Tokyo?',
  data: { userId: 'user-123', plan: 'pro', requestCount: 0 },
})

console.log('\nFinal response:')
console.log(result.output)
console.log('\nToken usage:', result.state.usage)

Next Steps

Event Handling - Use events for observability
Creating Agents - Integrate middleware into agents
Using Tools - Intercept tool calls with middleware

Get Started

Core Concepts

Guides

Packages

Middleware System

Overview

Middleware Scopes

Scoped Middleware Example

Middleware Context

Accessing Agent Data

Common Middleware Patterns

Logging Middleware

Timing Middleware

Rate Limiting

Authentication

Error Handling

Tool Validation

Token Budget Tracking

Modifying Tool Results

Using Built-in Middleware

Logger Middleware

Middleware Execution Order

Complete Example

Next Steps

Build docs developers (and LLMs) love

Get Started

Core Concepts

Guides

Packages

​Overview

​Middleware Scopes

​Scoped Middleware Example

​Middleware Context

​Accessing Agent Data

​Common Middleware Patterns

​Logging Middleware

​Timing Middleware

​Rate Limiting

​Authentication

​Error Handling

​Tool Validation

​Token Budget Tracking

​Modifying Tool Results

​Using Built-in Middleware

​Logger Middleware

​Middleware Execution Order

​Complete Example

​Next Steps

Build docs developers (and LLMs) love

Overview

Middleware Scopes

Scoped Middleware Example

Middleware Context

Accessing Agent Data

Common Middleware Patterns

Logging Middleware

Timing Middleware

Rate Limiting

Authentication

Error Handling

Tool Validation

Token Budget Tracking

Modifying Tool Results

Using Built-in Middleware

Logger Middleware

Middleware Execution Order

Complete Example

Next Steps