Middleware

What is Middleware?

Middleware provides lifecycle hooks for cross-cutting concerns like logging, authentication, rate limiting, validation, and observability. Inspired by Koa.js, AgentLIB’s middleware is:

Composable — Chain multiple middleware in order
Async-first — Full support for async operations
Scoped — Run middleware at specific lifecycle points
Type-safe — Access to fully-typed execution context

Middleware Interface

Every middleware implements this contract:

// Source: packages/core/src/types/index.ts:290-294
export interface Middleware<TData = unknown> {
  name?: string
  scope?: MiddlewareScope | MiddlewareScope[]
  run(mCtx: MiddlewareContext<TData>, next: NextFn): Promise<void>
}

Lifecycle Scopes

Middleware can target specific points in the agent execution:

// Source: packages/core/src/types/index.ts:269-276
export type MiddlewareScope =
  | 'run:before'
  | 'run:after'
  | 'step:before'
  | 'step:after'
  | 'tool:before'
  | 'tool:after'

Scope Diagram

run:before
  │
  ├─ [Memory Read]
  │
  ├─ step:before
  │    │
  │    ├─ [Reasoning Engine Executes]
  │    │    │
  │    │    ├─ tool:before
  │    │    ├─ [Tool Executes]
  │    │    ├─ tool:after
  │    │    │
  │    │    └─ (repeat for each tool call)
  │    │
  │    └─ step:after
  │
  ├─ [Memory Write]
  │
  └─ run:after

Creating Middleware

Basic Example

import { Middleware, MiddlewareContext, NextFn } from '@agentlib/core'

const loggingMiddleware: Middleware = {
  name: 'logger',
  scope: 'run:before',
  async run(mCtx, next) {
    console.log(`[${mCtx.scope}] Starting run with input: ${mCtx.ctx.input}`)
    await next() // Continue to next middleware or execution
  }
}

agent.use(loggingMiddleware)

Multi-scope Middleware

const timingMiddleware: Middleware = {
  name: 'timer',
  scope: ['run:before', 'run:after'],
  async run(mCtx, next) {
    if (mCtx.scope === 'run:before') {
      mCtx.ctx.data.startTime = Date.now()
    }
    
    await next()
    
    if (mCtx.scope === 'run:after') {
      const elapsed = Date.now() - mCtx.ctx.data.startTime
      console.log(`Run completed in ${elapsed}ms`)
    }
  }
}

Scope-agnostic Middleware

// Omit scope to run on ALL scopes
const traceMiddleware: Middleware = {
  name: 'tracer',
  async run(mCtx, next) {
    console.log(`[${mCtx.scope}] Trace point`)
    await next()
  }
}

Middleware Context

Middleware receives a MiddlewareContext with scope-specific information:

// Source: packages/core/src/types/index.ts:277-286
export interface MiddlewareContext<TData = unknown> {
  scope: MiddlewareScope
  ctx: ExecutionContext<TData>
  /** Tool-specific context, present when scope is tool:* */
  tool?: {
    name: string
    args: Record<string, unknown>
    result?: unknown
  }
}

Accessing Execution Context

const contextMiddleware: Middleware = {
  scope: 'run:before',
  async run(mCtx, next) {
    const { ctx } = mCtx
    
    // Access execution state
    console.log('Session ID:', ctx.sessionId)
    console.log('User input:', ctx.input)
    console.log('Custom data:', ctx.data)
    console.log('Messages so far:', ctx.state.messages.length)
    
    await next()
  }
}

Tool Context

When scope is tool:before or tool:after, the tool property is available:

const toolValidationMiddleware: Middleware = {
  scope: 'tool:before',
  async run(mCtx, next) {
    if (!mCtx.tool) {
      await next()
      return
    }
    
    const { name, args } = mCtx.tool
    
    // Validate tool arguments
    if (name === 'delete_file' && !args.confirm) {
      throw new Error('delete_file requires explicit confirmation')
    }
    
    await next()
  }
}

Source: packages/core/src/reasoning/context.ts:68-116

Middleware Pipeline

Middleware runs in a Koa-style pipeline:

// Source: packages/core/src/middleware/pipeline.ts:7-34
export class MiddlewarePipeline<TData = unknown> {
  private readonly middlewares: Middleware<TData>[] = []

  use(middleware: Middleware<TData>): this {
    this.middlewares.push(middleware)
    return this
  }

  async run(ctx: MiddlewareContext<TData>): Promise<void> {
    const scoped = this.middlewares.filter((m) => {
      if (!m.scope) return true
      const scopes = Array.isArray(m.scope) ? m.scope : [m.scope]
      return scopes.includes(ctx.scope as MiddlewareScope)
    })

    const dispatch = async (index: number): Promise<void> => {
      if (index >= scoped.length) return
      const middleware = scoped[index]
      if (!middleware) return
      const next: NextFn = () => dispatch(index + 1)
      await middleware.run(ctx, next)
    }

    await dispatch(0)
  }
}

Execution Order

agent
  .use(middleware1) // Runs first
  .use(middleware2) // Runs second
  .use(middleware3) // Runs third

Each middleware:

Executes its pre-next() logic
Calls next() to continue the chain
Executes its post-next() logic (if any)

const wrapperMiddleware: Middleware = {
  async run(mCtx, next) {
    console.log('Before next')
    await next() // Continue to next middleware or execution
    console.log('After next')
  }
}

Common Middleware Patterns

Authentication

interface AuthData {
  userId: string
  apiKey: string
}

const authMiddleware: Middleware<AuthData> = {
  name: 'auth',
  scope: 'run:before',
  async run(mCtx, next) {
    const { apiKey } = mCtx.ctx.data
    
    if (!apiKey) {
      throw new Error('API key required')
    }
    
    // Validate API key
    const isValid = await validateApiKey(apiKey)
    if (!isValid) {
      throw new Error('Invalid API key')
    }
    
    await next()
  }
}

Rate Limiting

const rateLimitMiddleware: Middleware = {
  name: 'rate-limit',
  scope: 'run:before',
  async run(mCtx, next) {
    const { sessionId } = mCtx.ctx
    
    const requestCount = await redis.incr(`rate:${sessionId}`)
    if (requestCount === 1) {
      await redis.expire(`rate:${sessionId}`, 60) // 1 minute window
    }
    
    if (requestCount > 10) {
      throw new Error('Rate limit exceeded: 10 requests per minute')
    }
    
    await next()
  }
}

Logging

const loggingMiddleware: Middleware = {
  name: 'logger',
  scope: ['run:before', 'run:after', 'tool:before', 'tool:after'],
  async run(mCtx, next) {
    if (mCtx.scope === 'run:before') {
      console.log(`[RUN START] Session: ${mCtx.ctx.sessionId}, Input: ${mCtx.ctx.input}`)
    }
    
    if (mCtx.scope === 'tool:before' && mCtx.tool) {
      console.log(`[TOOL CALL] ${mCtx.tool.name}`, mCtx.tool.args)
    }
    
    await next()
    
    if (mCtx.scope === 'tool:after' && mCtx.tool) {
      console.log(`[TOOL RESULT] ${mCtx.tool.name}`, mCtx.tool.result)
    }
    
    if (mCtx.scope === 'run:after') {
      console.log(`[RUN END] Tokens: ${mCtx.ctx.state.usage.totalTokens}`)
    }
  }
}

Cost Tracking

const costTrackingMiddleware: Middleware = {
  name: 'cost-tracker',
  scope: 'run:after',
  async run(mCtx, next) {
    await next()
    
    const { usage } = mCtx.ctx.state
    
    // GPT-4 pricing (example)
    const inputCost = (usage.promptTokens / 1000) * 0.03
    const outputCost = (usage.completionTokens / 1000) * 0.06
    const totalCost = inputCost + outputCost
    
    console.log(`Cost: $${totalCost.toFixed(4)}`)
    
    // Store in database
    await db.costs.insert({
      sessionId: mCtx.ctx.sessionId,
      tokens: usage.totalTokens,
      cost: totalCost,
      timestamp: new Date()
    })
  }
}

Input Validation

const validationMiddleware: Middleware = {
  name: 'validator',
  scope: 'run:before',
  async run(mCtx, next) {
    const { input } = mCtx.ctx
    
    // Check for empty input
    if (!input || input.trim().length === 0) {
      throw new Error('Input cannot be empty')
    }
    
    // Check for length
    if (input.length > 10000) {
      throw new Error('Input too long (max 10,000 characters)')
    }
    
    // Content moderation
    const isSafe = await moderateContent(input)
    if (!isSafe) {
      throw new Error('Input contains prohibited content')
    }
    
    await next()
  }
}

Caching

const cachingMiddleware: Middleware = {
  name: 'cache',
  scope: ['run:before', 'run:after'],
  async run(mCtx, next) {
    const cacheKey = `cache:${mCtx.ctx.sessionId}:${hashInput(mCtx.ctx.input)}`
    
    if (mCtx.scope === 'run:before') {
      // Check cache before execution
      const cached = await redis.get(cacheKey)
      if (cached) {
        console.log('Cache hit!')
        // Skip execution by not calling next()
        return
      }
    }
    
    await next()
    
    if (mCtx.scope === 'run:after') {
      // Cache the result
      await redis.set(cacheKey, JSON.stringify(mCtx.ctx.state), 'EX', 3600)
    }
  }
}

Error Handling

const errorHandlerMiddleware: Middleware = {
  name: 'error-handler',
  async run(mCtx, next) {
    try {
      await next()
    } catch (error) {
      console.error(`[${mCtx.scope}] Error:`, error)
      
      // Log to error tracking service
      await errorTracker.capture(error, {
        scope: mCtx.scope,
        sessionId: mCtx.ctx.sessionId,
        input: mCtx.ctx.input
      })
      
      // Re-throw or handle gracefully
      throw error
    }
  }
}

Middleware Registration

Multiple ways to register middleware:

import { createAgent } from '@agentlib/core'

// 1. During agent creation
const agent = createAgent({
  name: 'assistant',
  middleware: [loggingMiddleware, authMiddleware]
})

// 2. Using fluent API
agent
  .use(rateLimitMiddleware)
  .use(costTrackingMiddleware)

// 3. Chained registration
agent.use(middleware1).use(middleware2).use(middleware3)

Middleware Execution Points

Here’s where each scope executes in the agent lifecycle:

run:before

// Source: packages/core/src/agent/agent.ts:91
await this._middleware.run({ scope: 'run:before', ctx })

Executes before:

Memory loading
Reasoning engine execution

run:after

// Source: packages/core/src/agent/agent.ts:94
await this._middleware.run({ scope: 'run:after', ctx })

Executes after:

Reasoning engine completion
Memory persistence

tool:before

// Source: packages/core/src/reasoning/context.ts:68
await middleware.run({ scope: 'tool:before', ctx, tool: { name, args } })

Executes before each tool invocation.

tool:after

// Source: packages/core/src/reasoning/context.ts:116
await middleware.run({ scope: 'tool:after', ctx, tool: { name, args, result } })

Executes after each tool completes (or fails).

step:before and step:after are currently reserved for future use.

Canceling Execution

Middleware can cancel execution by throwing an error or calling ctx.cancel():

const emergencyStopMiddleware: Middleware = {
  scope: 'tool:before',
  async run(mCtx, next) {
    if (mCtx.tool?.name === 'dangerous_operation') {
      mCtx.ctx.cancel('Blocked dangerous operation')
      return // Don't call next()
    }
    await next()
  }
}

Modifying Context

Middleware can modify the execution context:

const contextEnricherMiddleware: Middleware = {
  scope: 'run:before',
  async run(mCtx, next) {
    // Add custom data
    mCtx.ctx.data.timestamp = Date.now()
    mCtx.ctx.data.requestId = crypto.randomUUID()
    
    // Inject additional system message
    mCtx.ctx.state.messages.unshift({
      role: 'system',
      content: 'Always respond concisely.'
    })
    
    await next()
  }
}

Avoid mutating ctx.state directly except in run:before middleware. Other scopes should treat state as read-only to prevent race conditions.

Best Practices

Name your middleware

const myMiddleware: Middleware = {
  name: 'my-middleware', // Helps with debugging
  // ...
}

Always call next()

// Bad: breaks the chain
async run(mCtx, next) {
  console.log('Hello')
  // Forgot to call next()!
}

// Good
async run(mCtx, next) {
  console.log('Before')
  await next()
  console.log('After')
}

Use specific scopes

// Bad: runs on every scope (wasteful)
const middleware: Middleware = {
  async run(mCtx, next) { /* ... */ }
}

// Good: targets specific scope
const middleware: Middleware = {
  scope: 'run:before',
  async run(mCtx, next) { /* ... */ }
}

Handle errors gracefully

async run(mCtx, next) {
  try {
    await next()
  } catch (err) {
    // Log, report, or transform the error
    console.error('Middleware error:', err)
    throw err
  }
}

Keep middleware focused
- One middleware = one concern
- Compose multiple small middleware instead of one large one

Next Steps

Agents - Learn about agent lifecycle
Events - Compare middleware vs. events
Tools - Intercept tool calls with middleware
Reasoning - Understand reasoning engine execution

Get Started

Core Concepts

Guides

Packages

What is Middleware?

Middleware Interface

Lifecycle Scopes

Scope Diagram

Creating Middleware

Basic Example

Multi-scope Middleware

Scope-agnostic Middleware

Middleware Context

Accessing Execution Context

Tool Context

Middleware Pipeline

Execution Order

Common Middleware Patterns

Authentication

Rate Limiting

Logging

Cost Tracking

Input Validation

Caching

Error Handling

Middleware Registration

Middleware Execution Points

run:before

run:after

tool:before

tool:after

Canceling Execution

Modifying Context

Best Practices

Next Steps

Build docs developers (and LLMs) love

Get Started

Core Concepts

Guides

Packages

​What is Middleware?

​Middleware Interface

​Lifecycle Scopes

​Scope Diagram

​Creating Middleware

​Basic Example

​Multi-scope Middleware

​Scope-agnostic Middleware

​Middleware Context

​Accessing Execution Context

​Tool Context

​Middleware Pipeline

​Execution Order

​Common Middleware Patterns

​Authentication

​Rate Limiting

​Logging

​Cost Tracking

​Input Validation

​Caching

​Error Handling

​Middleware Registration

​Middleware Execution Points

​run:before

​run:after

​tool:before

​tool:after

​Canceling Execution

​Modifying Context

​Best Practices

​Next Steps

Build docs developers (and LLMs) love

What is Middleware?

Middleware Interface

Lifecycle Scopes

Scope Diagram

Creating Middleware

Basic Example

Multi-scope Middleware

Scope-agnostic Middleware

Middleware Context

Accessing Execution Context

Tool Context

Middleware Pipeline

Execution Order

Common Middleware Patterns

Authentication

Rate Limiting

Logging

Cost Tracking

Input Validation

Caching

Error Handling

Middleware Registration

Middleware Execution Points

run:before

run:after

tool:before

tool:after

Canceling Execution

Modifying Context

Best Practices

Next Steps