Skip to main content

​
Overview

Request batching is a powerful pattern for optimizing external API calls. Instead of making individual requests for each piece of data, Effect’s RequestResolver automatically batches multiple concurrent requests into a single call, reducing network overhead and improving performance. This is particularly useful when:
  • Fetching data from external APIs that support batch queries
  • Loading multiple database records by ID
  • Calling any service that benefits from bulk operations

​
Defining Request Classes

Use Request.Class to model a single lookup operation. The request class defines:
  • The input parameters (e.g., an ID)
  • The success type (what the request returns)
  • The error type (what can go wrong)
  • Any service requirements (optional)
import { Request, Schema } from "effect"

export class User extends Schema.Class<User>("User")({
  id: Schema.Number,
  name: Schema.String,
  email: Schema.String
}) {}

export class UserNotFound extends Schema.TaggedErrorClass<UserNotFound>()("UserNotFound", {
  id: Schema.Number
}) {}

class GetUserById extends Request.Class<
  { readonly id: number },
  User,          // Success type
  UserNotFound,  // Error type
  never          // Requirements (use never if none)
> {}

​
Creating a RequestResolver

A RequestResolver receives a batch of requests and completes each one with either success or failure: 
import { Effect, Exit, RequestResolver } from "effect"

const usersTable = new Map<number, User>([
  [1, new User({ id: 1, name: "Ada Lovelace", email: "[email protected]" })],
  [2, new User({ id: 2, name: "Alan Turing", email: "[email protected]" })],
  [3, new User({ id: 3, name: "Grace Hopper", email: "[email protected]" })]
])

const resolver = yield* RequestResolver.make<GetUserById>(
  Effect.fnUntraced(function*(entries) {
    for (const entry of entries) {
      const user = usersTable.get(entry.request.id)
      
      if (user) {
        // Complete with success
        entry.completeUnsafe(Exit.succeed(user))
      } else {
        // Complete with error
        entry.completeUnsafe(Exit.fail(new UserNotFound({ id: entry.request.id })))
      }
    }
  })
).pipe(
  // Allow 10ms for requests to accumulate before executing
  RequestResolver.setDelay("10 millis"),
  // Add distributed tracing spans
  RequestResolver.withSpan("Users.getUserById.resolver"),
  // Cache results to avoid duplicate lookups
  RequestResolver.withCache({ capacity: 1024 })
)

​
Using the Resolver

Wrap the resolver in a service method to make it easy to use: 
import { Effect, Layer, ServiceMap } from "effect"

export class Users extends ServiceMap.Service<Users, {
  getUserById(id: number): Effect.Effect<User, UserNotFound>
}>()("app/Users") {
  static readonly layer = Layer.effect(
    Users,
    Effect.gen(function*() {
      // ... create resolver as shown above ...
      
      const getUserById = (id: number) =>
        Effect.request(new GetUserById({ id }), resolver).pipe(
          Effect.withSpan("Users.getUserById", { attributes: { userId: id } })
        )
      
      return { getUserById } as const
    })
  )
}

​
Batching in Action

When you make multiple concurrent requests, the resolver automatically batches them: 
import { Effect } from "effect"

export const batchedLookupExample = Effect.gen(function*() {
  const { getUserById } = yield* Users
  
  // This triggers only ONE call to the resolver with unique IDs [1, 2, 3]
  // Duplicate IDs are automatically deduplicated
  yield* Effect.forEach([1, 2, 1, 3, 2], getUserById, {
    concurrency: "unbounded"
  })
})

​
Configuration Options

​
Batching Delay

Control how long the resolver waits before executing to allow more requests to accumulate: 
RequestResolver.setDelay("10 millis")  // Short delay for real-time APIs
RequestResolver.setDelay("100 millis") // Longer delay for batch-optimized systems

​
Caching

Add a cache to avoid repeated lookups for the same data: 
RequestResolver.withCache({ capacity: 1024 })

​
Tracing

Add distributed tracing spans to monitor resolver performance: 
RequestResolver.withSpan("resolver-name")

​
Best Practices

  1. Use batching for external APIs: Databases, REST APIs, and GraphQL endpoints that support batch queries
  2. Set appropriate delays: Balance latency (shorter delay) vs. batch size (longer delay)
  3. Enable caching: Avoid redundant requests within the same operation
  4. Add tracing spans: Monitor how many requests are batched together
  5. Handle errors per-request: Use entry.completeUnsafe() to handle success/failure individually

​
Real-World Example

Here’s a complete service using request batching for a user lookup system: 
import { Effect, Exit, Layer, Request, RequestResolver, Schema, ServiceMap } from "effect"

export class User extends Schema.Class<User>("User")({
  id: Schema.Number,
  name: Schema.String,
  email: Schema.String
}) {}

export class UserNotFound extends Schema.TaggedErrorClass<UserNotFound>()("UserNotFound", {
  id: Schema.Number
}) {}

export class Users extends ServiceMap.Service<Users, {
  getUserById(id: number): Effect.Effect<User, UserNotFound>
}>()("app/Users") {
  static readonly layer = Layer.effect(
    Users,
    Effect.gen(function*() {
      class GetUserById extends Request.Class<
        { readonly id: number },
        User,
        UserNotFound,
        never
      > {}
      
      const usersTable = new Map<number, User>([
        [1, new User({ id: 1, name: "Ada Lovelace", email: "[email protected]" })],
        [2, new User({ id: 2, name: "Alan Turing", email: "[email protected]" })],
        [3, new User({ id: 3, name: "Grace Hopper", email: "[email protected]" })]
      ])
      
      const resolver = yield* RequestResolver.make<GetUserById>(
        Effect.fnUntraced(function*(entries) {
          for (const entry of entries) {
            const user = usersTable.get(entry.request.id)
            
            if (user) {
              entry.completeUnsafe(Exit.succeed(user))
            } else {
              entry.completeUnsafe(Exit.fail(new UserNotFound({ id: entry.request.id })))
            }
          }
        })
      ).pipe(
        RequestResolver.setDelay("10 millis"),
        RequestResolver.withSpan("Users.getUserById.resolver"),
        RequestResolver.withCache({ capacity: 1024 })
      )
      
      const getUserById = (id: number) =>
        Effect.request(new GetUserById({ id }), resolver).pipe(
          Effect.withSpan("Users.getUserById", { attributes: { userId: id } })
        )
      
      return { getUserById } as const
    })
  )
}

Build docs developers (and LLMs) love

Get started for freeTalk to us