Multi-Runtime Support

LlamaIndex.TS is designed to work seamlessly across all major JavaScript runtimes. This is achieved through runtime-specific entry points and the @llamaindex/env abstraction layer.

Supported Runtimes

Node.js

Full-featured support with filesystem access

Deno

Native ESM with secure permissions

Bun

Fast runtime with Node.js compatibility

Cloudflare Workers

Edge runtime with Workers API

Vercel Edge

Vercel Edge Runtime for serverless

Browser

Limited support for client-side use cases

Runtime-Specific Entry Points

The llamaindex package provides different entry points that are automatically selected based on your runtime:

package.json conditional exports

{
  "exports": {
    ".": {
      "react-server": {
        "types": "./dist/type/index.react-server.d.ts",
        "default": "./dist/index.react-server.js"
      },
      "workerd": {
        "types": "./dist/type/index.workerd.d.ts",
        "default": "./dist/index.workerd.js"
      },
      "edge-light": {
        "types": "./dist/type/index.edge.d.ts",
        "default": "./dist/index.edge.js"
      },
      "import": {
        "types": "./dist/type/index.d.ts",
        "default": "./dist/index.js"
      },
      "require": {
        "types": "./dist/type/index.d.cts",
        "default": "./dist/cjs/index.cjs"
      }
    }
  }
}

The correct entry point is automatically selected by your bundler or runtime based on export conditions.

Entry Point Mapping

Runtime	Export Condition	Entry Point File
Node.js ESM	`import`	`src/index.ts`
Node.js CJS	`require`	`src/index.ts` (compiled to CJS)
Cloudflare Workers	`workerd`	`src/index.workerd.ts`
Vercel Edge	`edge-light`	`src/index.edge.ts`
React Server Components	`react-server`	`src/index.react-server.ts`
Deno/Bun	`import`	`src/index.ts`

Environment Abstraction Layer

The @llamaindex/env package provides a unified API across all runtimes:

@llamaindex/env exports

// Node.js runtime (index.ts)
export { fs, path, createWriteStream, randomUUID, createHash } from 'node:*';
export { AsyncLocalStorage } from './als/index.node.js';

// Cloudflare Workers runtime (index.workerd.ts)
export { AsyncLocalStorage } from './als/index.workerd.js';
export { getEnv } from './utils/index.js';
// No filesystem exports

// Edge Runtime (index.edge-light.ts)
export { AsyncLocalStorage } from './als/index.non-node.js';
export * from './node-polyfill.js';  // Polyfills for Node APIs

@llamaindex/env handles all runtime differences internally, so you can write runtime-agnostic code.

Runtime-Specific Examples

Node.js

Full Features
Runtime Check

Node.js provides the complete LlamaIndex.TS feature set:

Node.js with filesystem

import { VectorStoreIndex, SimpleDirectoryReader, Settings } from "llamaindex";
import { OpenAI } from "@llamaindex/openai";

Settings.llm = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

// File system access available
const reader = new SimpleDirectoryReader();
const documents = await reader.loadData({ directoryPath: "./docs" });

// Full index functionality
const index = await VectorStoreIndex.fromDocuments(documents);
const queryEngine = index.asQueryEngine();

const response = await queryEngine.query({
  query: "What is LlamaIndex?"
});

Verify Node.js runtime

import { process } from "@llamaindex/env";

if (typeof process !== "undefined" && process.versions?.node) {
  console.log("Running in Node.js", process.versions.node);
}

Cloudflare Workers

Worker Implementation
wrangler.toml

src/index.ts

// Cloudflare Workers automatically use index.workerd.ts entry point
interface Env {
  OPENAI_API_KEY: string;
}

export default {
  async fetch(request: Request, env: Env, ctx: ExecutionContext): Promise<Response> {
    // Set environment variables from Workers env
    const { setEnvs } = await import("@llamaindex/env");
    setEnvs(env);
    
    // Dynamic imports for optimal bundle size
    const { OpenAI, OpenAIAgent } = await import("@llamaindex/openai");
    
    const text = await request.text();
    
    const agent = new OpenAIAgent({
      llm: new OpenAI({ apiKey: env.OPENAI_API_KEY }),
      tools: [],
    });
    
    const responseStream = await agent.chat({
      stream: true,
      message: text,
    });
    
    // Return streaming response
    const textEncoder = new TextEncoder();
    const stream = responseStream.pipeThrough(
      new TransformStream({
        transform: (chunk, controller) => {
          controller.enqueue(textEncoder.encode(chunk.delta));
        },
      })
    );
    
    return new Response(stream);
  },
};

Cloudflare Workers config

name = "llamaindex-agent"
main = "src/index.ts"
compatibility_date = "2024-04-23"
compatibility_flags = ["nodejs_compat"]

[vars]
# Non-sensitive environment variables

# For secrets, use: wrangler secret put OPENAI_API_KEY

Cloudflare Workers has no filesystem access. Use in-memory data or external storage services.

Vercel Edge Runtime

Next.js Route Handler
Runtime Detection
next.config.mjs

app/api/chat/route.ts

import { OpenAI } from "@llamaindex/openai";
import { Settings } from "llamaindex";

// Force Edge Runtime
export const runtime = "edge";

Settings.llm = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
});

export async function POST(request: Request) {
  const { message } = await request.json();
  
  const response = await Settings.llm.chat({
    messages: [{ role: "user", content: message }],
  });
  
  return Response.json(response);
}

utils/llm.ts

import "llamaindex";

// Verify Edge Runtime
// @ts-expect-error EdgeRuntime is not defined
if (typeof EdgeRuntime !== "string") {
  throw new Error("Expected to run in EdgeRuntime");
}

next.config.mjs

import { withLlamaIndex } from "llamaindex/next";

/** @type {import('next').NextConfig} */
const nextConfig = {
  // Your Next.js config
};

export default withLlamaIndex(nextConfig);

Deno

Deno with import maps

import { VectorStoreIndex, Document, Settings } from "npm:llamaindex";
import { OpenAI } from "npm:@llamaindex/openai";

Settings.llm = new OpenAI({ apiKey: Deno.env.get("OPENAI_API_KEY") });

const documents = [
  new Document({ text: "LlamaIndex is a data framework for LLMs" }),
];

const index = await VectorStoreIndex.fromDocuments(documents);
const queryEngine = index.asQueryEngine();

const response = await queryEngine.query({
  query: "What is LlamaIndex?"
});

console.log(response.toString());

Bun

Bun runtime

import { VectorStoreIndex, Settings } from "llamaindex";
import { OpenAI } from "@llamaindex/openai";

Settings.llm = new OpenAI({ apiKey: Bun.env.OPENAI_API_KEY });

// Bun has fast file I/O
const file = Bun.file("./document.txt");
const text = await file.text();

const documents = [new Document({ text })];
const index = await VectorStoreIndex.fromDocuments(documents);

Runtime Capabilities

Feature	Node.js	Deno	Bun	CF Workers	Vercel Edge	Browser
File System	✅	✅	✅	❌	❌	❌
Vector Indices	✅	✅	✅	✅	✅	⚠️
LLM Chat	✅	✅	✅	✅	✅	✅
Streaming	✅	✅	✅	✅	✅	✅
Embeddings	✅	✅	✅	✅	✅	⚠️
Agents	✅	✅	✅	✅	✅	⚠️
Async Local Storage	✅	✅	✅	✅	✅	❌
Local Models	✅	✅	✅	❌	❌	⚠️

⚠️ = Limited support or requires specific configuration

Environment Variables

Node.js / Deno / Bun

.env

OPENAI_API_KEY=sk-...
PINECONE_API_KEY=...

Load with dotenv

import "dotenv/config";
import { Settings } from "llamaindex";
import { OpenAI } from "@llamaindex/openai";

Settings.llm = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

Cloudflare Workers

Use setEnvs helper

interface Env {
  OPENAI_API_KEY: string;
}

export default {
  async fetch(request: Request, env: Env) {
    const { setEnvs } = await import("@llamaindex/env");
    setEnvs(env);  // Makes env.* available via getEnv()
    
    // Now use environment variables
    const { OpenAI } = await import("@llamaindex/openai");
    const llm = new OpenAI({ apiKey: env.OPENAI_API_KEY });
  }
};

Vercel Edge Runtime

Edge Runtime env

// Environment variables available via process.env
export const runtime = "edge";

Settings.llm = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

Runtime Limitations

Cloudflare Workers

Limitations:

No file system access (use KV, R2, or D1 for storage)
No native Node.js modules
1MB script size limit (use dynamic imports)
CPU time limits per request

Workarounds:

Use vectorStores with external services (Pinecone, Qdrant)
Dynamic imports to reduce initial bundle size
Stream responses for long-running operations

Vercel Edge Runtime

Limitations:

No file system access
Limited Node.js API compatibility
1MB middleware size limit
Cold start considerations

Best Practices:

Use withLlamaIndex Next.js plugin
Leverage Vercel KV or external vector stores
Keep dependencies minimal

Browser

Limitations:

No file system access
CORS restrictions for API calls
Bundle size concerns
No native crypto APIs in older browsers

Use Cases:

Client-side chat interfaces
Browser-based document processing with FileReader API
WebAssembly embeddings (limited)

AsyncLocalStorage

LlamaIndex.TS uses AsyncLocalStorage for context-aware Settings:

Settings context isolation

import { Settings } from "llamaindex";
import { OpenAI, Anthropic } from "@llamaindex/openai";

// Different LLMs in different contexts
Settings.withLLM(new OpenAI({ model: "gpt-4" }), async () => {
  // This code uses GPT-4
  const response = await Settings.llm.chat({ messages });
});

Settings.withLLM(new Anthropic({ model: "claude-3" }), async () => {
  // This code uses Claude-3
  const response = await Settings.llm.chat({ messages });
});

AsyncLocalStorage is polyfilled for runtimes that don’t have native support.

Best Practices

Use Dynamic Imports

In edge runtimes, use dynamic imports to reduce bundle size:

const { OpenAI } = await import("@llamaindex/openai");

Test Multiple Runtimes

Validate your code works across target runtimes in CI/CD

Handle Missing APIs

Check for API availability before using runtime-specific features

Optimize Bundle Size

Use sub-module imports and tree-shaking for edge deployments

Next Steps

Settings Configuration

Learn how to configure global settings for your runtime

Data Flow

Understand how data flows through the framework

Getting Started

Core Concepts

Building with LlamaIndex

Data Management

Models & Embeddings

Retrievers & Indices

Advanced Features

Supported Runtimes

Node.js

Deno

Bun

Cloudflare Workers

Vercel Edge

Browser

Runtime-Specific Entry Points

Entry Point Mapping

Environment Abstraction Layer

Runtime-Specific Examples

Node.js

Cloudflare Workers

Vercel Edge Runtime

Deno

Bun

Runtime Capabilities

Environment Variables

Node.js / Deno / Bun

Cloudflare Workers

Vercel Edge Runtime

Runtime Limitations

AsyncLocalStorage

Best Practices

Use Dynamic Imports

Test Multiple Runtimes

Handle Missing APIs

Optimize Bundle Size

Next Steps

Settings Configuration

Data Flow

Build docs developers (and LLMs) love

Getting Started

Core Concepts

Building with LlamaIndex

Data Management

Models & Embeddings

Retrievers & Indices

Advanced Features

​Supported Runtimes

Node.js

Deno

Bun

Cloudflare Workers

Vercel Edge

Browser

​Runtime-Specific Entry Points

​Entry Point Mapping

​Environment Abstraction Layer

​Runtime-Specific Examples

​Node.js

​Cloudflare Workers

​Vercel Edge Runtime

​Deno

​Bun

​Runtime Capabilities

​Environment Variables

​Node.js / Deno / Bun

​Cloudflare Workers

​Vercel Edge Runtime

​Runtime Limitations

​AsyncLocalStorage

​Best Practices

Use Dynamic Imports

Test Multiple Runtimes

Handle Missing APIs

Optimize Bundle Size

​Next Steps

Settings Configuration

Data Flow

Build docs developers (and LLMs) love

Supported Runtimes

Runtime-Specific Entry Points

Entry Point Mapping

Environment Abstraction Layer

Runtime-Specific Examples

Node.js

Cloudflare Workers

Vercel Edge Runtime

Deno

Bun

Runtime Capabilities

Environment Variables

Node.js / Deno / Bun

Cloudflare Workers

Vercel Edge Runtime

Runtime Limitations

AsyncLocalStorage

Best Practices

Next Steps