Skip to main content

Models

ADK-TS provides a provider-agnostic interface for working with Large Language Models (LLMs), enabling seamless integration with multiple providers including OpenAI, Google (Gemini/Vertex AI), Anthropic, and more.

Architecture

BaseLlm Interface

All LLM providers extend from BaseLlm, which defines a unified interface:
import { BaseLlm } from "@iqai/adk";
import type { LlmRequest, LlmResponse } from "@iqai/adk";

export abstract class BaseLlm {
  /**
   * The name of the LLM (e.g., "gpt-4o" or "gemini-2.5-flash")
   */
  model: string;

  /**
   * Returns list of supported models in regex for LLMRegistry
   */
  static supportedModels(): string[];

  /**
   * Generates content from the given request.
   * Yields one or more LlmResponse objects.
   */
  async *generateContentAsync(
    llmRequest: LlmRequest,
    stream?: boolean,
  ): AsyncGenerator<LlmResponse, void, unknown>;
}

Supported Providers

OpenAI

Supports GPT-4, GPT-4o, GPT-3.5-turbo, and all OpenAI models:
import { LlmAgent } from "@iqai/adk";

// Using model string (automatically resolved)
const agent = new LlmAgent({
  name: "openai_agent",
  model: "gpt-4o",
  instruction: "You are a helpful assistant",
});

// Supported models:
// - gpt-4o, gpt-4o-mini
// - gpt-4, gpt-4-turbo
// - gpt-3.5-turbo
// - o1-preview, o1-mini
Set the OPENAI_API_KEY environment variable to use OpenAI models.

Google (Gemini & Vertex AI)

Supports Gemini and Vertex AI models:
// Gemini models
const geminiAgent = new LlmAgent({
  name: "gemini_agent",
  model: "gemini-2.5-flash",
});

// Supported Gemini models:
// - gemini-2.5-flash, gemini-2.5-pro
// - gemini-2.0-flash
// - gemini-1.5-pro, gemini-1.5-flash

// Vertex AI models
const vertexAgent = new LlmAgent({
  name: "vertex_agent",
  model: "gemini-1.5-pro",
});
Set the GOOGLE_API_KEY environment variable for Gemini, or configure Google Cloud credentials for Vertex AI.

Anthropic

Supports Claude models:
const claudeAgent = new LlmAgent({
  name: "claude_agent",
  model: "claude-3-5-sonnet",
});

// Supported models:
// - claude-3-5-sonnet
// - claude-3-opus, claude-3-sonnet, claude-3-haiku
Set the ANTHROPIC_API_KEY environment variable to use Claude models.

Vercel AI SDK

Integrate with any provider supported by the Vercel AI SDK:
import { openai } from "@ai-sdk/openai";
import { LlmAgent } from "@iqai/adk";

const agent = new LlmAgent({
  name: "ai_sdk_agent",
  model: openai("gpt-4o"),
});

LLM Registry

The LLMRegistry automatically resolves model strings to provider instances:
import { LLMRegistry } from "@iqai/adk";

// Register a custom provider
class CustomLlm extends BaseLlm {
  static override supportedModels(): string[] {
    return ["custom-.*"]; // Regex pattern
  }

  protected async *generateContentAsyncImpl(
    llmRequest: LlmRequest,
    stream?: boolean,
  ): AsyncGenerator<LlmResponse, void, unknown> {
    // Implementation
  }
}

// Register the provider
LLMRegistry.register(CustomLlm);

// Now you can use it
const agent = new LlmAgent({
  name: "my_agent",
  model: "custom-model-v1", // Automatically uses CustomLlm
});

LlmRequest

LlmRequest is the standardized input format for all LLM providers:
import { LlmRequest } from "@iqai/adk";

const request = new LlmRequest({
  model: "gpt-4o",
  contents: [
    {
      role: "user",
      parts: [{ text: "Hello, how are you?" }],
    },
  ],
  config: {
    temperature: 0.7,
    maxOutputTokens: 1000,
    topP: 0.9,
  },
  toolsDict: {
    search: searchTool,
    calculate: calculatorTool,
  },
});

// Append instructions
request.appendInstructions([
  "Be concise and accurate",
  "Use tools when appropriate",
]);

// Append tools
request.appendTools([newTool]);

Request Configuration

interface GenerateContentConfig {
  /** System instruction for the model */
  systemInstruction?: string;
  
  /** Temperature (0.0 to 2.0) */
  temperature?: number;
  
  /** Maximum tokens to generate */
  maxOutputTokens?: number;
  
  /** Top-p sampling */
  topP?: number;
  
  /** Top-k sampling */
  topK?: number;
  
  /** Response format (for structured output) */
  responseFormat?: {
    type: "json_object" | "text";
  };
  
  /** Stop sequences */
  stopSequences?: string[];
}

LlmResponse

LlmResponse is the standardized output format:
import { LlmResponse } from "@iqai/adk";

interface LlmResponse {
  /** Response content with parts */
  content?: {
    role: string;
    parts: Part[];
  };
  
  /** Text content (extracted from parts) */
  text?: string;
  
  /** Whether response is partial (streaming) */
  partial?: boolean;
  
  /** Usage metadata */
  usageMetadata?: {
    promptTokenCount: number;
    candidatesTokenCount: number;
    totalTokenCount: number;
  };
}

Streaming

All providers support streaming responses:
import { Runner } from "@iqai/adk";

const runner = new Runner({
  appName: "my-app",
  agent,
  sessionService,
});

for await (const event of runner.runAsync({
  userId: "user-123",
  sessionId: session.id,
  newMessage: { parts: [{ text: "Tell me a story" }] },
})) {
  // Events stream in real-time
  if (event.text) {
    process.stdout.write(event.text);
  }
}
Streaming is enabled by default. The framework handles streaming differences across providers automatically.

Function Calling

ADK-TS normalizes function calling across providers:
import { BaseTool, FunctionDeclaration } from "@iqai/adk";

class WeatherTool extends BaseTool {
  constructor() {
    super({
      name: "get_weather",
      description: "Get current weather for a location",
    });
  }

  getDeclaration(): FunctionDeclaration {
    return {
      name: this.name,
      description: this.description,
      parameters: {
        type: "object",
        properties: {
          location: {
            type: "string",
            description: "City name or coordinates",
          },
        },
        required: ["location"],
      },
    };
  }

  async runAsync(args: Record<string, any>): Promise<any> {
    // Implementation
    return { temperature: 72, condition: "sunny" };
  }
}
Function calling is automatically translated to each provider’s format (OpenAI tools, Gemini function calling, Claude tool use, etc.).

Model Configuration

Configure model behavior at the agent level:
const agent = new LlmAgent({
  name: "configured_agent",
  model: "gpt-4o",
  config: {
    temperature: 0.3,
    maxOutputTokens: 2000,
    topP: 0.95,
    stopSequences: ["END"],
  },
});

Context Caching

Optimize costs with context caching (supported by select providers):
import { ContextCacheConfig } from "@iqai/adk";

const agent = new LlmAgent({
  name: "cached_agent",
  model: "gemini-2.5-flash",
  cacheConfig: new ContextCacheConfig({
    ttlSeconds: 3600, // Cache for 1 hour
  }),
});

Error Handling

Handle provider-specific errors gracefully:
try {
  for await (const event of runner.runAsync(params)) {
    // Process event
  }
} catch (error) {
  if (error.message.includes("rate limit")) {
    // Handle rate limiting
  } else if (error.message.includes("invalid_api_key")) {
    // Handle authentication errors
  } else {
    // Handle other errors
  }
}

Model Fallbacks

Implement automatic failover with the ModelFallbackPlugin:
import { ModelFallbackPlugin } from "@iqai/adk";

const agent = new LlmAgent({
  name: "resilient_agent",
  model: "gpt-4o",
  plugins: [
    new ModelFallbackPlugin({
      fallbackModels: ["gpt-4o-mini", "gemini-2.5-flash"],
    }),
  ],
});

Best Practices

  1. Use Model Strings: Prefer model string identifiers over direct provider instances for flexibility.
  2. Set Appropriate Temperatures: Use lower temperatures (0.1-0.3) for factual tasks, higher (0.7-1.0) for creative tasks.
  3. Monitor Token Usage: Track usageMetadata to optimize costs and performance.
  4. Implement Fallbacks: Use multiple models for production resilience.
  5. Leverage Context Caching: Reduce costs for repetitive system instructions and large contexts.
  • Agents - Create and configure agents
  • Tools - Extend model capabilities with tools
  • Flows - Understand the request processing pipeline

Build docs developers (and LLMs) love