JavaScript/TypeScript SDK

The Portkey JavaScript SDK provides a fully-typed interface to access 250+ LLMs through a unified API with production-grade routing, fallbacks, and observability.

Overview

The JavaScript/TypeScript SDK offers:

OpenAI-Compatible API: Drop-in replacement for OpenAI SDK
250+ LLMs: Access any LLM through a unified interface
TypeScript Support: Full type definitions and autocomplete
Cross-Platform: Works in Node.js, Deno, and modern browsers
Advanced Routing: Fallbacks, load balancing, and retries
Production Features: Caching, guardrails, and observability

Installation

npm install portkey-ai

Quick Start

Get Your API Keys

Import and Initialize

import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: "your-portkey-api-key",
    virtualKey: "your-openai-virtual-key"
});

Make Your First Request

const response = await portkey.chat.completions.create({
    model: "gpt-4",
    messages: [{role: "user", content: "Hello!"}]
});

console.log(response.choices[0].message.content);

Basic Usage

Chat Completions

import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: "your-portkey-api-key",
    virtualKey: "your-openai-virtual-key"
});

const response = await portkey.chat.completions.create({
    model: "gpt-4",
    messages: [
        {role: "system", content: "You are a helpful assistant."},
        {role: "user", content: "Explain quantum computing simply."}
    ],
    temperature: 0.7,
    maxTokens: 500
});

console.log(response.choices[0].message.content);

Streaming Responses

import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: "your-portkey-api-key",
    virtualKey: "your-openai-virtual-key"
});

const stream = await portkey.chat.completions.create({
    model: "gpt-4",
    messages: [{role: "user", content: "Write a story"}],
    stream: true
});

for await (const chunk of stream) {
    if (chunk.choices[0]?.delta?.content) {
        process.stdout.write(chunk.choices[0].delta.content);
    }
}

Completions (Legacy)

const response = await portkey.completions.create({
    model: "gpt-3.5-turbo-instruct",
    prompt: "Once upon a time",
    maxTokens: 100
});

console.log(response.choices[0].text);

Using Different Providers

Switch between providers easily:

import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: "your-portkey-api-key",
    provider: "openai",
    Authorization: "your-openai-api-key"
});

const response = await portkey.chat.completions.create({
    model: "gpt-4",
    messages: [{role: "user", content: "Hello!"}]
});

Advanced Routing with Configs

Fallback Strategy

Automatically fallback to backup providers:

import Portkey from 'portkey-ai';

const config = {
    strategy: {mode: "fallback"},
    targets: [
        {virtualKey: "openai-virtual-key"},
        {virtualKey: "anthropic-virtual-key"},
        {virtualKey: "together-virtual-key"}
    ]
};

const portkey = new Portkey({
    apiKey: "your-portkey-api-key",
    config: config
});

// Will automatically fallback if OpenAI fails
const response = await portkey.chat.completions.create({
    model: "gpt-4",
    messages: [{role: "user", content: "Hello!"}]
});

Load Balancing

Distribute traffic across multiple providers:

const config = {
    strategy: {mode: "loadbalance"},
    targets: [
        {virtualKey: "openai-key-1", weight: 0.7},
        {virtualKey: "openai-key-2", weight: 0.3}
    ]
};

const portkey = new Portkey({
    apiKey: "your-portkey-api-key",
    config: config
});

Automatic Retries

const config = {
    retry: {
        attempts: 5,
        onStatusCodes: [429, 500, 502, 503]
    }
};

const portkey = new Portkey({
    apiKey: "your-portkey-api-key",
    virtualKey: "your-virtual-key",
    config: config
});

Request Timeouts

const config = {
    requestTimeout: 30000  // 30 seconds
};

const portkey = new Portkey({
    apiKey: "your-portkey-api-key",
    virtualKey: "your-virtual-key",
    config: config
});

Caching

Enable caching to reduce costs and latency:

Simple Caching

import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: "your-portkey-api-key",
    virtualKey: "your-virtual-key"
});

const response = await portkey.chat.completions.create({
    model: "gpt-4",
    messages: [{role: "user", content: "What is 2+2?"}]
}, {
    cache: "simple",
    cacheForceRefresh: false
});

Semantic Caching

const config = {
    cache: {
        mode: "semantic",
        maxAge: 3600  // 1 hour
    }
};

const portkey = new Portkey({
    apiKey: "your-portkey-api-key",
    virtualKey: "your-virtual-key",
    config: config
});

const response = await portkey.chat.completions.create({
    model: "gpt-4",
    messages: [{role: "user", content: "What's two plus two?"}]
});

Guardrails

Add input/output guardrails:

const config = {
    outputGuardrails: [
        {
            "default.contains": {
                operator: "none",
                words: ["inappropriate", "offensive"]
            },
            deny: true
        }
    ]
};

const portkey = new Portkey({
    apiKey: "your-portkey-api-key",
    virtualKey: "your-virtual-key",
    config: config
});

Metadata and Tracking

Add custom metadata for better observability:

import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: "your-portkey-api-key",
    virtualKey: "your-virtual-key"
});

const response = await portkey.chat.completions.create({
    model: "gpt-4",
    messages: [{role: "user", content: "Hello!"}]
}, {
    metadata: {
        userId: "user_123",
        sessionId: "session_456",
        environment: "production"
    },
    traceId: "custom-trace-id"
});

Embeddings

Generate embeddings:

import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: "your-portkey-api-key",
    virtualKey: "your-openai-virtual-key"
});

const response = await portkey.embeddings.create({
    model: "text-embedding-3-small",
    input: "The quick brown fox jumps over the lazy dog"
});

console.log(response.data[0].embedding);

Image Generation

Generate images:

import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: "your-portkey-api-key",
    virtualKey: "your-openai-virtual-key"
});

const response = await portkey.images.generate({
    model: "dall-e-3",
    prompt: "A serene landscape with mountains",
    n: 1,
    size: "1024x1024"
});

console.log(response.data[0].url);

Audio

Speech to Text

import Portkey from 'portkey-ai';
import fs from 'fs';

const portkey = new Portkey({
    apiKey: "your-portkey-api-key",
    virtualKey: "your-openai-virtual-key"
});

const audioFile = fs.createReadStream("speech.mp3");
const response = await portkey.audio.transcriptions.create({
    model: "whisper-1",
    file: audioFile
});

console.log(response.text);

Text to Speech

import Portkey from 'portkey-ai';
import fs from 'fs';

const portkey = new Portkey({
    apiKey: "your-portkey-api-key",
    virtualKey: "your-openai-virtual-key"
});

const response = await portkey.audio.speech.create({
    model: "tts-1",
    voice: "alloy",
    input: "Hello, this is a test."
});

const buffer = Buffer.from(await response.arrayBuffer());
fs.writeFileSync("output.mp3", buffer);

Function Calling

import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: "your-portkey-api-key",
    virtualKey: "your-openai-virtual-key"
});

const tools = [
    {
        type: "function",
        function: {
            name: "get_weather",
            description: "Get the current weather in a location",
            parameters: {
                type: "object",
                properties: {
                    location: {
                        type: "string",
                        description: "City name"
                    },
                    unit: {
                        type: "string",
                        enum: ["celsius", "fahrenheit"]
                    }
                },
                required: ["location"]
            }
        }
    }
];

const response = await portkey.chat.completions.create({
    model: "gpt-4",
    messages: [{role: "user", content: "What's the weather in Paris?"}],
    tools: tools,
    toolChoice: "auto"
});

if (response.choices[0].message.toolCalls) {
    const toolCall = response.choices[0].message.toolCalls[0];
    console.log(`Function: ${toolCall.function.name}`);
    console.log(`Arguments: ${toolCall.function.arguments}`);
}

Vision

Analyze images:

import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: "your-portkey-api-key",
    virtualKey: "your-openai-virtual-key"
});

const response = await portkey.chat.completions.create({
    model: "gpt-4-vision-preview",
    messages: [
        {
            role: "user",
            content: [
                {type: "text", text: "What's in this image?"},
                {
                    type: "image_url",
                    imageUrl: {
                        url: "https://example.com/image.jpg"
                    }
                }
            ]
        }
    ],
    maxTokens: 300
});

console.log(response.choices[0].message.content);

Error Handling

import Portkey from 'portkey-ai';
import {
    PortkeyError,
    APIError,
    RateLimitError,
    APIConnectionError
} from 'portkey-ai';

const portkey = new Portkey({
    apiKey: "your-portkey-api-key",
    virtualKey: "your-virtual-key"
});

try {
    const response = await portkey.chat.completions.create({
        model: "gpt-4",
        messages: [{role: "user", content: "Hello!"}]
    });
} catch (error) {
    if (error instanceof RateLimitError) {
        console.error("Rate limit exceeded:", error);
    } else if (error instanceof APIConnectionError) {
        console.error("Connection error:", error);
    } else if (error instanceof APIError) {
        console.error("API error:", error);
    } else if (error instanceof PortkeyError) {
        console.error("Portkey error:", error);
    }
}

Browser Usage

Next.js API Route

// app/api/chat/route.ts
import Portkey from 'portkey-ai';
import { NextResponse } from 'next/server';

export async function POST(req: Request) {
    const { messages } = await req.json();
    
    const portkey = new Portkey({
        apiKey: process.env.PORTKEY_API_KEY!,
        virtualKey: process.env.OPENAI_VIRTUAL_KEY!
    });
    
    const response = await portkey.chat.completions.create({
        model: "gpt-4",
        messages: messages
    });
    
    return NextResponse.json(response);
}

React Hook

// hooks/useChat.ts
import { useState } from 'react';

export function useChat() {
    const [loading, setLoading] = useState(false);
    const [messages, setMessages] = useState([]);
    
    const sendMessage = async (content: string) => {
        setLoading(true);
        
        const newMessages = [...messages, {role: "user", content}];
        setMessages(newMessages);
        
        const response = await fetch('/api/chat', {
            method: 'POST',
            headers: {'Content-Type': 'application/json'},
            body: JSON.stringify({messages: newMessages})
        });
        
        const data = await response.json();
        setMessages([...newMessages, {
            role: "assistant",
            content: data.choices[0].message.content
        }]);
        
        setLoading(false);
    };
    
    return { messages, sendMessage, loading };
}

Best Practices

Use Environment Variables

Store API keys in environment variables:

const portkey = new Portkey({
    apiKey: process.env.PORTKEY_API_KEY,
    virtualKey: process.env.VIRTUAL_KEY
});

Implement Fallbacks

Always configure fallback providers for production:

const config = {strategy: {mode: "fallback"}, targets: [...]};

Enable Caching

Use caching for repeated queries to reduce costs:

const config = {cache: {mode: "semantic", maxAge: 3600}};

Add Metadata

Always add metadata for better tracking:

metadata: {userId: "user_123", environment: "production"}

Handle Errors Gracefully

Implement proper error handling for production applications.

Complete Example

import Portkey from 'portkey-ai';

// Configure with fallbacks and caching
const config = {
    strategy: {mode: "fallback"},
    targets: [
        {virtualKey: "openai-key"},
        {virtualKey: "anthropic-key"}
    ],
    retry: {attempts: 3},
    cache: {mode: "semantic", maxAge: 3600}
};

const portkey = new Portkey({
    apiKey: process.env.PORTKEY_API_KEY!,
    config: config
});

async function chat(userMessage: string) {
    try {
        const response = await portkey.chat.completions.create({
            model: "gpt-4",
            messages: [
                {role: "system", content: "You are a helpful assistant."},
                {role: "user", content: userMessage}
            ],
            temperature: 0.7,
            maxTokens: 500
        }, {
            metadata: {
                userId: "user_123",
                feature: "chat",
                environment: "production"
            },
            traceId: "chat-session-001"
        });
        
        return response.choices[0].message.content;
    } catch (error) {
        console.error("Chat error:", error);
        throw error;
    }
}

// Usage
const reply = await chat("Explain AI in simple terms.");
console.log(reply);

Resources

Questions? Join our Discord community for help.

Agent Frameworks

SDKs

Overview

Installation

Quick Start

Basic Usage

Chat Completions

Streaming Responses

Completions (Legacy)

Using Different Providers

Advanced Routing with Configs

Fallback Strategy

Load Balancing

Automatic Retries

Request Timeouts

Caching

Simple Caching

Semantic Caching

Guardrails

Metadata and Tracking

Embeddings

Image Generation

Audio

Speech to Text

Text to Speech

Function Calling

Vision

Error Handling

Browser Usage

Next.js API Route

React Hook

Best Practices

Complete Example

Resources

Build docs developers (and LLMs) love

Agent Frameworks

SDKs

​Overview

​Installation

​Quick Start

​Basic Usage

​Chat Completions

​Streaming Responses

​Completions (Legacy)

​Using Different Providers

​Advanced Routing with Configs

​Fallback Strategy

​Load Balancing

​Automatic Retries

​Request Timeouts

​Caching

​Simple Caching

​Semantic Caching

​Guardrails

​Metadata and Tracking

​Embeddings

​Image Generation

​Audio

​Speech to Text

​Text to Speech

​Function Calling

​Vision

​Error Handling

​Browser Usage

​Next.js API Route

​React Hook

​Best Practices

​Complete Example

​Resources

Build docs developers (and LLMs) love

Overview

Installation

Quick Start

Basic Usage

Chat Completions

Streaming Responses

Completions (Legacy)

Using Different Providers

Advanced Routing with Configs

Fallback Strategy

Load Balancing

Automatic Retries

Request Timeouts

Caching

Simple Caching

Semantic Caching

Guardrails

Metadata and Tracking

Embeddings

Image Generation

Audio

Speech to Text

Text to Speech

Function Calling

Vision

Error Handling

Browser Usage

Next.js API Route

React Hook

Best Practices

Complete Example

Resources