Services

Service classes for AI chat, text-to-speech, speech-to-text, translation, summarization, rewriting, and content generation.

AIService

Multi-provider AI service with unified interface for Chrome AI, OpenAI, and Ollama.

Methods

configure

function

configure(config: AIConfig, tabId?: number): Promise<boolean>

Configure AI client with provider settings.Parameters:

config: Configuration object
config.provider: ‘chrome-ai’ | ‘openai’ | ‘ollama’
config.chromeAi: Chrome AI settings (temperature, topK, enableImageSupport, enableAudioSupport)
config.openai: OpenAI settings (apiKey, model, temperature, maxTokens)
config.ollama: Ollama settings (endpoint, model, temperature, maxTokens)
tabId: Tab ID (extension mode only)

Returns: Success status

isConfigured

function

isConfigured(tabId?: number): boolean

Check if service is configured and ready.

chat

function

chat(
  messages: Message[],
  options?: ChatOptions,
  tabId?: number
): AsyncGenerator<string>

Stream chat completion responses.Parameters:

messages: Array of message objects with role and content
options: { systemPrompt?, temperature?, maxTokens?, images?, audios? }
tabId: Tab ID (extension mode only)

Yields: Text chunks as they arriveExample:

for await (const chunk of aiService.chat(messages)) {
  console.log(chunk);
}

abortRequest

function

abortRequest(tabId?: number): void

Abort ongoing chat request.

Supported Providers

Chrome AI (Gemini Nano)

On-device inference (no API key required)
Multi-modal: Images and audio support
Requirements: Chrome 138+ with flags enabled
Model: Gemini Nano (2B-4B parameters)

OpenAI

Cloud-based (API key required)
Models: GPT-4, GPT-4 Turbo, GPT-4o, GPT-3.5 Turbo
Multi-modal: Images support (GPT-4 Vision)
Audio: Via Whisper transcription

Ollama

Self-hosted (local server)
Models: Llama 2, Mistral, Mixtral, etc.
Multi-modal: Depends on model (LLaVA for images)
Endpoint: Default http://localhost:11434

TTSService

Multi-provider Text-to-Speech service with streaming generation and audio queue management.

Methods

configure

function

configure(config: TTSConfig, tabId?: number): boolean

Configure TTS client with provider settings.Parameters:

config.provider: ‘kokoro’ | ‘openai’ | ‘openai-compatible’
config.kokoro: Kokoro settings (modelId, voice, speed, device)
config.openai: OpenAI TTS settings (apiKey, model, voice, speed)
config['openai-compatible']: Generic TTS API settings
tabId: Tab ID (extension mode only)

isConfigured

function

isConfigured(tabId?: number): boolean

Check if service is configured and ready.

generateChunkedSpeech

function

generateChunkedSpeech(
  text: string,
  voice?: string,
  chunkSize?: number,
  minChunkSize?: number,
  sessionId?: string,
  tabId?: number
): Promise<string[]>

Generate TTS audio with intelligent text chunking.Parameters:

text: Text to synthesize
voice: Voice ID (overrides config)
chunkSize: Target chunk size in characters (default: 500)
minChunkSize: Minimum chunk size (default: 100)
sessionId: Session identifier for tracking
tabId: Tab ID (extension mode only)

Returns: Array of audio blob URLs

playAudioSequence

function

playAudioSequence(
  audioBlobUrls: string[],
  sessionId?: string
): Promise<void>

Play sequence of audio chunks with queue management.Parameters:

audioBlobUrls: Array of blob URLs to play
sessionId: Session identifier

stopPlayback

function

stopPlayback(): void

Stop current TTS playback and clear queue.

initializeBVMDConverter

function

initializeBVMDConverter(scene: Scene): void

Initialize lip-sync animation converter with Babylon.js scene.

setSpeakCallback

function

setSpeakCallback(callback: (text: string, bvmdUrl: string) => void): void

Set callback for triggering speak animations with lip sync.

Supported Providers

Kokoro TTS (Local)

On-device synthesis using ONNX runtime
Voices: 30+ high-quality neural voices
Backends: WebGPU (fast) or WASM (compatible)
Languages: English, Japanese, Chinese, Korean, French, Spanish
Lip Sync: Automatic BVMD generation for 3D character

OpenAI TTS

Cloud-based (API key required)
Models: tts-1 (fast), tts-1-hd (high quality)
Voices: alloy, echo, fable, onyx, nova, shimmer
Speed: Adjustable 0.25x to 4.0x

OpenAI-Compatible TTS

Generic TTS API (Cartesia, ElevenLabs, etc.)
Custom endpoint and voice configuration
OpenAI SDK compatibility

STTService

Multi-provider Speech-to-Text service supporting one-shot transcription and continuous streaming.

Methods

configure

function

configure(config: STTConfig, tabId?: number): boolean

Configure STT client with provider settings.Parameters:

config.provider: ‘chrome-ai-multimodal’ | ‘openai’ | ‘openai-compatible’
config['chrome-ai-multimodal']: Chrome AI settings
config.openai: OpenAI Whisper settings
config['openai-compatible']: Generic STT API settings

isConfigured

function

isConfigured(tabId?: number): boolean

Check if service is configured.

startRecording

function

startRecording(): Promise<boolean>

Start recording audio from microphone.Returns: Success status

stopRecording

function

stopRecording(): Promise<void>

Stop recording and transcribe audio.

setTranscriptionCallback

function

setTranscriptionCallback(callback: (text: string) => void): void

Set callback for transcription results.

setErrorCallback

function

setErrorCallback(callback: (error: Error) => void): void

Set callback for error handling.

TranslatorService

Multi-provider translation service with support for Chrome AI Translator API and LLM-based translation.

Methods

configure

function

configure(config: TranslatorConfig, tabId?: number): Promise<boolean>

Configure translator with provider settings.Parameters:

config.provider: ‘chrome-ai’ | ‘openai’ | ‘ollama’

translateStreaming

function

translateStreaming(
  text: string,
  sourceLang: string,
  targetLang: string,
  tabId?: number
): AsyncGenerator<string>

Stream translation results.Parameters:

text: Text to translate
sourceLang: Source language code (e.g., ‘en’)
targetLang: Target language code (e.g., ‘es’)
tabId: Tab ID (extension mode only)

Yields: Translated text chunks

checkAvailability

function

checkAvailability(
  sourceLang: string,
  targetLang: string,
  tabId?: number
): Promise<'readily' | 'downloading' | 'downloadable' | 'unavailable'>

Check if language pair is available for translation.

abort

function

abort(tabId?: number): void

Abort ongoing translation request.

SummarizerService

Multi-provider text summarization service supporting Chrome AI Summarizer API and LLM-based summarization.

Methods

configure

function

configure(config: SummarizerConfig, tabId?: number): Promise<boolean>

Configure summarizer with provider settings.

summarizeStreaming

function

summarizeStreaming(
  text: string,
  options?: SummarizerOptions,
  tabId?: number
): AsyncGenerator<string>

Stream summarization results.Parameters:

text: Text to summarize
options.type: ‘tldr’ | ‘headline’ | ‘key-points’ | ‘teaser’
options.format: ‘plain-text’ | ‘markdown’
options.length: ‘short’ | ‘medium’ | ‘long’
tabId: Tab ID (extension mode only)

Yields: Summary text chunks

checkAvailability

function

checkAvailability(tabId?: number): Promise<string>

Check if summarization is available.

abort

function

abort(tabId?: number): void

Abort ongoing summarization request.

RewriterService

Multi-provider text rewriting service with tone, format, and length adjustments.

Methods

configure

function

configure(config: RewriterConfig, tabId?: number): Promise<boolean>

Configure rewriter with provider settings.

rewriteStreaming

function

rewriteStreaming(
  text: string,
  options?: RewriterOptions,
  tabId?: number
): AsyncGenerator<string>

Stream rewriting results.Parameters:

text: Text to rewrite
options.tone: ‘as-is’ | ‘more-formal’ | ‘more-casual’ | ‘professional’ | ‘friendly’
options.format: ‘as-is’ | ‘plain-text’ | ‘markdown’
options.length: ‘as-is’ | ‘shorter’ | ‘longer’
options.context: Additional context for rewriting
tabId: Tab ID (extension mode only)

Yields: Rewritten text chunks

abort

function

abort(tabId?: number): void

Abort ongoing rewrite request.

WriterService

Multi-provider content generation service for creating new text from prompts.

Methods

configure

function

configure(config: WriterConfig, tabId?: number): Promise<boolean>

Configure writer with provider settings.

writeStreaming

function

writeStreaming(
  prompt: string,
  options?: WriterOptions,
  tabId?: number
): AsyncGenerator<string>

Stream content generation results.Parameters:

prompt: Writing prompt
options.tone: ‘neutral’ | ‘formal’ | ‘casual’ | ‘professional’ | ‘friendly’
options.format: ‘plain-text’ | ‘markdown’
options.length: ‘short’ | ‘medium’ | ‘long’
options.context: Additional context
tabId: Tab ID (extension mode only)

Yields: Generated text chunks

abort

function

abort(tabId?: number): void

Abort ongoing write request.

ChatHistoryService

Service for managing persistent chat history with message trees and media storage.

Methods

saveChat

function

saveChat(chatData: ChatData): Promise<string>

Save a chat with all associated data.Parameters:

chatData.chatId: Chat ID (auto-generated if not provided)
chatData.chatService: ChatService instance (for tree structure)
chatData.messages: Flat message array (backward compatibility)
chatData.title: Chat title (auto-generated if not provided)
chatData.isTemp: Skip saving if true
chatData.metadata: Additional metadata

Returns: Chat ID

loadChat

function

loadChat(chatId: string): Promise<ChatRecord>

Load a chat with all data and media.Returns: Chat record with messages and metadata

listChats

function

listChats(
  offset?: number,
  limit?: number
): Promise<ChatRecord[]>

List saved chats with pagination.Parameters:

offset: Skip first N chats (default: 0)
limit: Max chats to return (default: 20)

Returns: Array of chat records (sorted by updatedAt descending)

deleteChat

function

deleteChat(chatId: string): Promise<void>

Delete a chat and all associated media.

updateChatTitle

function

updateChatTitle(chatId: string, newTitle: string): Promise<void>

Update the title of a saved chat.

searchChats

function

searchChats(query: string): Promise<ChatRecord[]>

Search chats by title or content.

Usage Examples

AI Chat with Streaming

import AIServiceProxy from './services/proxies/AIServiceProxy';

// Configure
await AIServiceProxy.configure({
  provider: 'chrome-ai',
  chromeAi: {
    temperature: 1.0,
    topK: 3,
    enableImageSupport: true,
  },
});

// Chat with streaming
const messages = [
  { role: 'user', content: 'Hello!' },
];

let fullResponse = '';
for await (const chunk of AIServiceProxy.chat(messages)) {
  fullResponse += chunk;
  console.log(chunk);
}

Text-to-Speech with Kokoro

import { TTSServiceProxy } from './services/proxies';

// Configure
TTSServiceProxy.configure({
  enabled: true,
  provider: 'kokoro',
  kokoro: {
    modelId: 'onnx-community/Kokoro-82M-v1.0-ONNX',
    voice: 'af_heart',
    speed: 1.0,
    device: 'auto',
  },
});

// Generate and play
const audioUrls = await TTSServiceProxy.generateChunkedSpeech(
  "Hello, how can I help you?",
  null,
  500,
  100,
  'session_1'
);

await TTSServiceProxy.playAudioSequence(audioUrls, 'session_1');

Speech-to-Text Recording

import { STTServiceProxy } from './services/proxies';

// Set up callbacks
STTServiceProxy.setTranscriptionCallback((text) => {
  console.log('Transcription:', text);
});

STTServiceProxy.setErrorCallback((error) => {
  console.error('STT Error:', error);
});

// Start recording
await STTServiceProxy.startRecording();

// Stop when ready
await STTServiceProxy.stopRecording();

Translation with Streaming

import { TranslatorServiceProxy } from './services/proxies';

// Configure
await TranslatorServiceProxy.configure({
  provider: 'chrome-ai',
});

// Translate with streaming
let translation = '';
for await (const chunk of TranslatorServiceProxy.translateStreaming(
  'Hello, world!',
  'en',
  'es'
)) {
  translation += chunk;
  console.log(chunk);
}
// Result: "¡Hola, mundo!"

Contributing

API Reference

Services Reference

Services

AIService

Methods

Supported Providers

TTSService

Methods

Supported Providers

STTService

Methods

TranslatorService

Methods

SummarizerService

Methods

RewriterService

Methods

WriterService

Methods

ChatHistoryService

Methods

Usage Examples

Build docs developers (and LLMs) love

Contributing

API Reference

​Services

​AIService

​Methods

​Supported Providers

​TTSService

​Methods

​Supported Providers

​STTService

​Methods

​TranslatorService

​Methods

​SummarizerService

​Methods

​RewriterService

​Methods

​WriterService

​Methods

​ChatHistoryService

​Methods

​Usage Examples

Build docs developers (and LLMs) love

Services

AIService

Methods

Supported Providers

TTSService

Methods

Supported Providers

STTService

Methods

TranslatorService

Methods

SummarizerService

Methods

RewriterService

Methods

WriterService

Methods

ChatHistoryService

Methods

Usage Examples