Skip to main content
NAVAI provides built-in support for multiple languages through OpenAI’s Realtime API. Configure your agent to speak in different languages with appropriate accents and tones.

​
Language Configuration

Set the default language for your AI agent using environment variables:
.env
OPENAI_REALTIME_LANGUAGE=Spanish
When a language is set, NAVAI automatically injects “Always reply in .” into the session instructions.

​
How Language Instructions Work

NAVAI uses the buildSessionInstructions function to combine language settings with other instructions: 
// From packages/voice-backend/src/index.ts:134-158
function buildSessionInstructions(input: {
  baseInstructions: string;
  language?: string;
  voiceAccent?: string;
  voiceTone?: string;
}): string {
  const lines = [input.baseInstructions.trim()];
  
  if (language) {
    lines.push(`Always reply in ${language}.`);
  }
  
  if (voiceAccent) {
    lines.push(`Use a ${voiceAccent} accent while speaking.`);
  }
  
  if (voiceTone) {
    lines.push(`Use a ${voiceTone} tone while speaking.`);
  }
  
  return lines.join("\n");
}

​
Common Language Configurations

.env
OPENAI_REALTIME_LANGUAGE=Spanish
OPENAI_REALTIME_VOICE_ACCENT=neutral Latin American Spanish
OPENAI_REALTIME_VOICE_TONE=friendly and professional
OPENAI_REALTIME_VOICE=marin
Perfect for Latin American markets.
.env
OPENAI_REALTIME_LANGUAGE=Spanish
OPENAI_REALTIME_VOICE_ACCENT=European Spanish
OPENAI_REALTIME_VOICE_TONE=professional
OPENAI_REALTIME_VOICE=sage
Ideal for Spain and European Spanish speakers.
.env
OPENAI_REALTIME_LANGUAGE=French
OPENAI_REALTIME_VOICE_ACCENT=Parisian French
OPENAI_REALTIME_VOICE_TONE=elegant and polite
OPENAI_REALTIME_VOICE=coral
.env
OPENAI_REALTIME_LANGUAGE=German
OPENAI_REALTIME_VOICE_ACCENT=High German
OPENAI_REALTIME_VOICE_TONE=clear and direct
OPENAI_REALTIME_VOICE=alloy
.env
OPENAI_REALTIME_LANGUAGE=Japanese
OPENAI_REALTIME_VOICE_ACCENT=Tokyo dialect
OPENAI_REALTIME_VOICE_TONE=polite and respectful
OPENAI_REALTIME_VOICE=shimmer
.env
OPENAI_REALTIME_LANGUAGE=Portuguese
OPENAI_REALTIME_VOICE_ACCENT=Brazilian Portuguese
OPENAI_REALTIME_VOICE_TONE=warm and friendly
OPENAI_REALTIME_VOICE=verse

​
Backend Configuration

1

Set environment variables

Configure language settings in your .env file:
.env
OPENAI_API_KEY=sk-...
OPENAI_REALTIME_LANGUAGE=Spanish
OPENAI_REALTIME_VOICE_ACCENT=neutral Latin American Spanish
OPENAI_REALTIME_VOICE_TONE=friendly and professional
2

Load configuration

NAVAI automatically loads these settings:
server.ts
import { registerNavaiExpressRoutes } from "@navai/voice-backend";

// Configuration is loaded from environment variables
registerNavaiExpressRoutes(app);
3

Verify the configuration

Start your backend and test the voice agent:
npm run dev

​
Dynamic Language Selection

Allow users to choose their preferred language at runtime: 
// Client-side: Request a specific language session
const response = await fetch('/navai/realtime/client-secret', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    language: 'French',
    voiceAccent: 'Parisian French',
    voiceTone: 'elegant and polite',
    voice: 'coral'
  })
});

const { value } = await response.json();

​
Language-Specific Instructions

Customize instructions for different languages:
server.ts
import { createRealtimeClientSecret } from "@navai/voice-backend";

const spanishInstructions = `Eres un asistente de navegación por voz.
Ayuda a los usuarios a navegar por la aplicación de manera eficiente.
Sé conciso y amigable.`;

const clientSecret = await createRealtimeClientSecret(
  {
    openaiApiKey: process.env.OPENAI_API_KEY,
    defaultLanguage: "Spanish",
    defaultVoiceAccent: "neutral Latin American Spanish"
  },
  {
    instructions: spanishInstructions
  }
);
While you can provide instructions in the target language, the system will still append the English instruction hints for language, accent, and tone. The AI model is capable of understanding these mixed-language instructions.

​
Multi-Language Application Example

Implement language switching in your application: 
type LanguageConfig = {
  language: string;
  accent: string;
  tone: string;
  voice: string;
  instructions: string;
};

const languageConfigs: Record<string, LanguageConfig> = {
  en: {
    language: "English",
    accent: "neutral American English",
    tone: "friendly and professional",
    voice: "marin",
    instructions: "You are a helpful voice navigation assistant."
  },
  es: {
    language: "Spanish",
    accent: "neutral Latin American Spanish",
    tone: "amigable y profesional",
    voice: "coral",
    instructions: "Eres un asistente de navegación por voz útil."
  },
  fr: {
    language: "French",
    accent: "Parisian French",
    tone: "élégant et poli",
    voice: "sage",
    instructions: "Vous êtes un assistant de navigation vocale utile."
  }
};

async function createLanguageSession(languageCode: string) {
  const config = languageConfigs[languageCode];
  
  const response = await fetch('/navai/realtime/client-secret', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify(config)
  });
  
  return await response.json();
}

​
Environment Variable Reference

Complete backend environment configuration for multilingual support:
.env
# API Configuration
OPENAI_API_KEY=sk-...
OPENAI_REALTIME_MODEL=gpt-realtime

# Language Settings
OPENAI_REALTIME_LANGUAGE=Spanish
OPENAI_REALTIME_VOICE_ACCENT=neutral Latin American Spanish
OPENAI_REALTIME_VOICE_TONE=friendly and professional

# Voice Selection
OPENAI_REALTIME_VOICE=marin

# Base Instructions
OPENAI_REALTIME_INSTRUCTIONS=You are a helpful assistant.

# Session Configuration
OPENAI_REALTIME_CLIENT_SECRET_TTL=600

​
Frontend Configuration

For web applications:
.env
NAVAI_API_URL=http://localhost:3000
NAVAI_REALTIME_MODEL=gpt-realtime
For mobile applications:
.env
NAVAI_API_URL=http://192.168.1.100:3000
NAVAI_REALTIME_MODEL=gpt-realtime

​
Testing Different Languages

1

Create test configurations

Set up different .env files for each language you want to test:
  • .env.spanish
  • .env.french
  • .env.german
2

Switch configurations

cp .env.spanish .env
npm run dev
3

Test the voice agent

Start a voice session and verify the language, accent, and tone.

​
Common Issues

Language Mixing: If the agent switches between languages unexpectedly, ensure your instructions clearly specify the language and avoid mixed-language prompts from users.
Accent Accuracy: The accent setting is a hint to the AI model. While it generally works well, perfect accent replication is not guaranteed and depends on the model’s capabilities.

​
Best Practices

  1. Be Specific with Accents: Use detailed accent descriptions like “neutral Latin American Spanish” instead of just “Spanish accent”
  2. Match Tone to Culture: Different cultures have different communication styles. Adjust tone accordingly.
  3. Test with Native Speakers: Have native speakers test the voice quality and naturalness.
  4. Provide Fallback: Always have a default language configuration if dynamic selection fails.
  5. Document Supported Languages: Clearly communicate which languages your application supports.

​
Complete Example: Spanish Support

Here’s a complete working example for Spanish language support:
.env
# Backend Configuration
OPENAI_API_KEY=sk-...
OPENAI_REALTIME_MODEL=gpt-realtime
OPENAI_REALTIME_VOICE=marin
OPENAI_REALTIME_INSTRUCTIONS=You are a helpful assistant.
OPENAI_REALTIME_LANGUAGE=Spanish
OPENAI_REALTIME_VOICE_ACCENT=neutral Latin American Spanish
OPENAI_REALTIME_VOICE_TONE=friendly and professional
OPENAI_REALTIME_CLIENT_SECRET_TTL=600
NAVAI_FUNCTIONS_FOLDERS=src/ai/...
NAVAI_CORS_ORIGIN=http://localhost:5173
NAVAI_ALLOW_FRONTEND_API_KEY=false
PORT=3000
This configuration creates a Spanish-speaking voice agent with a neutral Latin American accent and a friendly, professional tone.

​
Next Steps

Voice Customization

Learn more about voice and tone customization

Debugging

Troubleshoot common issues

Build docs developers (and LLMs) love

Get started for freeTalk to us