Skip to main content

​
Overview

The createNavaiMobileAgentRuntime function creates an agent runtime that handles tool execution for mobile voice agents. It manages navigation, frontend function execution, and backend function calls.

​
Usage

import { createNavaiMobileAgentRuntime } from '@navai/voice-mobile';

const agentRuntime = createNavaiMobileAgentRuntime({
  navigate: (path) => navigation.navigate(path),
  routes: [
    { name: 'home', path: '/' },
    { name: 'profile', path: '/profile' },
    { name: 'settings', path: '/settings' }
  ],
  functionsRegistry,
  backendFunctions: response.backendFunctions,
  executeBackendFunction: session.executeBackendFunction
});

// Use the runtime session configuration
await session.sendRealtimeEvent({
  type: 'session.update',
  session: {
    type: 'realtime',
    instructions: agentRuntime.session.instructions,
    tools: agentRuntime.session.tools,
    tool_choice: 'auto'
  }
});

// Execute tool calls
const result = await agentRuntime.executeToolCall({
  name: 'navigate_to',
  payload: { target: 'profile' }
});

​
Options

​
navigate
(path: string) => void
required
Navigation callback function invoked when the navigate_to tool is executed.
navigate: (path) => navigation.navigate(path)
​
routes
NavaiRoute[]
required
Array of allowed route definitions. Each route has name and path properties.
routes: [
  { name: 'home', path: '/' },
  { name: 'perfil', path: '/profile' },
  { name: 'ajustes', path: '/settings' }
]
​
functionsRegistry
NavaiFunctionsRegistry
required
Registry of frontend functions loaded from function modules. Obtain from loadNavaiFunctions.
​
backendFunctions
NavaiBackendFunctionDefinition[]
Array of backend function definitions. Each definition includes name, description, and source fields.
backendFunctions: [
  { name: 'get_weather', description: 'Get current weather' },
  { name: 'send_email', description: 'Send an email' }
]
​
executeBackendFunction
(input: ExecuteNavaiBackendFunctionInput) => Promise<unknown>
Function to execute backend functions. Typically bound to session.executeBackendFunction.
executeBackendFunction: session.executeBackendFunction
​
baseInstructions
string
Custom base instructions for the agent. Defaults to:
"You are a voice assistant embedded in a mobile app."
Additional instructions about routes, functions, and tool usage are automatically appended.

​
Return Value

Returns NavaiMobileAgentRuntime object:
​
session
NavaiMobileAgentRuntimeSession
Session configuration for the real-time API.
​
warnings
string[]
Array of warning messages from function loading and backend function processing.
​
availableFunctionNames
string[]
Combined list of frontend and backend function names available for execution.
​
executeToolCall
(input: NavaiMobileToolCallInput) => Promise<unknown>
Function to execute a tool call.

​
Built-in Tools

The runtime provides two built-in tools: Navigates to an allowed route. 
// Tool definition
{
  type: "function",
  name: "navigate_to",
  description: "Navigate to an allowed route in the current app.",
  parameters: {
    type: "object",
    properties: {
      target: {
        type: "string",
        description: "Route name or route path. Example: perfil, ajustes, /profile, /settings"
      }
    },
    required: ["target"]
  }
}

// Usage
await runtime.executeToolCall({
  name: 'navigate_to',
  payload: { target: 'profile' }
});
// Result: { ok: true, path: '/profile' }

​
execute_app_function

Executes a frontend or backend function. 
// Tool definition
{
  type: "function",
  name: "execute_app_function",
  description: "Execute an allowed internal app function by name.",
  parameters: {
    type: "object",
    properties: {
      function_name: {
        type: "string",
        description: "Allowed function name from the list."
      },
      payload: {
        anyOf: [
          { type: "object", additionalProperties: true },
          { type: "null" }
        ],
        description: "Payload object. Use null when no arguments are needed."
      }
    },
    required: ["function_name"]
  }
}

// Usage
await runtime.executeToolCall({
  name: 'execute_app_function',
  payload: {
    function_name: 'get_weather',
    payload: { location: 'San Francisco' }
  }
});
// Result: { ok: true, function_name: 'get_weather', source: 'frontend', result: {...} }

​
Examples

​
Complete Setup

import {
  createNavaiMobileAgentRuntime,
  loadNavaiFunctions,
  createNavaiMobileVoiceSession
} from '@navai/voice-mobile';

// Load functions
const functionsRegistry = await loadNavaiFunctions(functionModuleLoaders);

// Start session
const session = createNavaiMobileVoiceSession({ transport, backendClient });
const response = await session.start();

// Create agent runtime
const agentRuntime = createNavaiMobileAgentRuntime({
  navigate: (path) => navigation.navigate(path),
  routes: [
    { name: 'home', path: '/' },
    { name: 'profile', path: '/profile' }
  ],
  functionsRegistry,
  backendFunctions: response.backendFunctions,
  executeBackendFunction: session.executeBackendFunction
});

// Configure session
await session.sendRealtimeEvent({
  type: 'session.update',
  session: {
    type: 'realtime',
    instructions: agentRuntime.session.instructions,
    tools: agentRuntime.session.tools,
    tool_choice: 'auto'
  }
});

​
Custom Instructions

const agentRuntime = createNavaiMobileAgentRuntime({
  navigate,
  routes,
  functionsRegistry,
  backendFunctions,
  executeBackendFunction,
  baseInstructions: `You are a helpful fitness assistant in a mobile workout app.
You can help users navigate to different workout sections and track their progress.
Be encouraging and motivational in your responses.`
});

​
Handling Tool Calls from Real-time Events

import { extractNavaiRealtimeToolCalls, buildNavaiRealtimeToolResultEvents } from '@navai/voice-mobile';

session.onRealtimeEvent = async (event) => {
  const toolCalls = extractNavaiRealtimeToolCalls(event);
  
  for (const toolCall of toolCalls) {
    try {
      const result = await agentRuntime.executeToolCall({
        name: toolCall.name,
        payload: toolCall.payload
      });
      
      const events = buildNavaiRealtimeToolResultEvents({
        callId: toolCall.callId,
        output: result
      });
      
      for (const evt of events) {
        await session.sendRealtimeEvent(evt);
      }
    } catch (error) {
      const events = buildNavaiRealtimeToolResultEvents({
        callId: toolCall.callId,
        output: { ok: false, error: error.message }
      });
      
      for (const evt of events) {
        await session.sendRealtimeEvent(evt);
      }
    }
  }
};

​
Tool Execution Results

// Input
{ name: 'navigate_to', payload: { target: 'profile' } }

// Result
{ ok: true, path: '/profile' }

// Input
{ name: 'navigate_to', payload: { target: 'unknown' } }

// Result
{ ok: false, error: 'Unknown or disallowed route.' }

​
Function Success

// Input
{ name: 'execute_app_function', payload: { function_name: 'get_weather', payload: { city: 'SF' } } }

// Result
{
  ok: true,
  function_name: 'get_weather',
  source: 'frontend',
  result: { temperature: 68, condition: 'sunny' }
}

​
Function Error

// Input
{ name: 'execute_app_function', payload: { function_name: 'unknown_fn' } }

// Result
{
  ok: false,
  error: 'Unknown or disallowed function.',
  available_functions: ['get_weather', 'send_message', ...]
}

Build docs developers (and LLMs) love

Get started for freeTalk to us