Skip to main content

Overview

This guide shows you how to integrate the NAVAI voice agent into a React Native or Expo application using @navai/voice-mobile.

What You’ll Build

A voice-controlled mobile app that allows users to:
  • Navigate between screens using voice commands
  • Execute app functions through voice
  • Access backend functions via the agent

Prerequisites

1

Install dependencies

npm install @navai/voice-mobile @openai/agents zod react-native
For Expo projects:
npx expo install @navai/voice-mobile @openai/agents zod
react-native is a peer dependency. This package works with React Native 0.70+ and Expo SDK 49+.
2

Configure environment variables

For Expo, add to app.config.js or app.json:
app.config.js
export default {
  expo: {
    // ... other config
    extra: {
      NAVAI_API_URL: process.env.NAVAI_API_URL || "http://192.168.1.100:3000",
      NAVAI_FUNCTIONS_FOLDERS: "src/ai/functions-modules",
      NAVAI_ROUTES_FILE: "src/ai/routes.ts"
    }
  }
};
Create .env in your mobile app root:
# Required: Backend API URL (use your LAN IP for physical devices)
NAVAI_API_URL=http://192.168.1.100:3000

# Optional: Functions folder
NAVAI_FUNCTIONS_FOLDERS=src/ai/functions-modules

# Optional: Routes file
NAVAI_ROUTES_FILE=src/ai/routes.ts
For physical devices and emulators, use your machine’s LAN IP address, not localhost. Find your IP:
  • macOS/Linux: ifconfig | grep inet
  • Windows: ipconfig
3

Run the setup script

npx navai-setup-voice-mobile
This adds to your package.json:
{
  "scripts": {
    "generate:ai-modules": "navai-generate-mobile-loaders",
    "prestart": "npm run generate:ai-modules"
  }
}

Define Routes

Create a routes configuration file:
src/ai/routes.ts
import type { NavaiRoute } from "@navai/voice-mobile";

export const NAVAI_ROUTE_ITEMS: NavaiRoute[] = [
  {
    name: "inicio",
    path: "/",
    description: "Pantalla principal del playground mobile",
    synonyms: ["home", "principal", "start"]
  },
  {
    name: "perfil",
    path: "/profile",
    description: "Pantalla de perfil",
    synonyms: ["profile", "mi perfil", "account"]
  },
  {
    name: "ajustes",
    path: "/settings",
    description: "Pantalla de configuracion",
    synonyms: ["settings", "configuracion", "config"]
  },
  {
    name: "ayuda",
    path: "/help",
    description: "Pantalla de ayuda y soporte",
    synonyms: ["help", "soporte", "support"]
  }
];

Resolve Runtime Configuration

Create a function to read environment variables and resolve runtime config:
import Constants from "expo-constants";
import {
  resolveNavaiMobileApplicationRuntimeConfig,
  resolveNavaiMobileEnv,
  type ResolveNavaiMobileApplicationRuntimeConfigResult
} from "@navai/voice-mobile";
import { NAVAI_MOBILE_MODULE_LOADERS } from "./src/ai/generated-module-loaders";
import { NAVAI_ROUTE_ITEMS } from "./src/ai/routes";

function readPlaygroundMobileEnv() {
  const extra = (Constants.expoConfig?.extra ?? {}) as Record<string, unknown>;
  return resolveNavaiMobileEnv({
    sources: [extra, process.env as Record<string, unknown>]
  });
}

async function resolvePlaygroundMobileRuntimeConfig(): Promise<ResolveNavaiMobileApplicationRuntimeConfigResult> {
  const env = readPlaygroundMobileEnv();
  const apiBaseUrl = env.NAVAI_API_URL?.trim();

  if (!apiBaseUrl) {
    throw new Error(
      "[navai] NAVAI_API_URL is required. Example: NAVAI_API_URL=http://192.168.1.100:3000"
    );
  }

  return resolveNavaiMobileApplicationRuntimeConfig({
    moduleLoaders: NAVAI_MOBILE_MODULE_LOADERS,
    defaultRoutes: NAVAI_ROUTE_ITEMS,
    env,
    apiBaseUrl,
    emptyModuleLoadersWarning:
      "[navai] No generated module loaders were found. Run `npm run generate:ai-modules`."
  });
}
This example is from apps/playground-mobile/App.tsx:22-47 in the NAVAI source.

Create the Voice Navigator Component

Use the useMobileVoiceAgent hook:
src/voice/VoiceNavigator.tsx
import {
  useMobileVoiceAgent,
  type ResolveNavaiMobileApplicationRuntimeConfigResult
} from "@navai/voice-mobile";
import { ActivityIndicator, Pressable, StyleSheet, Text, View } from "react-native";

export type VoiceNavigatorProps = {
  activePath: string;
  runtime: ResolveNavaiMobileApplicationRuntimeConfigResult | null;
  runtimeLoading: boolean;
  runtimeError: string | null;
  navigate: (path: string) => void;
};

export function VoiceNavigator({
  activePath,
  runtime,
  runtimeLoading,
  runtimeError,
  navigate
}: VoiceNavigatorProps) {
  const agent = useMobileVoiceAgent({
    runtime,
    runtimeLoading,
    runtimeError,
    navigate
  });

  const canStart = !agent.isConnecting && !agent.isConnected;

  return (
    <View style={styles.card}>
      <Text style={styles.sectionTitle}>Voice Navigator</Text>
      <Text style={styles.status}>Ruta activa: {activePath}</Text>

      {runtimeLoading ? <Text style={styles.muted}>Loading runtime configuration...</Text> : null}
      {runtimeError ? <Text style={styles.error}>{runtimeError}</Text> : null}

      {!agent.isConnected ? (
        <Pressable style={styles.button} onPress={() => void agent.start()} disabled={!canStart}>
          {agent.isConnecting ? (
            <ActivityIndicator color="#111827" />
          ) : (
            <Text style={styles.buttonText}>Start Voice</Text>
          )}
        </Pressable>
      ) : (
        <Pressable style={[styles.button, styles.buttonStop]} onPress={() => void agent.stop()}>
          <Text style={styles.buttonText}>Stop Voice</Text>
        </Pressable>
      )}

      <Text style={styles.status}>Status: {agent.status}</Text>
      {agent.error ? <Text style={styles.error}>{agent.error}</Text> : null}
    </View>
  );
}

const styles = StyleSheet.create({
  card: {
    borderWidth: 1,
    borderColor: "#1E293B",
    backgroundColor: "#111827",
    borderRadius: 12,
    padding: 12,
    gap: 8
  },
  sectionTitle: {
    color: "#F8FAFC",
    fontWeight: "700"
  },
  status: {
    color: "#C4B5FD"
  },
  muted: {
    color: "#94A3B8"
  },
  error: {
    color: "#FCA5A5"
  },
  button: {
    marginTop: 4,
    borderRadius: 8,
    backgroundColor: "#67E8F9",
    alignItems: "center",
    justifyContent: "center",
    minHeight: 42
  },
  buttonStop: {
    backgroundColor: "#FCA5A5"
  },
  buttonText: {
    color: "#111827",
    fontWeight: "700"
  }
});

Integrate into Your App

Put it all together in your root component:
App.tsx
import Constants from "expo-constants";
import {
  resolveNavaiMobileApplicationRuntimeConfig,
  resolveNavaiMobileEnv,
  resolveNavaiRoute,
  type NavaiRoute,
  type ResolveNavaiMobileApplicationRuntimeConfigResult
} from "@navai/voice-mobile";
import { useCallback, useEffect, useMemo, useState, type ReactElement } from "react";
import { Image, Pressable, ScrollView, StyleSheet, Text, View } from "react-native";
import { NAVAI_MOBILE_MODULE_LOADERS } from "./src/ai/generated-module-loaders";
import { NAVAI_ROUTE_ITEMS } from "./src/ai/routes";
import { HomeScreen } from "./src/pages/HomeScreen";
import { ProfileScreen } from "./src/pages/ProfileScreen";
import { SettingsScreen } from "./src/pages/SettingsScreen";
import { HelpScreen } from "./src/pages/HelpScreen";
import { VoiceNavigator } from "./src/voice/VoiceNavigator";

function readPlaygroundMobileEnv() {
  const extra = (Constants.expoConfig?.extra ?? {}) as Record<string, unknown>;
  return resolveNavaiMobileEnv({
    sources: [extra, process.env as Record<string, unknown>]
  });
}

async function resolvePlaygroundMobileRuntimeConfig(): Promise<ResolveNavaiMobileApplicationRuntimeConfigResult> {
  const env = readPlaygroundMobileEnv();
  const apiBaseUrl = env.NAVAI_API_URL?.trim();

  if (!apiBaseUrl) {
    throw new Error(
      "[navai] NAVAI_API_URL is required. Example: NAVAI_API_URL=http://192.168.1.100:3000"
    );
  }

  return resolveNavaiMobileApplicationRuntimeConfig({
    moduleLoaders: NAVAI_MOBILE_MODULE_LOADERS,
    defaultRoutes: NAVAI_ROUTE_ITEMS,
    env,
    apiBaseUrl,
    emptyModuleLoadersWarning:
      "[navai] No generated module loaders were found. Run `npm run generate:ai-modules`."
  });
}

function renderScreen(path: string): ReactElement | null {
  switch (path) {
    case "/":
      return <HomeScreen />;
    case "/profile":
      return <ProfileScreen />;
    case "/settings":
      return <SettingsScreen />;
    case "/help":
      return <HelpScreen />;
    default:
      return null;
  }
}

export default function App() {
  const [runtime, setRuntime] = useState<ResolveNavaiMobileApplicationRuntimeConfigResult | null>(null);
  const [runtimeLoading, setRuntimeLoading] = useState(true);
  const [runtimeError, setRuntimeError] = useState<string | null>(null);
  const [activePath, setActivePath] = useState("/");

  const routes = useMemo<NavaiRoute[]>(
    () => (runtime?.routes.length ? runtime.routes : NAVAI_ROUTE_ITEMS),
    [runtime]
  );

  useEffect(() => {
    let cancelled = false;

    void resolvePlaygroundMobileRuntimeConfig()
      .then((result) => {
        if (cancelled) return;
        setRuntime(result);
        setRuntimeError(null);
      })
      .catch((nextError) => {
        if (cancelled) return;
        setRuntime(null);
        setRuntimeError(nextError instanceof Error ? nextError.message : String(nextError));
      })
      .finally(() => {
        if (!cancelled) setRuntimeLoading(false);
      });

    return () => {
      cancelled = true;
    };
  }, []);

  const navigate = useCallback(
    (input: string) => {
      const trimmed = input.trim();
      if (!trimmed) return;

      if (trimmed.startsWith("/")) {
        setActivePath(trimmed);
        return;
      }

      const nextPath =
        routes.find((route) => route.path === trimmed)?.path ??
        resolveNavaiRoute(trimmed, routes) ??
        null;

      if (nextPath) {
        setActivePath(nextPath);
      }
    },
    [routes]
  );

  const mainScreen = renderScreen(activePath);

  return (
    <View style={styles.root}>
      <ScrollView contentContainerStyle={styles.content}>
        <Image source={require("./assets/icon_navai.jpg")} style={styles.logo} />
        <Text style={styles.title}>Voice-first app navigation</Text>
        <Text style={styles.description}>
          Say: "llevame a perfil", "abre ajustes" or "cierra sesion". The agent can
          navigate routes and execute internal app functions through tools.
        </Text>

        <View style={styles.navRow}>
          {routes.map((route) => {
            const active = route.path === activePath;
            return (
              <Pressable
                key={route.path}
                style={[styles.routeChip, active ? styles.routeChipActive : null]}
                onPress={() => setActivePath(route.path)}
              >
                <Text style={active ? styles.routeChipTextActive : styles.routeChipText}>
                  {route.name}
                </Text>
              </Pressable>
            );
          })}
        </View>

        <VoiceNavigator
          activePath={activePath}
          runtime={runtime}
          runtimeLoading={runtimeLoading}
          runtimeError={runtimeError}
          navigate={navigate}
        />

        {mainScreen}
      </ScrollView>
    </View>
  );
}

const styles = StyleSheet.create({
  root: {
    flex: 1,
    backgroundColor: "#0B1220"
  },
  content: {
    padding: 16,
    gap: 12
  },
  logo: {
    width: 64,
    height: 64,
    borderRadius: 16
  },
  title: {
    color: "#FFFFFF",
    fontSize: 36,
    fontWeight: "700"
  },
  description: {
    color: "#CBD5E1",
    fontSize: 14
  },
  navRow: {
    flexDirection: "row",
    flexWrap: "wrap",
    gap: 8
  },
  routeChip: {
    borderWidth: 1,
    borderColor: "#334155",
    backgroundColor: "#0F172A",
    borderRadius: 999,
    paddingHorizontal: 10,
    paddingVertical: 6
  },
  routeChipActive: {
    borderColor: "#67E8F9",
    backgroundColor: "#164E63"
  },
  routeChipText: {
    color: "#CBD5E1",
    fontSize: 12
  },
  routeChipTextActive: {
    color: "#ECFEFF",
    fontSize: 12,
    fontWeight: "700"
  }
});
This complete app example is from apps/playground-mobile/App.tsx:1-258 in the NAVAI source.

How It Works

The mobile integration flow:
1

Read environment config

Read from Expo config and process.env using resolveNavaiMobileEnv
2

Resolve runtime configuration

Call resolveNavaiMobileApplicationRuntimeConfig with module loaders, routes, and env
3

Start voice agent

Pass runtime config to useMobileVoiceAgent hook
4

Navigate on voice commands

Agent calls your navigate function with resolved paths
The mobile example uses a custom navigation pattern (not React Navigation):
const navigate = useCallback(
  (input: string) => {
    const trimmed = input.trim();
    if (!trimmed) return;

    // Direct path
    if (trimmed.startsWith("/")) {
      setActivePath(trimmed);
      return;
    }

    // Resolve route by name or natural language
    const nextPath =
      routes.find((route) => route.path === trimmed)?.path ??
      resolveNavaiRoute(trimmed, routes) ??
      null;

    if (nextPath) {
      setActivePath(nextPath);
    }
  },
  [routes]
);
You can adapt this to work with React Navigation, Expo Router, or any other navigation library.

Generate Module Loaders

Run before starting your app:
npm run generate:ai-modules
This creates src/ai/generated-module-loaders.ts:
import type { NavaiFunctionModuleLoaders } from "@navai/voice-mobile";

export const NAVAI_MOBILE_MODULE_LOADERS: NavaiFunctionModuleLoaders = [
  {
    modulePath: "src/ai/functions-modules/session/logout.fn.ts",
    loader: () => import("../functions-modules/session/logout.fn")
  }
];

Troubleshooting

”NAVAI_API_URL is required”

Ensure .env and app.config.js contain your backend URL:
extra: {
  NAVAI_API_URL: "http://192.168.1.100:3000"
}

“Network request failed” or “Connection refused”

Check:
  1. Backend server is running
  2. You’re using your LAN IP, not localhost
  3. Mobile device/emulator is on the same network
  4. Firewall allows connections on backend port

”No generated module loaders were found”

Run:
npm run generate:ai-modules

Voice agent connects but doesn’t navigate

Verify:
  1. navigate function is passed to useMobileVoiceAgent
  2. Routes are defined in NAVAI_ROUTE_ITEMS
  3. Route paths match your screen rendering logic

Platform-Specific Considerations

iOS

Add microphone permission to Info.plist:
<key>NSMicrophoneUsageDescription</key>
<string>This app uses the microphone for voice navigation.</string>

Android

Add permissions to AndroidManifest.xml:
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.INTERNET" />

Next Steps

Build docs developers (and LLMs) love