Skip to main content

​
Overview

createReactNativeWebRtcTransport creates a WebRTC transport for connecting to OpenAI’s Realtime API from React Native applications. It handles peer connection setup, audio streaming, and data channel management.

​
Import

import { createReactNativeWebRtcTransport } from '@navai/voice-mobile';

​
Usage

import { createReactNativeWebRtcTransport } from '@navai/voice-mobile';
import { mediaDevices, RTCPeerConnection } from 'react-native-webrtc';

const transport = createReactNativeWebRtcTransport({
  globals: {
    mediaDevices,
    RTCPeerConnection
  },
  model: 'gpt-4o-realtime-preview-2024-12-17',
  remoteAudioTrackVolume: 8
});

// Connect to realtime API
await transport.connect({
  clientSecret: 'client_secret_...',
  onEvent: (event) => console.log('Event:', event),
  onError: (error) => console.error('Error:', error)
});

// Send events
await transport.sendEvent({
  type: 'session.update',
  session: { instructions: 'You are a helpful assistant' }
});

// Disconnect
await transport.disconnect();

​
Type Signature

function createReactNativeWebRtcTransport(
  options: CreateReactNativeWebRtcTransportOptions
): NavaiRealtimeTransport

​
Parameters

​
options
CreateReactNativeWebRtcTransportOptions
required
Configuration options for the WebRTC transport

​
Return Value

​
connect
(options: NavaiRealtimeTransportConnectOptions) => Promise<void>
Connect to OpenAI Realtime API via WebRTC.Throws if already connecting/connected, or if WebRTC setup fails.
​
disconnect
() => Promise<void>
Disconnect and clean up all resources. Safe to call multiple times.
​
sendEvent
(event: unknown) => Promise<void>
Send an event to the realtime API.Accepts object (auto-serializes to JSON) or string.Throws if data channel is not open.
​
getState
() => NavaiRealtimeTransportState
Get current transport state:
  • idle - Not connected
  • connecting - Connection in progress
  • connected - Active connection
  • error - Connection error
  • closed - Disconnected

​
Connection Flow

​
WebRTC Configuration

​
Audio Constraints

Default audio constraints: 
{
  audio: true,
  video: false
}
Custom constraints: 
const transport = createReactNativeWebRtcTransport({
  globals,
  audioConstraints: {
    audio: {
      echoCancellation: true,
      noiseSuppression: true,
      autoGainControl: true
    },
    video: false
  }
});

​
RTC Configuration

Custom RTC configuration (STUN/TURN servers): 
const transport = createReactNativeWebRtcTransport({
  globals,
  rtcConfiguration: {
    iceServers: [
      { urls: 'stun:stun.l.google.com:19302' },
      {
        urls: 'turn:turn.example.com:3478',
        username: 'user',
        credential: 'pass'
      }
    ]
  }
});

​
Remote Audio Volume

Control remote audio volume (0-10): 
const transport = createReactNativeWebRtcTransport({
  globals,
  remoteAudioTrackVolume: 8 // 80% volume
});
Volume is applied to:
  • Audio tracks from ontrack events
  • All receivers’ tracks

​
Examples

​
Basic Usage

import { createReactNativeWebRtcTransport } from '@navai/voice-mobile';
import { mediaDevices, RTCPeerConnection } from 'react-native-webrtc';

const transport = createReactNativeWebRtcTransport({
  globals: { mediaDevices, RTCPeerConnection }
});

await transport.connect({
  clientSecret,
  onEvent: (event) => {
    if (event.type === 'response.audio_transcript.delta') {
      console.log('Transcript:', event.delta);
    }
  },
  onError: (error) => {
    console.error('Transport error:', error);
  }
});

await transport.sendEvent({
  type: 'session.update',
  session: {
    instructions: 'You are a helpful assistant.'
  }
});

​
With State Tracking

const transport = createReactNativeWebRtcTransport({ globals });
const [state, setState] = useState('idle');

const connect = async () => {
  try {
    setState('connecting');
    await transport.connect({
      clientSecret,
      onEvent: handleEvent,
      onError: handleError
    });
    setState(transport.getState());
  } catch (error) {
    setState('error');
  }
};

​
Custom Model and URL

const transport = createReactNativeWebRtcTransport({
  globals,
  model: 'gpt-4o-realtime-preview-2024-12-17',
  realtimeUrl: 'https://api.openai.com/v1/realtime/calls'
});

await transport.connect({
  clientSecret,
  model: 'gpt-4o-mini-realtime-preview' // Override default
});

​
Event Processing

await transport.connect({
  clientSecret,
  onEvent: (event) => {
    // Events are auto-parsed from JSON
    switch (event.type) {
      case 'session.created':
        console.log('Session created');
        break;
      case 'conversation.item.created':
        console.log('Item created:', event.item);
        break;
      case 'error':
        console.error('Realtime error:', event.error);
        break;
    }
  },
  onError: (error) => {
    // Connection-level errors
    console.error('Transport error:', error);
  }
});

​
Clean Disconnect

useEffect(() => {
  return () => {
    // Clean up on unmount
    transport.disconnect();
  };
}, []);

const handleStop = async () => {
  try {
    await transport.disconnect();
    console.log('Disconnected');
  } catch (error) {
    console.error('Disconnect error:', error);
  }
};

​
Implementation Details

​
Data Channel

The transport creates a data channel named "oai-events" for bidirectional communication:
  • Incoming: Events are auto-parsed from JSON strings
  • Outgoing: Objects are auto-serialized to JSON
  • Timeout: 12 seconds for initial connection, 6 seconds for send operations

​
Connection State Monitoring

The transport monitors WebRTC connection state: 
peerConnection.onconnectionstatechange = () => {
  if (state === 'failed' || state === 'disconnected') {
    onError(new Error(`WebRTC connection state: ${state}`));
  }
};

​
Audio Track Management

Local audio tracks:
  • Obtained via getUserMedia()
  • Added to peer connection
  • Stopped on disconnect
Remote audio tracks:
  • Received via ontrack events
  • Volume applied automatically
  • Extracted from receivers

​
Error Handling

Errors are normalized to readable messages: 
function toErrorMessage(error: unknown): string {
  if (error instanceof Error) return error.message;
  if (typeof error === 'string') return error;
  if (error?.code && error?.message) return `${code}: ${message}`;
  // ... more cases
}

​
Timeouts

OperationTimeoutConfigurable
Data channel connect12 secondsNo
Send event6 secondsNo
WebRTC negotiationNone (fetch default)Via fetchImpl

​
Platform Considerations

​
iOS

  • Requires microphone permissions in Info.plist: 
    <key>NSMicrophoneUsageDescription</key>
<string>Voice assistant needs microphone access</string>

​
Android

  • Requires RECORD_AUDIO permission in AndroidManifest.xml: 
    <uses-permission android:name="android.permission.RECORD_AUDIO" />
  • Runtime permission handled by useMobileVoiceAgent hook

​
Best Practices

  1. Error Handling: Always provide both onEvent and onError handlers
  2. Cleanup: Call disconnect() in component cleanup (useEffect return)
  3. Volume: Start with lower volume (6-8) and adjust based on user feedback
  4. State: Use getState() to check connection status before sending events
  5. Retries: Implement exponential backoff for reconnection logic

​
Troubleshooting

IssuePossible CauseSolution
”WebRTC native module is not available”react-native-webrtc not linkedRun pod install / gradle sync
”No audio tracks returned”Microphone permission deniedCheck permissions
”Data channel is not open”Sent event before connection completeWait for connected state
”WebRTC connection state: failed”Network issueCheck network connectivity
No audio outputVolume too low or mutedIncrease remoteAudioTrackVolume

​
See Also

Build docs developers (and LLMs) love

Get started for freeTalk to us