Establishes a WebRTC connection with a real-time model and returns a RealTimeClient instance for controlling the stream.
Method Signature connect ( stream : MediaStream | null , options : RealTimeClientConnectOptions ): Promise < RealTimeClient >
Parameters stream
MediaStream | null
required
The local media stream to send to the server. Pass null for models that don’t require input (like live_avatar without microphone input). For camera/microphone input, use navigator.mediaDevices.getUserMedia().
options
RealTimeClientConnectOptions
required
Connection configuration options. model
ModelDefinition | CustomModelDefinition
required
The real-time model to use. Access models via models.realtime() factory functions:
models.realtime.mirage()models.realtime.mirageV2()models.realtime.lucyV2V720pRt()models.realtime.lucy2Rt() onRemoteStream
(stream: MediaStream) => void
required
Callback invoked when the remote video/audio stream is available. Attach this stream to a video element to display the output. onRemoteStream : ( stream ) => { videoElement . srcObject = stream ; }
initialState
RealTimeClientInitialState
Initial state to set on connection. Initial prompt to apply. Whether to enhance the prompt using AI.
Initial reference image. Can be a Blob, File, data URL, or HTTP(S) URL.
customizeOffer
(offer: RTCSessionDescriptionInit) => Promise<void>
Advanced: Callback to customize the WebRTC offer before sending it to the server.
Return Value A promise that resolves to a RealTimeClient instance with the following methods and properties: set
(input: SetInput) => Promise<void>
Update prompt and/or reference image. See set() . setPrompt
(prompt: string, options?: { enhance?: boolean }) => Promise<void>
Returns true if the connection is active.
Returns the current connection state: "connecting", "connected", "generating", "disconnected", or "reconnecting".
Close the connection and clean up resources. See disconnect() . on
<K extends keyof Events>(event: K, listener: (data: Events[K]) => void) => void
Subscribe to events. See Events . off
<K extends keyof Events>(event: K, listener: (data: Events[K]) => void) => void
Unsubscribe from events. See Events . The unique session ID, available after connection. Use this for session sharing.
Token for subscribing to this session from another client.
setImage
(image: Blob | File | string | null, options?: { prompt?: string; enhance?: boolean; timeout?: number }) => Promise<void>
Update the reference image. Pass null to clear the image.
playAudio
(audio: Blob | File | ArrayBuffer) => Promise<void>
Only available for live_avatar model : Play audio through the avatar. This method is only present when using the live_avatar model without a user-provided stream.
Error Handling The method throws an error if:
Options validation fails (invalid parameters)
WebRTC connection fails
WebSocket connection fails
Initial image/prompt fails to send
try { const client = await decart . realtime . connect ( stream , options ); } catch ( error ) { if ( error instanceof DecartSDKError ) { console . error ( 'Connection failed:' , error . code , error . message ); } }
Usage Examples Basic Connection import { createDecartClient , models } from '@decart/sdk' ; const decart = createDecartClient ({ apiKey: 'your-api-key' }); // Get user's camera const stream = await navigator . mediaDevices . getUserMedia ({ video: true , audio: false }); // Connect to Mirage model const client = await decart . realtime . connect ( stream , { model: models . realtime . mirage (), onRemoteStream : ( remoteStream ) => { const video = document . getElementById ( 'output' ); video . srcObject = remoteStream ; } }); console . log ( 'Connected! Session ID:' , client . sessionId );
Connection with Initial State const client = await decart . realtime . connect ( stream , { model: models . realtime . lucyV2V720pRt (), onRemoteStream : ( remoteStream ) => { videoElement . srcObject = remoteStream ; }, initialState: { prompt: { text: 'Transform into a watercolor painting' , enhance: true } } });
// No input stream required for live_avatar const client = await decart . realtime . connect ( null , { model: models . realtime . liveAvatar (), onRemoteStream : ( remoteStream ) => { videoElement . srcObject = remoteStream ; }, initialState: { image: 'https://example.com/avatar.jpg' , prompt: { text: 'Friendly greeting expression' } } }); // Play audio through the avatar await client . playAudio ( audioBlob );
Connection with Event Listeners const client = await decart . realtime . connect ( stream , { model: models . realtime . mirage (), onRemoteStream : ( remoteStream ) => { videoElement . srcObject = remoteStream ; } }); // Listen for connection state changes client . on ( 'connectionChange' , ( state ) => { console . log ( 'Connection state:' , state ); }); // Listen for errors client . on ( 'error' , ( error ) => { console . error ( 'Error:' , error . message ); }); // Listen for generation ticks client . on ( 'generationTick' , ({ seconds }) => { console . log ( 'Generating for' , seconds , 'seconds' ); });
See Also