Skip to main content

Components

React components for building VAssist’s 3D character interface, chat UI, and AI-powered toolbar.

VirtualAssistant

High-level wrapper component managing the 3D scene, animations, and character positioning.

Props

onReady
function
Callback when assistant is initialized. Receives object with animationManager, positionManager, and scene.
isPreview
boolean
default:false
Enable preview mode for setup wizard (renders inline instead of portal)
previewWidth
string
default:"100%"
Width for preview mode
previewHeight
string
default:"100%"
Height for preview mode
previewClassName
string
default:""
Additional CSS classes for preview container
portraitMode
boolean
default:false
Enable portrait mode in preview (clips model at waist)
previewPosition
string
default:"bottom-center"
Position preset for preview mode

Ref Methods

Access these methods via ref:
const assistantRef = useRef();
<VirtualAssistant ref={assistantRef} />
// Call methods:
assistantRef.current.speak(text, vmdBlobUrl, 'talking');
speak
function
speak(
  text: string,
  mouthAnimationBlobUrl: string,
  emotionCategory: string = 'talking',
  options?: { primaryWeight?: number, fillWeight?: number }
): Promise<void>
Make the assistant speak with lip sync and emotion-based body animation.Parameters:
  • text: Text to speak
  • mouthAnimationBlobUrl: Blob URL to BVMD file with lip-sync animation
  • emotionCategory: Animation category (‘talking’, ‘idle’, ‘thinking’, ‘celebrating’, ‘walking’)
  • options: Blending weights for composite animations
Example:
assistantRef.current.speak("Hello!", vmdBlobUrl, "talking");
idle
function
idle(): Promise<void>
Returns assistant to idle state.
setState
function
setState(stateOrEmotion: string): Promise<void>
Sets assistant state directly using state from AssistantState enum or emotion string.
triggerAction
function
triggerAction(action: string): Promise<void>
Triggers specific action: ‘think’, ‘walk’, ‘celebrate’, ‘speak’.
playComposite
function
playComposite(
  primaryAnimName: string,
  fillCategory: string = 'talking',
  options?: { primaryWeight?: number, fillWeight?: number }
): Promise<void>
Plays composite animation with stitched fill animations.
getState
function
getState(): string
Returns current assistant state.
isReady
function
isReady(): boolean
Checks if AnimationManager is initialized.
getAnimationManager
function
getAnimationManager(): AnimationManager | null
Gets direct access to AnimationManager instance.
setPosition
function
setPosition(preset: string): void
Sets model position using preset (‘bottom-right’, ‘bottom-left’, ‘center’, etc.).
getPositionManager
function
getPositionManager(): PositionManager | null
Gets direct access to PositionManager instance.
queueAnimation
function
queueAnimation(animationName: string, force?: boolean): void
Queues a simple animation to play after current animation finishes.
clearQueue
function
clearQueue(): void
Clears all queued animations.

AIToolbar

Floating context-aware toolbar for text and image operations. Appears on text selection, input focus, or image hover.

Features

  • Summarization: TL;DR, headline, key points, teaser
  • Translation: Auto-detect source language, translate to target
  • Image Analysis: Describe, extract text, analyze images
  • Text Rewriting: Grammar fix, tone adjustment, length changes
  • Content Writing: Generate new content from prompts
  • Voice Dictation: Speech-to-text for editable fields
  • Language Detection: Identify text language

Usage

AIToolbar is automatically rendered and managed by the application. It appears contextually based on user actions.
  • Text Selection: Shows summarize, translate, rewrite options
  • Input Focus: Shows dictation button
  • Image Hover: Shows image analysis options
  • Editable Content: Shows insert/replace options after processing

ChatContainer

Main chat interface managing messages, TTS playback, history, and settings.

Props

modelDisabled
boolean
default:false
Whether 3D model is disabled (chat-only mode)
onDragDrop
function
Callback to handle dropped content (images, audio, text)

Features

  • Message Display: Streaming text with smooth animation
  • TTS Playback: Text-to-speech for AI messages
  • Branch Navigation: Navigate between message variations
  • Chat History: Save/load conversations
  • Settings Panel: Configure AI, TTS, STT, UI
  • Drag & Drop: Attach images and audio files
  • Background Detection: Adaptive light/dark theme

ChatInput

Chat input component with voice recording, multimodal attachments, and drag-drop support.

Props

onSend
function
required
onSend(text: string, images: string[], audios: string[]): void
Callback when message is sent. Receives text, image data URLs, and audio data URLs.
onClose
function
Callback when input is closed (Esc key)
onVoiceTranscription
function
Callback for voice transcription results
onVoiceMode
function
onVoiceMode(isActive: boolean): void
Callback when voice conversation mode changes

Features

  • Text Input: Multi-line textarea with auto-resize
  • Voice Recording: One-shot speech-to-text
  • Voice Conversation: Continuous back-and-forth conversation
  • Image Attachments: Upload or paste up to 5 images (max 10MB each)
  • Audio Attachments: Upload up to 3 audio files (max 25MB each)
  • Drag & Drop: Drop files directly onto input
  • Keyboard Shortcuts: Enter to send, Shift+Enter for new line, Esc to close

BabylonScene

Babylon.js 3D scene component managing scene initialization, rendering, and cleanup.

Props

sceneBuilder
function
sceneBuilder(
  canvas: HTMLCanvasElement,
  engine: Engine,
  config: SceneConfig
): Promise<Scene>
Custom scene builder function. Receives canvas, engine, and config.
onSceneReady
function
onSceneReady(scene: Scene): void
Callback when scene is ready. Receives Babylon.js Scene object.
onLoadProgress
function
onLoadProgress(progress: number): void
Callback for loading progress updates (0-100).
sceneConfig
object
Scene configuration object passed to builder
positionManagerRef
object
Ref to PositionManager instance for drag-drop overlay positioning
isPreview
boolean
default:false
Render as inline preview instead of full-screen portal
previewWidth
string
default:"100%"
Width for preview mode
previewHeight
string
default:"100%"
Height for preview mode
previewClassName
string
default:""
Additional CSS classes for preview container

Features

  • Automatic Initialization: Creates engine and scene on mount
  • FPS Limiting: Configurable frame rate limiting (30/60/90/native)
  • Drag & Drop Overlay: Positioned over 3D model for file drops
  • Resource Cleanup: Proper disposal of Babylon resources on unmount
  • Portal Rendering: Renders to document.body in production mode
  • Preview Mode: Inline rendering for setup/configuration UIs

Scene Metadata

The scene builder should attach managers to scene.metadata:
scene.metadata = {
  animationManager: animationManager,
  positionManager: positionManager,
};

Usage Examples

import { useRef } from 'react';
import VirtualAssistant from './components/VirtualAssistant';

function App() {
  const assistantRef = useRef();

  const handleReady = ({ animationManager, positionManager }) => {
    console.log('Assistant ready!');
  };

  const handleSpeak = async () => {
    await assistantRef.current.speak(
      "Hello, how can I help you?",
      vmdBlobUrl,
      "talking"
    );
  };

  return (
    <>
      <VirtualAssistant ref={assistantRef} onReady={handleReady} />
      <button onClick={handleSpeak}>Speak</button>
    </>
  );
}
import ChatContainer from './components/ChatContainer';
import ChatInput from './components/ChatInput';

function ChatApp() {
  const handleSend = (text, images, audios) => {
    console.log('Message:', text);
    console.log('Images:', images.length);
    console.log('Audios:', audios.length);
  };

  const handleDragDrop = (data) => {
    console.log('Dropped data:', data);
  };

  return (
    <>
      <ChatContainer onDragDrop={handleDragDrop} />
      <ChatInput onSend={handleSend} />
    </>
  );
}
import BabylonScene from './components/BabylonScene';

function Custom3DApp() {
  const buildScene = async (canvas, engine, config) => {
    const scene = new Scene(engine);
    // ... setup camera, lights, meshes
    return scene;
  };

  const handleSceneReady = (scene) => {
    console.log('Scene ready!', scene);
  };

  const handleProgress = (progress) => {
    console.log('Loading:', progress + '%');
  };

  return (
    <BabylonScene
      sceneBuilder={buildScene}
      onSceneReady={handleSceneReady}
      onLoadProgress={handleProgress}
    />
  );
}

Build docs developers (and LLMs) love