Model Detection

Overview

Model detection allows you to identify STT and TTS model types and structures without initializing the full recognizer or engine. This is useful for:

UI/UX: Show model information before initialization
Validation: Verify model files are complete and correct
Configuration: Auto-configure based on detected model type
Error prevention: Catch model issues early

Model detection is stateless and requires no engine instance. It uses the same native file-based detection as model initialization.

STT Model Detection

detectSttModel

Detect STT model type and structure without initializing the recognizer.

import { detectSttModel } from 'react-native-sherpa-onnx/stt';

const result = await detectSttModel(
  { type: 'asset', path: 'models/sherpa-onnx-whisper-tiny-en' },
  { preferInt8: false }
);

if (result.success && result.detectedModels.length > 0) {
  console.log('Detected type:', result.modelType);
  console.log('All detected models:', result.detectedModels);
  // Output:
  // Detected type: whisper
  // All detected models: [{ type: 'whisper', modelDir: '/path/to/model' }]
} else {
  console.error('Model detection failed - no valid model found');
}

modelPath

ModelPathConfig

required

Model path configuration

Show ModelPathConfig types

type ModelPathConfig =
  | { type: 'asset'; path: string }  // Asset from app bundle
  | { type: 'file'; path: string }   // Absolute file path
  | { type: 'auto'; path: string };  // Auto-detect (tries asset, then file)

options

object

Show Detection options

preferInt8

boolean

default:false

Prefer int8 quantized models when multiple variants are found

modelType

STTModelType

Hint for the expected model type (e.g., 'whisper', 'transducer')

result

object

Detection result object

Show Detection result properties

success

boolean

true if at least one valid model was detected

detectedModels

Array<{ type: string; modelDir: string }>

Array of all detected model configurations

type: Model type (e.g., 'whisper', 'transducer', 'paraformer')
modelDir: Absolute path to the model directory

modelType

string

Primary detected model type (first entry from detectedModels)

Example: Pre-validation

Validate model before showing initialization UI:

import { detectSttModel } from 'react-native-sherpa-onnx/stt';

const modelPath = { type: 'file', path: downloadedModelPath };

// Detect model type before initialization
const detection = await detectSttModel(modelPath);

if (!detection.success) {
  Alert.alert(
    'Invalid Model',
    'The downloaded model files are incomplete or corrupted. Please download again.'
  );
  return;
}

// Show detected model info to user
setModelInfo({
  type: detection.modelType,
  path: detection.detectedModels[0].modelDir,
  ready: true,
});

// Proceed with initialization
const stt = await createSTT({
  modelPath,
  modelType: detection.modelType,
});

Example: Auto-configuration

Auto-configure based on detected model type:

import { detectSttModel, createSTT } from 'react-native-sherpa-onnx/stt';

const modelPath = { type: 'asset', path: 'models/my-model' };

// Detect model type first
const detection = await detectSttModel(modelPath);

if (!detection.success) {
  throw new Error('Model detection failed');
}

// Auto-configure based on detected type
const modelType = detection.modelType;
let config = {};

if (modelType === 'transducer' || modelType === 'nemo_transducer') {
  // Enable hotwords for transducer models
  config = {
    hotwordsFile: '/path/to/hotwords.txt',
    modelingUnit: 'bpe',
  };
} else if (modelType === 'whisper') {
  // Configure language for Whisper
  config = {
    modelOptions: {
      whisper: {
        language: 'en',
        task: 'transcribe',
      },
    },
  };
}

// Initialize with auto-configured options
const stt = await createSTT({
  modelPath,
  modelType,
  ...config,
});

TTS Model Detection

detectTtsModel

Detect TTS model type and structure without initializing the engine.

import { detectTtsModel } from 'react-native-sherpa-onnx/tts';

const result = await detectTtsModel(
  { type: 'asset', path: 'models/vits-piper-en_US-lessac-medium' },
  { modelType: 'auto' }
);

if (result.success) {
  console.log('Detected TTS type:', result.modelType);
  console.log('Detected models:', result.detectedModels);
  // Output:
  // Detected TTS type: vits
  // Detected models: [{ type: 'vits', modelDir: '/path/to/model' }]
}

modelPath

ModelPathConfig

required

Model path configuration (same as STT)

options

object

Show Detection options

modelType

TTSModelType

default:"'auto'"

Hint for the expected model type (e.g., 'vits', 'kokoro', 'matcha')

result

object

Detection result (same structure as STT detection)

Show Detection result properties

success

boolean

true if at least one valid model was detected

detectedModels

Array<{ type: string; modelDir: string }>

Array of all detected model configurations

type: Model type (e.g., 'vits', 'kokoro', 'matcha', 'kitten', 'pocket', 'zipvoice')
modelDir: Absolute path to the model directory

modelType

string

Primary detected model type

Example: Show model info before TTS init

import { detectTtsModel, createTTS } from 'react-native-sherpa-onnx/tts';

const modelPath = { type: 'file', path: downloadedModelPath };

// Detect TTS model
const detection = await detectTtsModel(modelPath);

if (!detection.success) {
  Alert.alert('Invalid TTS Model', 'Model files are incomplete.');
  return;
}

// Show model info
setTtsModelInfo({
  type: detection.modelType,
  supportsMultipleSpeakers: ['vits', 'kokoro'].includes(detection.modelType),
  ready: true,
});

// Initialize TTS
const tts = await createTTS({
  modelPath,
  modelType: detection.modelType,
});

// Get additional info after init
const info = await tts.getModelInfo();
console.log('Sample rate:', info.sampleRate);
console.log('Number of speakers:', info.numSpeakers);

Common Use Cases

Download Validation

Verify downloaded models are complete before marking them as “ready” in the UI.

const detection = await detectSttModel(downloadedPath);
if (detection.success) {
  markModelAsReady(modelId);
}

Model Type Display

Show model type information to users before initialization.

const detection = await detectTtsModel(modelPath);
setModelType(detection.modelType);

Auto-configuration

Automatically configure options based on detected model type.

const detection = await detectSttModel(modelPath);
const hotwordsEnabled = 
  sttSupportsHotwords(detection.modelType);

Error Prevention

Catch model issues early before attempting full initialization.

const detection = await detectSttModel(modelPath);
if (!detection.success) {
  showErrorAndRedownload();
}

Detection vs Initialization

When to use detection

Use detection when you need to:

Validate model files without loading them into memory
Show model information in UI before initialization
Check if a model is compatible with certain features (e.g., hotwords)
Implement a model browser or manager
Verify downloaded models are complete

When to use initialization

Use initialization when you’re ready to:

Actually perform transcription or speech synthesis
Load model weights into memory
Create streams or generate audio
Use the model for its intended purpose

Performance comparison

Detection is much faster than initialization:

Detection: ~10-100ms (just checks file existence and structure)
Initialization: ~500-5000ms (loads model weights, allocates memory)

Detection only validates file structure without loading model weights.

Multiple Model Detection

Some model directories contain multiple model variants (e.g., fp32 and int8 quantized). Detection returns all found variants:

const detection = await detectSttModel(
  { type: 'asset', path: 'models/zipformer-en' },
  { preferInt8: true }
);

if (detection.detectedModels.length > 1) {
  console.log('Multiple model variants found:');
  detection.detectedModels.forEach((model, index) => {
    console.log(`  ${index + 1}. Type: ${model.type}, Path: ${model.modelDir}`);
  });
}

// Primary model (respects preferInt8 option)
console.log('Selected model type:', detection.modelType);

The preferInt8 option influences which model is selected as the primary modelType when multiple variants are found. Int8 models are smaller and faster but may have slightly lower accuracy.

Error Handling

import { detectSttModel } from 'react-native-sherpa-onnx/stt';

try {
  const detection = await detectSttModel(modelPath);
  
  if (!detection.success) {
    // No valid model found
    console.error('Model detection failed');
    console.log('Attempted path:', modelPath.path);
    
    // Show user-friendly error
    Alert.alert(
      'Invalid Model',
      'The model directory is missing required files. Please ensure the model was fully downloaded and extracted.'
    );
    return;
  }
  
  if (detection.detectedModels.length === 0) {
    // Edge case: success but no models (shouldn't happen)
    console.error('Unexpected detection state');
    return;
  }
  
  // Success - proceed with model
  console.log('Model valid:', detection.modelType);
  
} catch (error) {
  // File system or permission errors
  console.error('Detection error:', error);
  Alert.alert('Detection Failed', error.message);
}

Best Practices

Validate Before Init

Always detect models after download before marking them as ready or attempting initialization.

Cache Detection Results

Cache detection results to avoid redundant file system checks. Only re-detect when model files change.

Show Progress

Show loading indicators during detection, even though it’s fast. Some devices may take longer.

Handle Multiple Variants

When multiple models are detected, let users choose or automatically select based on device capabilities (e.g., int8 for lower-end devices).

Integration with Download Manager

Combine model detection with the download manager for robust model management:

import { 
  ModelCategory,
  downloadModelByCategory,
  getLocalModelPathByCategory 
} from 'react-native-sherpa-onnx/download';
import { detectSttModel, createSTT } from 'react-native-sherpa-onnx/stt';

// 1. Download model
await downloadModelByCategory(
  ModelCategory.Stt,
  'sherpa-onnx-whisper-tiny',
  { onProgress: (p) => console.log(`${p.percent}%`) }
);

// 2. Get local path
const localPath = await getLocalModelPathByCategory(
  ModelCategory.Stt,
  'sherpa-onnx-whisper-tiny'
);

if (!localPath) {
  throw new Error('Model download failed');
}

// 3. Detect and validate
const modelPath = { type: 'file' as const, path: localPath };
const detection = await detectSttModel(modelPath);

if (!detection.success) {
  // Model files are corrupted - delete and re-download
  await deleteModelByCategory(ModelCategory.Stt, 'sherpa-onnx-whisper-tiny');
  throw new Error('Model validation failed - please re-download');
}

// 4. Initialize with detected type
const stt = await createSTT({
  modelPath,
  modelType: detection.modelType,
});

console.log('STT ready with model type:', detection.modelType);

Type Definitions

// STT detection
export function detectSttModel(
  modelPath: ModelPathConfig,
  options?: { preferInt8?: boolean; modelType?: STTModelType }
): Promise<{
  success: boolean;
  detectedModels: Array<{ type: string; modelDir: string }>;
  modelType?: string;
}>;

// TTS detection
export function detectTtsModel(
  modelPath: ModelPathConfig,
  options?: { modelType?: TTSModelType }
): Promise<{
  success: boolean;
  detectedModels: Array<{ type: string; modelDir: string }>;
  modelType?: string;
}>;

// Model path configuration
type ModelPathConfig =
  | { type: 'asset'; path: string }  // Asset from app bundle
  | { type: 'file'; path: string }   // Absolute file path  
  | { type: 'auto'; path: string };  // Auto-detect

Model Download Manager - Download and manage models
STT Initialization - Initialize STT with detected model type
TTS Initialization - Initialize TTS with detected model type
Hotwords - Use detection to check hotword support

Get Started

Core Features

Advanced

Configuration

Overview

STT Model Detection

detectSttModel

Example: Pre-validation

Example: Auto-configuration

TTS Model Detection

detectTtsModel

Example: Show model info before TTS init

Common Use Cases

Download Validation

Model Type Display

Auto-configuration

Error Prevention

Detection vs Initialization

Multiple Model Detection

Error Handling

Best Practices

Validate Before Init

Cache Detection Results

Show Progress

Handle Multiple Variants

Integration with Download Manager

Type Definitions

Build docs developers (and LLMs) love

Get Started

Core Features

Advanced

Configuration

​Overview

​STT Model Detection

​detectSttModel

​Example: Pre-validation

​Example: Auto-configuration

​TTS Model Detection

​detectTtsModel

​Example: Show model info before TTS init

​Common Use Cases

Download Validation

Model Type Display

Auto-configuration

Error Prevention

​Detection vs Initialization

​Multiple Model Detection

​Error Handling

​Best Practices

Validate Before Init

Cache Detection Results

Show Progress

Handle Multiple Variants

​Integration with Download Manager

​Type Definitions

​Related APIs

Build docs developers (and LLMs) love

Overview

STT Model Detection

detectSttModel

Example: Pre-validation

Example: Auto-configuration

TTS Model Detection

detectTtsModel

Example: Show model info before TTS init

Common Use Cases

Detection vs Initialization

Multiple Model Detection

Error Handling

Best Practices

Integration with Download Manager

Type Definitions

Related APIs