Skip to main content

​
Overview

The RettoOrtWorker provides an ONNX Runtime backend for running PaddleOCR models with support for CPU, CUDA, and DirectML execution providers.

​
RettoOrtWorker

The main worker struct that manages ONNX Runtime sessions for detection, classification, and recognition.

​
Type Definition

pub struct RettoOrtWorker {
    cfg: RettoOrtWorkerConfig,
    det_session: ort::session::Session,
    rec_session: ort::session::Session,
    cls_session: ort::session::Session,
}
Source: retto-core/src/worker/ort_worker.rs:113

​
Methods

​
new()

Creates a new ONNX Runtime worker with the specified configuration. 
fn new(cfg: RettoOrtWorkerConfig) -> RettoResult<Self>
Parameters:
  • cfg: Configuration including device settings and model providers
Returns:
  • RettoResult<Self>: The initialized worker or an error
Example: 
use retto_core::prelude::*;

let config = RettoOrtWorkerConfig {
    device: RettoOrtWorkerDevice::CPU,
    models: RettoOrtWorkerModelProvider::from_hf_hub_v4_default(),
};

let worker = RettoOrtWorker::new(config)?;

​
init()

Initializes the worker (currently a no-op for ORT backend). 
fn init(&self) -> RettoResult<()>

​
RettoOrtWorkerConfig

Configuration struct for the ONNX Runtime worker.

​
Type Definition

pub struct RettoOrtWorkerConfig {
    pub device: RettoOrtWorkerDevice,
    pub models: RettoOrtWorkerModelProvider,
}
Source: retto-core/src/worker/ort_worker.rs:53

​
Fields

​
device
RettoOrtWorkerDevice
The execution device to use (CPU, CUDA, or DirectML)
​
models
RettoOrtWorkerModelProvider
Model provider specifying sources for det, cls, and rec models

​
Example

let config = RettoOrtWorkerConfig {
    device: RettoOrtWorkerDevice::CPU,
    models: RettoOrtWorkerModelProvider::from_hf_hub_v4_default(),
};

​
RettoOrtWorkerDevice

Enum specifying which execution provider to use for inference.

​
Type Definition

pub enum RettoOrtWorkerDevice {
    CPU,
    #[cfg(feature = "backend-ort-cuda")]
    Cuda(i32),
    #[cfg(feature = "backend-ort-directml")]
    DirectML(i32),
}
Source: retto-core/src/worker/ort_worker.rs:21

​
Variants

​
CPU
variant
Use CPU execution provider only (default)
​
Cuda(i32)
variant
Use NVIDIA CUDA execution provider with specified device IDRequires backend-ort-cuda feature flag
​
DirectML(i32)
variant
Use DirectML execution provider with specified device IDRequires backend-ort-directml feature flag

​
Examples

let device = RettoOrtWorkerDevice::CPU;

​
Backend Configuration

The ORT worker configures execution providers based on the selected device:

​
CPU Backend

Always available as a fallback. Uses CPUExecutionProvider. 
let config = RettoOrtWorkerConfig {
    device: RettoOrtWorkerDevice::CPU,
    models: RettoOrtWorkerModelProvider::from_hf_hub_v4_default(),
};

​
CUDA Backend

Available when compiled with backend-ort-cuda feature. Configures:
  • Arena extend strategy: NextPowerOfTwo
  • CuDNN conv algorithm search: Exhaustive
  • Device ID selection
let config = RettoOrtWorkerConfig {
    device: RettoOrtWorkerDevice::Cuda(0), // Use GPU 0
    models: RettoOrtWorkerModelProvider::from_hf_hub_v4_default(),
};

​
DirectML Backend

Available when compiled with backend-ort-directml feature. Supports Windows DirectX acceleration. 
let config = RettoOrtWorkerConfig {
    device: RettoOrtWorkerDevice::DirectML(0),
    models: RettoOrtWorkerModelProvider::from_hf_hub_v4_default(),
};
The CPU execution provider is always added as a fallback, even when GPU acceleration is enabled.

​
Complete Example

use retto_core::prelude::*;

// Create configuration with CUDA support
let config = RettoOrtWorkerConfig {
    device: RettoOrtWorkerDevice::Cuda(0),
    models: RettoOrtWorkerModelProvider::from_hf_hub_v4_default(),
};

// Initialize worker
let worker = RettoOrtWorker::new(config)?;
worker.init()?;

// Use with RettoSession
let session_config = RettoSessionConfig {
    worker_config: config,
    ..Default::default()
};

let mut session = RettoSession::new(session_config)?;
For WASM targets, the worker automatically initializes the ORT environment with a global thread pool.

Build docs developers (and LLMs) love

Get started for freeTalk to us