Skip to main content

​
Overview

ORB-SLAM3 supports six different sensor modes, combining three camera types (monocular, stereo, RGB-D) with optional IMU integration. Each mode is optimized for different hardware configurations and use cases.

​
Sensor Mode Enumeration

The sensor modes are defined in include/System.h:87-94: 
enum eSensor{
    MONOCULAR=0,
    STEREO=1,
    RGBD=2,
    IMU_MONOCULAR=3,
    IMU_STEREO=4,
    IMU_RGBD=5,
};

​
Visual-Only Modes

​
Monocular Mode

Single camera tracking without depth information.
System SLAM(strVocFile, strSettingsFile, System::MONOCULAR, true);
Characteristics:
  • Scale ambiguity: Map is computed up to an unknown scale factor
  • Initialization: Requires sufficient parallax between first frames
  • Use cases: Lightweight systems, drones, smartphones
Monocular SLAM cannot determine the absolute scale of the environment. The map scale is arbitrary.

​
Visual-Inertial Modes

Visual-inertial modes combine camera input with IMU measurements for enhanced robustness and accuracy.

​
IMU-Monocular Mode

Single camera with inertial sensor (accelerometer + gyroscope).
System SLAM(strVocFile, strSettingsFile, System::IMU_MONOCULAR, true);
Characteristics:
  • Metric scale: IMU provides scale observability
  • Fast motion: Handles rapid camera movements
  • Initialization: 2-15 seconds for IMU initialization
  • Gravity estimation: Estimates gravity direction and magnitude
IMU-Monocular achieves scale error < 5% after 2 seconds of initialization, refined to ~1% within 15 seconds.

​
IMU Data Structure

When using visual-inertial modes, IMU measurements are passed as IMU::Point objects defined in include/ImuTypes.h:46-59: 
class Point
{
public:
    Point(const float &acc_x, const float &acc_y, const float &acc_z,
          const float &ang_vel_x, const float &ang_vel_y, const float &ang_vel_z,
          const double &timestamp);
    
    Eigen::Vector3f a;  // Accelerometer measurement (m/s²)
    Eigen::Vector3f w;  // Gyroscope measurement (rad/s)
    double t;           // Timestamp
};

// Single measurement
IMU::Point imu1(ax, ay, az, wx, wy, wz, timestamp);

// Vector of measurements
vector<IMU::Point> vImuMeas;
vImuMeas.push_back(IMU::Point(ax1, ay1, az1, wx1, wy1, wz1, t1));
vImuMeas.push_back(IMU::Point(ax2, ay2, az2, wx2, wy2, wz2, t2));

​
Comparison Matrix

FeatureMonocularStereoRGB-DIMU-MonoIMU-StereoIMU-RGBD
Metric Scale❌✅✅✅✅✅
Initialization TimeMediumInstantInstant2-15sFast2-15s
Fast Motion⚠️⚠️⚠️✅✅✅
Hardware CostLowMediumMediumLowHighHigh
Outdoor Use✅✅❌✅✅⚠️
Computational CostLowMediumMediumMediumHighHigh

​
Choosing the Right Mode

Lightweight & Portable

Monocular or IMU-Monocular for drones, smartphones, and resource-constrained devices.

High Accuracy

Stereo or IMU-Stereo for robotics applications requiring precise metric reconstruction.

Indoor Environments

RGB-D or IMU-RGBD for dense indoor mapping with depth sensors.

Fast Motion

Any IMU mode for aggressive motion, temporal occlusions, or challenging dynamics.

​
Best Practices

  • Stereo: Images must be synchronized and rectified (for pinhole models)
  • RGB-D: Depth must be registered to RGB frame
  • All modes: RGB images are automatically converted to grayscale
  • Input formats: RGB (CV_8UC3) or grayscale (CV_8U)
  • Provide all IMU measurements between consecutive frames
  • Typical IMU rate: 100-200 Hz (much higher than camera frame rate)
  • Ensure proper IMU-camera calibration (extrinsics and time offset)
  • Configure noise parameters in settings file
  • Use consistent timestamp units (typically seconds)
  • For IMU modes, timestamps must be accurately synchronized
  • Account for any time offset between sensors

Camera Models

Configure pinhole or fisheye lens models for your sensors

IMU Integration

Deep dive into IMU calibration and preintegration

Configuration Files

Set up YAML configuration for your sensor mode

Example Usage

See complete examples for each sensor mode

Build docs developers (and LLMs) love

Get started for freeTalk to us