Overview
ORB-SLAM3 implements sophisticated visual-inertial fusion, integrating IMU (Inertial Measurement Unit) data with visual features to achieve:Metric Scale
Recovers absolute scale without scale ambiguity, even in monocular mode.
Robustness
Handles fast motion, temporary occlusions, and texture-poor environments.
Gravity Estimation
Estimates gravity direction and magnitude for consistent world frame.
Velocity Estimation
Provides camera velocity for motion prediction and feature tracking.
The visual-inertial system is based on the paper “Inertial-Only Optimization for Visual-Inertial Initialization” (ICRA 2020) and “Visual-inertial monocular SLAM with map reuse” (RA-L 2017).
IMU Data Structures
All IMU-related types are defined ininclude/ImuTypes.h:40-262.
IMU Measurement Point
Represents a single IMU measurement with accelerometer, gyroscope, and timestamp:IMU Bias
Accelerometer and gyroscope biases that are estimated and refined over time:Biases are automatically estimated during IMU initialization and continuously refined. You don’t need to provide them manually.
IMU Calibration
Stores IMU-camera extrinsic calibration and noise parameters:IMU Preintegration
ORB-SLAM3 uses IMU preintegration to efficiently integrate high-frequency IMU data between keyframes.Preintegrated Class
The core preintegration class defined ininclude/ImuTypes.h:143-251:
Preintegration Workflow
IMU Initialization
Visual-inertial initialization is a critical phase that estimates scale, gravity, and initial velocities.Initialization Sequence
- Phase 1: Visual Init
- Phase 2: Coarse IMU Init
- Phase 3: Refined IMU Init
Duration: First few frames
- Initialize visual SLAM (map with scale ambiguity)
- Accumulate IMU measurements
- Create initial keyframes
During this phase, the system operates in visual-only mode until sufficient motion and IMU data are available.
Checking Initialization Status
Gravity Constant
The expected gravity magnitude is defined ininclude/ImuTypes.h:43:
This value is used as a prior during initialization. The actual gravity magnitude and direction are estimated from IMU measurements.
IMU Configuration
IMU parameters are specified in the YAML settings file:Noise Parameters
Gyroscope measurement noise density (rad/s/√Hz). Obtain from IMU datasheet or calibration.
Accelerometer measurement noise density (m/s²/√Hz).
Gyroscope random walk (rad/s²/√Hz). Models bias drift over time.
Accelerometer random walk (m/s³/√Hz).
Integration with Visual Tracking
Motion Prediction
IMU provides motion prediction for the next frame:Visual-Inertial Optimization
The system jointly optimizes visual and inertial terms:E_visual: Reprojection error from visual featuresE_inertial: Preintegrated IMU residualsρ: Robust cost function (Huber)π: Camera projection functionr_imu: IMU preintegration residual
Integrated Rotation
Helper class for integrating single gyroscope measurements:Lie Algebra Functions
Utility functions for SO(3) operations defined ininclude/ImuTypes.h:254-260:
These functions are used for manifold operations during optimization on the SO(3) rotation group.
Best Practices
IMU Data Collection
IMU Data Collection
- High frequency: Use IMU at 100-200 Hz (much higher than camera frame rate)
- All measurements: Provide ALL IMU samples between consecutive frames
- Synchronization: Ensure accurate time synchronization between camera and IMU
- Coordinate frames: Ensure IMU measurements are in the correct body frame
Calibration Quality
Calibration Quality
- Extrinsic calibration: Accurately calibrate IMU-camera transformation (Tbc)
- Temporal calibration: Estimate time offset between sensors
- Noise parameters: Use accurate noise values from datasheet or Allan variance
- Verify calibration: Test with known motions to validate setup
Initialization
Initialization
- Initial motion: Perform slow, smooth motion for first 2-3 seconds
- Sufficient excitation: Ensure rotation and translation for observability
- Wait for completion: Check
isImuInitialized()before relying on scale - Avoid aggressive motion: During initialization, avoid fast rotations
Runtime Considerations
Runtime Considerations
- Bias drift: Long-term bias drift is automatically tracked and corrected
- Temperature effects: Some IMUs drift with temperature changes
- Magnetic interference: Keep IMU away from magnetic sources
- Vibrations: Mount IMU rigidly to camera to avoid vibration noise
Coordinate Frames
Camera Frame (C)
- Origin at camera optical center
- Z-axis pointing forward (viewing direction)
- X-axis pointing right, Y-axis pointing down
Body Frame (B/IMU)
- Origin at IMU center
- Axes follow IMU orientation
- Related to camera by Tbc transformation
World Frame (W)
- Aligned with gravity (Z-axis up or down)
- Arbitrary horizontal orientation
- Established during initialization
Navigation Frame (N)
- East-North-Up or similar
- Used for expressing poses and velocities
- Related to world frame by rotation
Performance Benefits
Fast Motion
IMU prediction enables tracking during rapid camera motion (>10 m/s linear, >200°/s angular).
Occlusions
Maintains pose estimate during temporary visual feature loss using inertial integration.
Scale Accuracy
Achieves less than 1% scale error in monocular mode, comparable to stereo systems.
Related Topics
Sensor Modes
Visual-inertial sensor mode configurations
Calibration
IMU-camera calibration procedures
Configuration
YAML settings for IMU parameters
EuRoC Examples
Running visual-inertial examples
Troubleshooting
Initialization Fails
Initialization Fails
- Insufficient motion: Ensure camera moves with rotation and translation
- Poor calibration: Verify Tbc extrinsic calibration
- Wrong noise values: Check IMU noise parameters in config
- Too fast motion: Slow down during first 2-3 seconds
Scale Drift
Scale Drift
- Poor IMU quality: Low-cost IMUs may have significant bias drift
- Long sequences: Scale drift accumulates over time without loops
- Temperature changes: Some IMUs are temperature sensitive
- Insufficient visual constraints: Ensure good feature tracking
Tracking Loss with IMU
Tracking Loss with IMU
- Calibration errors: Large Tbc errors cause divergence
- Time synchronization: Ensure accurate timestamps
- Coordinate frame mismatch: Verify IMU axes orientation
- IMU saturation: Check for gyro/accelerometer saturation
Advanced Topics
- Bias estimation: Continuous bias refinement during operation
- Scale observability: Conditions for scale estimation in monocular-inertial
- Covariance propagation: Uncertainty tracking through preintegration
- Manifold optimization: SO(3) optimization for rotation states