RealSense Camera Examples

ORB-SLAM3 provides native support for Intel RealSense cameras, enabling real-time visual and visual-inertial SLAM with live camera feeds.

Supported Cameras

RealSense D435i

Depth Camera with IMU

Dual infrared stereo cameras
Active depth sensor
Built-in IMU (accel + gyro)
640x480 @ 30Hz
USB 3.0 connection

RealSense T265

Tracking Camera

Dual fisheye cameras
Built-in IMU
848x800 @ 30Hz
Purpose-built for SLAM
USB 2.0/3.0 connection

Prerequisites

Install librealsense2

Install the RealSense SDK:

Ubuntu (apt)
From Source

sudo apt-key adv --keyserver keyserver.ubuntu.com \
    --recv-key F6E65AC044F831AC80A06380C8B3A55A6F3EFCDE

sudo add-apt-repository \
    "deb https://librealsense.intel.com/Debian/apt-repo \
    $(lsb_release -sc) main"

sudo apt update
sudo apt install librealsense2-dev librealsense2-utils

git clone https://github.com/IntelRealSense/librealsense.git
cd librealsense
mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release
make -j$(nproc)
sudo make install

Verify installation

Check that your camera is detected:

rs-enumerate-devices

You should see your camera listed with its serial number and firmware version.

Test camera stream

realsense-viewer

This opens the RealSense viewer to verify camera functionality.

Build RealSense examples

RealSense examples are built automatically with the main library:

cd ORB_SLAM3
./build.sh

RealSense D435i Examples

The D435i provides stereo infrared cameras and RGB-D depth.

Stereo-Inertial Mode (Recommended)

Use infrared stereo cameras with IMU for best performance:

./Examples/Stereo-Inertial/stereo_inertial_realsense_D435i \
    Vocabulary/ORBvoc.txt \
    Examples/Stereo-Inertial/RealSense_D435i.yaml

From the README.md:110-111, this is the recommended way to use the D435i with ORB-SLAM3.

What Happens Internally

The example (stereo_inertial_realsense_D435i.cc:162-165) configures streams:

rs2::config cfg;
cfg.enable_stream(RS2_STREAM_INFRARED, 1, 640, 480, RS2_FORMAT_Y8, 30);
cfg.enable_stream(RS2_STREAM_INFRARED, 2, 640, 480, RS2_FORMAT_Y8, 30);
cfg.enable_stream(RS2_STREAM_ACCEL, RS2_FORMAT_MOTION_XYZ32F);
cfg.enable_stream(RS2_STREAM_GYRO, RS2_FORMAT_MOTION_XYZ32F);

Key features:

Infrared cameras: Better for SLAM (no RGB distraction, consistent lighting)
IMU synchronization: Accelerometer data interpolated to gyro timestamps
Auto-exposure: Automatically enabled for varying lighting
Emitter disabled: IR projector turned off (stereo_inertial_realsense_D435i.cc:143)

RGB-D Mode

Use the depth sensor with RGB camera:

./Examples/RGB-D/rgbd_realsense_D435i \
    Vocabulary/ORBvoc.txt \
    Examples/RGB-D/RealSense_D435i.yaml

Useful for:

Indoor environments
Objects at close range (less than 10m)
Dense reconstruction needs

RGB-D mode has limited range and doesn’t work outdoors. Use Stereo-Inertial for most applications.

Monocular and Stereo Modes

Also available but less commonly used:

Monocular
Stereo
Monocular-Inertial

./Examples/Monocular/mono_realsense_D435i \
    Vocabulary/ORBvoc.txt \
    Examples/Monocular/RealSense_D435i.yaml

./Examples/Stereo/stereo_realsense_D435i \
    Vocabulary/ORBvoc.txt \
    Examples/Stereo/RealSense_D435i.yaml

./Examples/Monocular-Inertial/mono_inertial_realsense_D435i \
    Vocabulary/ORBvoc.txt \
    Examples/Monocular-Inertial/RealSense_D435i.yaml

RealSense T265 Examples

The T265 is specifically designed for tracking and SLAM with fisheye cameras.

Stereo-Inertial Mode

./Examples/Stereo-Inertial/stereo_inertial_realsense_t265 \
    Vocabulary/ORBvoc.txt \
    Examples/Stereo-Inertial/RealSense_T265.yaml

Configuration

The T265 example (stereo_inertial_realsense_t265.cc:89-94) uses fisheye streams:

rs2::config cfg;
cfg.enable_stream(RS2_STREAM_FISHEYE, 1, RS2_FORMAT_Y8, 30);
cfg.enable_stream(RS2_STREAM_FISHEYE, 2, RS2_FORMAT_Y8, 30);
cfg.enable_stream(RS2_STREAM_ACCEL, RS2_FORMAT_MOTION_XYZ32F);
cfg.enable_stream(RS2_STREAM_GYRO, RS2_FORMAT_MOTION_XYZ32F);

Advantages:

Wide field of view (170°)
Optimized for tracking
Low latency
Works in low light

Monocular-Inertial Mode

./Examples/Monocular-Inertial/mono_inertial_realsense_t265 \
    Vocabulary/ORBvoc.txt \
    Examples/Monocular-Inertial/RealSense_T265.yaml

Comparison: T265 vs D435i

Feature	T265	D435i
Camera Type	Fisheye	Infrared Stereo
FOV	170°	~85°
Best For	Tracking, Motion	Depth, Mapping
Depth Sensor	No	Yes
Outdoor Use	Excellent	Good
Indoor Use	Excellent	Excellent
Price	Higher	Lower

Camera Configuration

RealSense cameras are automatically configured with optimal settings.

D435i Auto-Configuration

The examples automatically set (stereo_inertial_realsense_D435i.cc:141-154):

// Enable auto-exposure
sensor.set_option(RS2_OPTION_ENABLE_AUTO_EXPOSURE, 1);
sensor.set_option(RS2_OPTION_AUTO_EXPOSURE_LIMIT, 5000);

// Disable IR emitter (for stereo)
sensor.set_option(RS2_OPTION_EMITTER_ENABLED, 0);

// Disable motion correction (we do our own)
sensor.set_option(RS2_OPTION_ENABLE_MOTION_CORRECTION, 0);

Custom Configuration

To modify camera settings, edit the YAML file:

# Camera resolution
Camera.width: 640
Camera.height: 480
Camera.fps: 30

# Camera intrinsics (measured)
Camera.fx: 387.065
Camera.fy: 387.065  
Camera.cx: 321.293
Camera.cy: 238.532

# IMU parameters
IMU.NoiseGyro: 0.00016
IMU.NoiseAcc: 0.0028
IMU.Frequency: 200

Running Real-Time SLAM

Connect camera

Plug in your RealSense camera via USB 3.0. Verify with rs-enumerate-devices.

Launch ORB-SLAM3

./Examples/Stereo-Inertial/stereo_inertial_realsense_D435i \
    Vocabulary/ORBvoc.txt \
    Examples/Stereo-Inertial/RealSense_D435i.yaml

Initialize

Move the camera slowly with visible features in view. Wait for initialization to complete (status shown in viewer).

Operate

Move the camera through your environment. The system tracks in real-time and builds a 3D map.

Stop

Press Ctrl+C to stop. Trajectories are automatically saved:

CameraTrajectory.txt
KeyFrameTrajectory.txt

Visualizer Controls

The Pangolin viewer shows:

Main View
Current Frame
Controls

3D map points (small dots)
Keyframes (camera frustums)
Current pose (red/green frustum)
Trajectory (line trail)

Troubleshooting

Camera Not Detected

Solutions:

Check USB 3.0 connection (blue port)
Try different USB port
Verify with rs-enumerate-devices
Update firmware: rs-fw-update
Check permissions: sudo chmod a+rw /dev/bus/usb/*/*

Initialization Failed

Tips:

Move camera slowly
Ensure good lighting
Point at textured surfaces
Wait for auto-exposure to stabilize
Check IMU is working: rs-enumerate-devices -c

Dropped Frames

If seeing “dropped frames” messages:

Check USB bandwidth (disconnect other devices)
Verify USB 3.0 connection
Close realsense-viewer if running
Reduce resolution if needed

Tracking Lost

Recovery:

System will attempt relocalization
Move to area with good features
Return to previously mapped area
Restart if necessary

IMU Synchronization Issues

The examples handle IMU synchronization automatically. If issues persist:

Check IMU.Frequency in YAML (should be 200)
Verify motion correction disabled
Ensure librealsense2 version >= 2.40

Performance Optimization

Reduce Resolution
Fewer Features
Disable Viewer

For faster processing, modify the example:

// Lower resolution (in .cc file)
cfg.enable_stream(RS2_STREAM_INFRARED, 1, 424, 240, RS2_FORMAT_Y8, 30);

Update YAML accordingly.

Reduce ORB features in YAML:

ORBextractor.nFeatures: 800  # Default: 1000

Modify System initialization:

ORB_SLAM3::System SLAM(argv[1], argv[2], 
    ORB_SLAM3::System::IMU_STEREO, 
    false);  // false = no viewer

Saving Trajectory

Trajectories are automatically saved when stopping:

# TUM format (default)
timestamp tx ty tz qx qy qz qw

# Files created
CameraTrajectory.txt        # All frames
KeyFrameTrajectory.txt      # Keyframes only

Custom Output Name

Specify output filename:

./Examples/Stereo-Inertial/stereo_inertial_realsense_D435i \
    Vocabulary/ORBvoc.txt \
    Examples/Stereo-Inertial/RealSense_D435i.yaml \
    my_trajectory

Creates:

f_my_trajectory.txt
kf_my_trajectory.txt

Advanced: ROS Integration

For ROS integration with RealSense:

Install ROS RealSense

sudo apt install ros-$ROS_DISTRO-realsense2-camera

Launch camera node

roslaunch realsense2_camera rs_camera.launch

Run ORB-SLAM3 ROS node

rosrun ORB_SLAM3 Stereo_Inertial \
    Vocabulary/ORBvoc.txt \
    Examples/ROS/ORB_SLAM3/RealSense_D435i.yaml \
    false

See ROS Examples for more details.

Comparison with Dataset Examples

Aspect	Live Camera	Datasets
Setup	Plug & play	Download required
Timing	Real-time	Can replay slower
Debugging	Harder	Easier (repeatable)
Ground Truth	No	Yes
Use Case	Deployment	Development

Develop and tune your system on datasets (EuRoC, TUM-VI) before testing with live cameras.

Next Steps

Camera Calibration

Calibrate your own RealSense setup

IMU Configuration

Fine-tune IMU parameters

ROS Integration

Use ORB-SLAM3 with ROS

System Tuning

Optimize for your application

Get Started

Core Concepts

Guides

Examples

Advanced

RealSense Camera Examples

Supported Cameras

RealSense D435i

RealSense T265

Prerequisites

RealSense D435i Examples

Stereo-Inertial Mode (Recommended)

What Happens Internally

RGB-D Mode

Monocular and Stereo Modes

RealSense T265 Examples

Stereo-Inertial Mode

Configuration

Monocular-Inertial Mode

Comparison: T265 vs D435i

Camera Configuration

D435i Auto-Configuration

Custom Configuration

Running Real-Time SLAM

Visualizer Controls

Troubleshooting

Performance Optimization

Saving Trajectory

Custom Output Name

Advanced: ROS Integration

Comparison with Dataset Examples

Next Steps

Camera Calibration

IMU Configuration

ROS Integration

System Tuning

Build docs developers (and LLMs) love

Get Started

Core Concepts

Guides

Examples

Advanced

​Supported Cameras

RealSense D435i

RealSense T265

​Prerequisites

​RealSense D435i Examples

​Stereo-Inertial Mode (Recommended)

​What Happens Internally

​RGB-D Mode

​Monocular and Stereo Modes

​RealSense T265 Examples

​Stereo-Inertial Mode

​Configuration

​Monocular-Inertial Mode

​Comparison: T265 vs D435i

​Camera Configuration

​D435i Auto-Configuration

​Custom Configuration

​Running Real-Time SLAM

​Visualizer Controls

​Troubleshooting

​Performance Optimization

​Saving Trajectory

​Custom Output Name

​Advanced: ROS Integration

​Comparison with Dataset Examples

​Next Steps

Camera Calibration

IMU Configuration

ROS Integration

System Tuning

Build docs developers (and LLMs) love

Supported Cameras

Prerequisites

RealSense D435i Examples

Stereo-Inertial Mode (Recommended)

What Happens Internally

RGB-D Mode

Monocular and Stereo Modes

RealSense T265 Examples

Stereo-Inertial Mode

Configuration

Monocular-Inertial Mode

Comparison: T265 vs D435i

Camera Configuration

D435i Auto-Configuration

Custom Configuration

Running Real-Time SLAM

Visualizer Controls

Troubleshooting

Performance Optimization

Saving Trajectory

Custom Output Name

Advanced: ROS Integration

Comparison with Dataset Examples

Next Steps