Skip to main content

​
Hardware Requirements

Before starting the software installation, ensure you have the following hardware components:

Computing Platform

Raspberry Pi (3B+ or newer recommended)
  • 1GB+ RAM
  • MicroSD card (16GB minimum)
  • Raspberry Pi OS (Bullseye or newer)

VEX Robotics Platform

VEX IQ2 Brain
  • 4x VEX Motors (Base, Shoulder, Elbow, Gripper)
  • Inertial Sensor
  • 2x Distance Sensors
  • Bumper Switch
  • Touchled RGB indicator

Camera System

USB Camera
  • Compatible with V4L2
  • Minimum 720p resolution
  • USB 2.0 or higher

Connectivity

Serial Connection
  • USB cable (Raspberry Pi ↔ VEX Brain)
  • Stable power supply for both devices
Ensure both the Raspberry Pi and VEX Brain have adequate power supplies. Insufficient power can cause communication failures and motor control issues.

​
Software Dependencies

The system requires Python 3.9+ with various libraries for computer vision, robotics control, and AI inference.

​
Key Dependencies

  • Python 3.9+ — Main programming language
  • pyserial 3.5 — Serial communication with VEX Brain
  • numpy 1.24.2 — Numerical computing for image processing
  • opencv-python 4.11.0.86 — Computer vision and camera interface
  • torch 2.6.0 — PyTorch deep learning framework
  • torchvision 0.21.0 — Computer vision models and transforms
  • ultralytics 8.3.77 — YOLO object detection framework
  • ncnn 1.0.20241226 — High-performance neural network inference
  • picamera2 0.3.25 — Raspberry Pi camera interface (if using Pi Camera)
  • gpiozero 2.0.1 — GPIO pin control
  • lgpio 0.2.2.0 — Low-level GPIO library
  • RPi.GPIO — Alternative GPIO library
  • SpeechRecognition 3.14.1 — Voice command processing
  • PyAudio 0.2.14 — Audio input/output
  • matplotlib 3.10.0 — Plotting and visualization
  • streamlit 1.41.1 — Web-based UI dashboard
  • plotly 6.0.0 — Interactive plots

​
Raspberry Pi Installation

1

Update System Packages

Start by updating your Raspberry Pi OS to ensure all system packages are current:
sudo apt update && sudo apt upgrade -y
This may take several minutes depending on your internet connection and installed packages.
2

Install System Dependencies

Install required system libraries for OpenCV, camera support, and GPIO:
sudo apt install -y \
  python3-pip \
  python3-dev \
  libopencv-dev \
  python3-opencv \
  libatlas-base-dev \
  libjasper-dev \
  libqtgui4 \
  libqt4-test \
  libhdf5-dev \
  libhdf5-serial-dev \
  libharfbuzz0b \
  libwebp6 \
  libtiff5 \
  libopenexr24 \
  libilmbase24 \
  libgstreamer1.0-0 \
  libavcodec-extra \
  libavformat58 \
  libswscale5 \
  v4l-utils
3

Clone the Repository

Clone the robotic arm system repository from GitHub:
cd ~
git clone https://github.com/AprendeIngenia/robotic_arm_system_beginners.git
cd robotic_arm_system_beginners
4

Create Virtual Environment (Recommended)

Create an isolated Python environment to avoid dependency conflicts:
python3 -m venv venv
source venv/bin/activate
Always activate the virtual environment before running the system:
source ~/robotic_arm_system_beginners/venv/bin/activate
5

Install Python Dependencies

Install all required Python packages from the requirements file:
pip install -r requirements.txt
PyTorch Installation: The requirements.txt includes PyTorch 2.6.0. If you encounter issues on Raspberry Pi, install the CPU-only version:
pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu
Installation may take 30-60 minutes on Raspberry Pi due to compiling some packages from source.
6

Configure Camera

Verify your camera is detected:
v4l2-ctl --list-devices
Test camera capture:
python3 -c "import cv2; cap = cv2.VideoCapture(0); print('Camera OK' if cap.isOpened() else 'Camera Error')"
If using the official Pi Camera, ensure the camera interface is enabled:
sudo raspi-config
# Navigate to: Interface Options → Camera → Enable
Then install picamera2:
sudo apt install -y python3-picamera2
7

Configure Serial Port

Enable serial hardware and disable serial console:
sudo raspi-config
# Navigate to: Interface Options → Serial Port
# Select: "No" for login shell over serial
# Select: "Yes" for serial port hardware
Verify the serial port:
ls -l /dev/ttyACM*
# or
ls -l /dev/ttyUSB*
The default serial port in the code is /dev/ttyACM1. You may need to adjust this in arm_system/communication/serial_manager.py:23 based on your setup.
Add your user to the dialout group for serial access:
sudo usermod -a -G dialout $USER
# Log out and log back in for this to take effect
8

Download YOLO Model (Optional)

The system uses YOLO11s for object detection. The model will be downloaded automatically on first run, or you can download it manually:
cd arm_system/perception/vision/detection/models
# The model files should be in yolo11s_ncnn_model/
To use your own trained YOLO model:
  1. Train your model using Ultralytics YOLO
  2. Export to NCNN format for Raspberry Pi optimization
  3. Place model files in arm_system/perception/vision/detection/models/
  4. Update the model path in model_loader.py
See the Model Export Guide for detailed instructions.

​
VEX Brain Setup

1

Install VEX Coding Studio

Download and install VEX Coding Studio on your development computer:
2

Connect VEX Brain to Computer

Connect your VEX IQ2 Brain to your development computer via USB.Verify the connection:
  • VEX Brain screen should show “Connected”
  • VEX Coding Studio should detect the brain
3

Open VEX Project

Open the VEX Brain code from the repository:
# Navigate to the VEX project directory
cd arm_system/vex_brain/
In VEX Coding Studio:
  1. Click File → Open
  2. Navigate to arm_system/vex_brain/
  3. Open the project
4

Configure Hardware Ports

Verify the port configuration matches your physical setup in src/main.py:
# Motor Configuration
self.base_motor = Motor(Ports.PORT1, True)
self.shoulder_motor = Motor(Ports.PORT2, True)
self.elbow_motor = Motor(Ports.PORT3, True)
self.gripper_motor = Motor(Ports.PORT4, True)

# Sensor Configuration
self.gripper_distance = Distance(Ports.PORT7)
self.touchled = Touchled(Ports.PORT8)
self.base_distance = Distance(Ports.PORT9)
self.bumper = Bumper(Ports.PORT10)
self.inertial = Inertial()  # Default port
Incorrect port mapping will cause hardware failures. Double-check your wiring against this configuration.
5

Upload Code to VEX Brain

Build and upload the program:
  1. In VEX Coding Studio, click Download button
  2. Wait for compilation and upload to complete
  3. Verify “Download Successful” message
  4. The program will start automatically
The VEX Brain will show a white LED when initialized and waiting for serial commands from the Raspberry Pi.
6

Test Hardware Components

Before connecting to Raspberry Pi, verify all hardware:Send a check_service command from the Raspberry Pi (or use VEX Brain screen buttons if configured):
# This will be done automatically by the Raspberry Pi
# The VEX Brain checks:
# - All motors installed
# - All sensors installed
# - Returns: {"state": "approved"} or error

​
System Verification

1

Connect Hardware

  1. Power on the VEX Brain
  2. Connect USB cable from Raspberry Pi to VEX Brain
  3. Verify the serial connection:
ls -l /dev/ttyACM*
# Should show: /dev/ttyACM0 or /dev/ttyACM1
2

Test Serial Communication

Run a basic serial test:
cd ~/robotic_arm_system_beginners
source venv/bin/activate
python3 -c "from arm_system.communication.serial_manager import CommunicationManager; \
            cm = CommunicationManager(port='/dev/ttyACM1'); \
            print('Connected' if cm.connect() else 'Failed')"
Expected output: Connected
3

Test Camera Capture

Verify the camera and image processor:
python3 -c "from arm_system.perception.vision.camera.main import CameraManager; \
            cam = CameraManager(); \
            img = cam.capture_image(); \
            print(f'Image saved: {img}')"
Check that an image file was created in the output directory.
4

Run Complete System Test

Start the main robot controller:
cd ~/robotic_arm_system_beginners/arm_system
python3 main.py
You should see:
=== Main menu ===
 [c] check service
 [s] safety service
 [n] scan service
 [p] pick & place service
 [q] exit
input command:
Test each command:
  • [c] Check Service: Verifies all hardware
  • [s] Safety Service: Moves arm to safe position
  • [n] Scan Service: Performs 360° scan with object detection
  • [p] Pick & Place: Executes pick and place operation
5

Verify Object Detection

Place a test object (apple, orange, or bottle) in front of the camera and run a scan:
# In the main menu, select [n] for scan
# The system will:
# 1. Rotate the base 360°
# 2. Detect objects using distance sensor
# 3. Capture images when objects are found
# 4. Run YOLO inference on captured images
# 5. Display detected objects with positions
Expected output:
=== objects scanned: (1) ===
Obj 1 -> angle: 145.0°, distance: 220mm, class: apple, conf: 0.87

​
Troubleshooting

Problem: Cannot connect to VEX BrainSolutions:
  1. Check USB cable connection
  2. Verify serial port name: 
    ls -l /dev/tty*
  3. Ensure user is in dialout group: 
    groups $USER
  4. Try different USB port on Raspberry Pi
  5. Update port in code if needed: 
    # arm_system/main.py:10
self.serial_manager = CommunicationManager(port='/dev/ttyACM0')
Problem: Camera initialization failsSolutions:
  1. Check camera connection and power
  2. List available cameras: 
    v4l2-ctl --list-devices
  3. Test with different camera index: 
    # Change camera_index in serial_manager.py:49
self.camera = CameraManager(camera_index=1)
  4. Verify camera permissions: 
    sudo usermod -a -G video $USER
Problem: Object detection model fails to loadSolutions:
  1. Verify model files exist: 
    ls -la arm_system/perception/vision/detection/models/
  2. Check PyTorch installation: 
    python3 -c "import torch; print(torch.__version__)"
  3. Reinstall ultralytics: 
    pip install --upgrade ultralytics
  4. Reduce confidence threshold if no detections: 
    # arm_system/communication/serial_manager.py:50
self.object_detect_model = ImageProcessor(confidence_threshold=0.35)
Problem: Motors not responding or moving incorrectlySolutions:
  1. Run check service to verify hardware: 
    # In main menu, press [c]
  2. Verify motor port connections match code
  3. Check VEX Brain battery level
  4. Recalibrate inertial sensor: 
    # Uncomment in vex_brain/src/main.py:283
# self.sensor.calibrate_inertial()
  5. Test individual motors in VEX Coding Studio
Problem: pip install fails for certain packagesSolutions:
  1. Update pip: 
    pip install --upgrade pip setuptools wheel
  2. Install problematic packages separately: 
    # Common issues:
pip install numpy==1.24.2
pip install opencv-python==4.11.0.86
  3. Use system packages where available: 
    sudo apt install python3-numpy python3-opencv
  4. For PyTorch issues on Raspberry Pi: 
    pip install torch torchvision --extra-index-url https://download.pytorch.org/whl/cpu

​
Configuration Files

# arm_system/communication/serial_manager.py
class CommunicationManager:
    def __init__(self, 
                 port: str='/dev/ttyACM1',  # Change if needed
                 baudrate: int = 115200,     # Match VEX Brain setting
                 camera_index: int = 0):     # Camera device index
        self.port = port
        self.baudrate = baudrate
        # ...

​
Next Steps

Now that your system is installed and verified, you can:

Quick Start Tutorial

Run your first complete scan and pick operation

Hardware Setup Guide

Learn about assembly and component details

API Documentation

Explore the complete API reference

Vision System Details

Deep dive into computer vision and YOLO models
Pro Tip: Join the community discussions and share your builds! Check the GitHub repository for issues, contributions, and updates.

Build docs developers (and LLMs) love

Get started for freeTalk to us