​

Theoretical Concepts

​

Learning Objectives

• Core robotics principles and coordinate systems
• Computer vision fundamentals and object detection
• Control system architectures
• Communication protocol design principles
• Real-time system considerations

​

1. Robotics Fundamentals

​

Coordinate Systems and Kinematics

• World frame: Fixed reference point in the environment
• Robot base frame: Origin at the robot’s mounting point
• End-effector frame: Tool center point (TCP)

• Each motor/actuator has an angular position
• Robot configuration defined by joint angles: θ₁, θ₂, θ₃…

​

Degrees of Freedom (DOF)

• 3-DOF: Position in 3D space (X, Y, Z)
• 6-DOF: Position + orientation (roll, pitch, yaw)
• VEX IQ robots typically have 3-4 DOF

​

Robot Control Architecture

​

2. Computer Vision Basics

​

Image Representation

• Grayscale: Single channel (0-255 intensity)
• RGB Color: Three channels (Red, Green, Blue)
• Resolution: Width × Height pixels

# Example: Image dimensions
import cv2
image = cv2.imread('object.jpg')
height, width, channels = image.shape  # e.g., (480, 640, 3)

​

Object Detection Fundamentals

1. Classification: What is the object? (apple, orange, bottle)
2. Localization: Where is it? (bounding box coordinates)
3. Confidence Score: How certain is the detection? (0.0 to 1.0)

• Rectangular regions containing detected objects
• Represented as: (x1, y1, x2, y2) or (x, y, width, height)
• x1, y1: Top-left corner
• x2, y2: Bottom-right corner

​

Deep Learning for Vision

• Learn features automatically from training data
• Hierarchical layers: edges → shapes → objects
• Trained using labeled datasets

• Specialized for image processing
• Convolutional layers detect spatial patterns
• Pooling layers reduce dimensionality

• Single-stage detector (fast inference)
• Divides image into grid cells
• Each cell predicts bounding boxes and class probabilities
• Output: List of detections with class, confidence, and location

​

Model Inference Pipeline

# Conceptual inference flow
image = capture_frame()           # Get image from camera
preprocessed = resize_and_normalize(image)  # Prepare for model
results = model.predict(preprocessed)       # Run inference
detections = parse_results(results)         # Extract bounding boxes
best = select_highest_confidence(detections)  # Choose target

​

3. Control Systems

​

Feedback Control Loop

• Setpoint: Desired target position
• Controller: Calculates corrective actions
• Plant: The physical robot
• Feedback: Sensor measurements of actual position
• Error: Difference between target and actual position

​

Control Strategies

• Send commands without feedback
• Simple but less accurate
• Example: “Move motor forward for 2 seconds”

• Use sensor feedback to correct errors
• More accurate and robust
• Example: “Move arm until camera detects object centered”

• P: Correct based on current error
• I: Correct based on accumulated past errors
• D: Correct based on rate of error change

​

4. Serial Communication Concepts

​

Why Serial Communication?

• Simple hardware: Fewer wires than parallel communication
• Long distances: More reliable over extended cables
• Universal: Supported by most embedded systems

​

UART Protocol

• Baud Rate: Bits per second (common: 9600, 115200)
• Data Bits: Usually 8 bits per byte
• Stop Bits: Signal end of byte (usually 1)
• Parity: Error checking (often none)

​

Communication Patterns

​

Message Framing

Hello\n    # Newline character marks message end

5:Hello     # Number indicates message length

HELLO       # Always exactly 5 characters

​

5. Real-Time System Considerations

​

Latency and Timing

• Camera capture: 30-60ms (30 FPS)
• Model inference: 50-200ms (depends on model size)
• Serial transmission: 1-5ms
• Motor response: 10-100ms

​

Threading and Concurrency

• Main Thread: Capture frames and process vision
• Communication Thread: Listen for incoming serial messages
• Control Thread: Send commands and monitor status

from threading import Thread

# Separate threads for reading and writing
read_thread = Thread(target=serial_read_loop)
write_thread = Thread(target=vision_processing_loop)

read_thread.start()
write_thread.start()

​

Error Handling

• Connection Loss: Retry serial connection automatically
• Invalid Data: Validate and discard malformed messages
• Timeout: Don’t wait forever for responses
• Sensor Failure: Continue operating with degraded performance

​

6. System Integration

​

Component Interaction

1. Camera captures frame → Raspberry Pi
2. YOLO model processes image → detections
3. Best detection formatted as JSON → serial TX
4. VEX Brain receives and parses message
5. Motion planning executes arm movement
6. Status message sent back → Raspberry Pi

​

Design Principles

class ModelLoader:    # Only loads models
class DetectionModel: # Only runs inference  
class SerialComm:     # Only handles communication

class DetectionModelInterface(ABC):
    @abstractmethod
    def inference(self, image: np.ndarray):
        pass

try:
    result = model.predict(image)
except Exception as e:
    log.error(f'Inference failed: {e}')
    return default_result

​

Knowledge Check

1. What is the difference between forward and inverse kinematics?
2. How does YOLO differ from traditional object detection methods?
3. Why is full-duplex communication important for robotics?
4. What causes latency in vision-based robot control?
5. Why do we use threading in serial communication?

​

Next Steps