Robot Kit API
Complete API reference for the six core robot tools.
Tool Inputs Outputs Hardware Latency driveaction, distance/angle confirmation Motors 100ms-10s lookaction, prompt (optional) image description Camera 2-5s listenduration (seconds) transcribed text Microphone 1-10s speaktext confirmation Speaker 1-5s senseaction, direction sensor readings LIDAR, PIR, ultrasonic 100ms-1s emoteexpression, sound (optional) confirmation LED matrix, buzzer 100-500ms
drive
Omni-directional motor control with safety integration.
Parameters
{
"action" : "forward" | "backward" | "strafe_left" | "strafe_right" | "rotate_left" | "rotate_right" | "stop" ,
"distance" : 1.0 , // meters (for linear movement)
"angle" : 90 , // degrees (for rotation)
"speed" : 0.5 // 0.0-1.0, optional, defaults to config max_speed
}
Examples
{
"action" : "forward" ,
"distance" : 1.5
}
Response
{
"success" : true ,
"output" : "Moved forward 1.5 meters"
}
Backends
The drive tool supports multiple motor control backends:
Backend Description Configuration mock Logs commands, no hardware backend = "mock"serial Arduino/motor controller backend = "serial", serial_port = "/dev/ttyACM0"ros2 ROS2 nav stack backend = "ros2", ros2_topic = "/cmd_vel"gpio Direct GPIO control backend = "gpio" (requires peripheral-rpi feature)
Safety Integration
When wrapped with SafeDrive, the tool:
Pre-movement check : Queries sensors for obstacles
Real-time monitoring : Adjusts speed based on proximity
Collision response : Reverses if bump sensor triggered
Watchdog : Auto-stops after timeout
E-stop : Immediate halt on hardware button press
Always use SafeDrive wrapper in production. Direct DriveTool bypasses safety checks.
look
Capture images and generate AI descriptions using vision models.
Parameters
{
"action" : "capture" | "describe" | "find" ,
"prompt" : "a red ball" , // optional, for "find" action
"save_path" : "/tmp/image.jpg" // optional, defaults to temp file
}
Examples
Describe Scene
Find Object
Just Capture
Response
{
"success" : true ,
"output" : "I see a couch, a TV on the wall, and some toys on the floor. The room is well-lit with natural light from a window."
}
Vision Models
Supported models (via Ollama):
Model Size Speed Accuracy Recommended For moondream 1.7B Fast Good Pi 4/5, general use llava 7B Slow Better Pi 5 only, detailed scenes bakllava 7B Slow Better Alternative to llava
Configure in robot.toml:
[ camera ]
vision_model = "moondream"
ollama_url = "http://localhost:11434"
listen
Transcribe speech to text using Whisper.cpp.
Parameters
{
"duration" : 5 , // seconds to record
"language" : "en" // optional, defaults to "en"
}
Examples
Listen 5 Seconds
Listen 10 Seconds
Response
{
"success" : true ,
"output" : "Hey Buddy, can you help me find my toy?"
}
Whisper Models
Model Size Speed Accuracy RAM Usage tiny 75 MB Fastest Fair 200 MB base 142 MB Fast Good 400 MB small 466 MB Medium Better 1 GB medium 1.5 GB Slow Best 2.5 GB
Recommendation: base for Pi 4/5
Configure in robot.toml:
[ audio ]
whisper_model = "base"
whisper_path = "/usr/local/bin/whisper-cpp"
speak
Text-to-speech using Piper TTS.
Parameters
{
"text" : "Hello! I'm ready to play!" ,
"voice" : "en_US-lessac-medium" // optional, defaults to config
}
Examples
Simple Speech
Custom Voice
{
"text" : "I found the ball under the couch!"
}
Response
{
"success" : true ,
"output" : "Spoke: Hello! I'm ready to play!"
}
Piper Voices
Available voices:
Voice Gender Style Speed en_US-lessac-medium Male Clear Medium en_US-amy-medium Female Friendly Medium en_US-ryan-high Male Energetic Fast en_GB-alan-medium Male British Medium
Download from Piper Voices
sense
Query sensors: LIDAR, motion (PIR), ultrasonic distance.
Parameters
{
"action" : "scan" | "motion" | "distance" ,
"direction" : "front" | "all" , // for scan action
"max_range" : 5.0 // meters, optional
}
Examples
LIDAR Scan
Motion Detection
Distance Check
{
"action" : "scan" ,
"direction" : "front"
}
Response
{
"success" : true ,
"output" : "Front: clear (2.3m), Left: obstacle at 0.8m, Right: clear (3.1m)"
}
Sensor Types
Sensor Range Accuracy Use Case RPLidar A1 12m, 360° ±5cm Obstacle avoidance, mapping HC-SR04 4m, narrow ±1cm Front distance check PIR Motion 7m, 120° Binary Detect people/animals
Configure in robot.toml:
[ sensors ]
lidar_port = "/dev/ttyUSB0"
lidar_type = "rplidar"
motion_pins = [ 17 , 27 ]
ultrasonic_pins = [ 23 , 24 ]
emote
Display LED expressions and play sound effects.
Parameters
{
"expression" : "happy" | "sad" | "excited" | "thinking" | "sleepy" | "surprised" ,
"sound" : "beep" | "chime" | "buzz" // optional
}
Examples
Happy Face
Excited with Sound
Thinking
{
"expression" : "happy"
}
Response
{
"success" : true ,
"output" : "Displayed: happy face"
}
LED Patterns
8x8 LED matrix patterns (WS2812B):
Happy: Sad: Excited: Thinking:
● ● ● ● ● ● ● ●
● ● ● ● ● ● ○ ●
●●●●● ●●●●● ●●●●● ●●●
Error Handling
All tools return a ToolResult:
pub struct ToolResult {
pub success : bool ,
pub output : String ,
pub error : Option < String >,
}
Common Errors
{
"success" : false ,
"output" : "" ,
"error" : "Obstacle detected at 0.2m, stopping"
}
{
"success" : false ,
"output" : "" ,
"error" : "Camera device /dev/video0 not found"
}
{
"success" : false ,
"output" : "" ,
"error" : "E-stop pressed, all movement disabled"
}
Usage Examples
Hide and Seek
use zeroclaw_robot_kit ::* ;
#[tokio :: main]
async fn main () {
let config = RobotConfig :: default ();
let tools = create_tools ( & config );
// Game sequence
speak ( "I'll count to 20. Go hide!" );
emote ( "excited" );
tokio :: time :: sleep ( Duration :: from_secs ( 20 )) . await ;
speak ( "Ready or not, here I come!" );
loop {
let obstacles = sense ( "scan" , "all" );
drive ( "forward" , 1.0 );
let scene = look ( "find" , "a child hiding" );
if scene . contains ( "child" ) {
speak ( "Found you!" );
emote ( "happy" , Some ( "chime" ));
break ;
}
}
}
Voice-Controlled Movement
loop {
speak ( "Where should I go?" );
let command = listen ( 5 ) . await ? ;
match command . as_str () {
cmd if cmd . contains ( "forward" ) => drive ( "forward" , 1.0 ),
cmd if cmd . contains ( "back" ) => drive ( "backward" , 0.5 ),
cmd if cmd . contains ( "left" ) => drive ( "rotate_left" , 90 ),
cmd if cmd . contains ( "right" ) => drive ( "rotate_right" , 90 ),
cmd if cmd . contains ( "stop" ) => {
drive ( "stop" , 0.0 );
break ;
},
_ => speak ( "I didn't understand that" ),
}
}
Autonomous Patrol
loop {
// Check for obstacles
let scan = sense ( "scan" , "front" );
if scan . min_distance < 0.5 {
speak ( "Obstacle ahead, turning" );
drive ( "rotate_right" , 90 );
continue ;
}
// Check for motion
let motion = sense ( "motion" );
if motion . detected {
speak ( "I see movement!" );
let scene = look ( "describe" );
speak ( & format! ( "I see: {}" , scene ));
}
// Continue patrol
drive ( "forward" , 2.0 );
}
Safety Monitor
The SafetyMonitor runs as an independent task:
use zeroclaw_robot_kit :: { SafetyMonitor , SafeDrive , DriveTool };
let config = RobotConfig :: default ();
// Create safety monitor
let ( safety , safety_rx ) = SafetyMonitor :: new ( config . safety . clone ());
let safety = Arc :: new ( safety );
// Start sensor feed
tokio :: spawn ( async move {
loop {
let reading = get_lidar_data () . await ;
safety . update_sensor ( reading ) . await ;
}
});
// Wrap drive with safety
let drive = Arc :: new ( DriveTool :: new ( config . clone ()));
let safe_drive = SafeDrive :: new ( drive , safety . clone ());
// Now all drive commands go through safety checks
safe_drive . execute ( json! ({ "action" : "forward" , "distance" : 2.0 })) . await ;
Safety Events
pub enum SafetyEvent {
ObstacleDetected { distance : f64 , angle : i32 },
BumpSensorTriggered { sensor : String },
EStopPressed ,
SensorTimeout ,
SpeedLimited { from : f64 , to : f64 , reason : String },
MovementBlocked { reason : String },
}
Troubleshooting
Drive not working
# Check backend
grep backend ~/.zeroclaw/robot.toml
# Serial: verify port
ls /dev/ttyACM *
# ROS2: check topic
ros2 topic list | grep cmd_vel
# Test in mock mode
sed -i 's/backend = ".*"/backend = "mock"/' ~/.zeroclaw/robot.toml
Camera not found
# List cameras
ls /dev/video *
ffmpeg -f v4l2 -list_formats all -i /dev/video0
# Check permissions
sudo usermod -aG video $USER
Ollama vision fails
# Check Ollama is running
curl http://localhost:11434/api/tags
# Test vision model
ollama run moondream "Describe this image: /tmp/test.jpg"
# Use smaller model
sed -i 's/vision_model = ".*"/vision_model = "moondream"/' ~/.zeroclaw/robot.toml
Safety prevents movement
# Check sensor readings
tail -f /tmp/zeroclaw_sensors.fifo
# Increase safety thresholds (testing only!)
sed -i 's/min_obstacle_distance = 0.3/min_obstacle_distance = 0.15/' ~/.zeroclaw/robot.toml
# Disable safety (dangerous!)
sed -i 's/predict_collisions = true/predict_collisions = false/' ~/.zeroclaw/robot.toml
Next Steps
Setup Guide Complete hardware and software setup
Overview Architecture and features
Examples Full example programs
Source Code View robot kit source