Common errors
Accessibility permission denied
Error message:
Accessibility access denied. Enable in System Settings > Privacy & Security > Accessibility
Cause: Your terminal or IDE doesn’t have Accessibility permissions.
Solution:
Open System Settings
open "x-apple.systempreferences:com.apple.preference.security?Privacy_Accessibility"
Or manually: System Settings > Privacy & Security > Accessibility
Add your terminal
Click the + button and add:
Terminal.app
iTerm.app
VS Code
Your IDE
If the app is already in the list, try toggling it off and on.
Restart your terminal
Quit and relaunch your terminal completely for permissions to take effect.
Test access
If this works, permissions are correctly set.
See AXEngine.swift:378-386 for the access check implementation.
Changes to Accessibility permissions require a full restart of the app. Closing a window is not enough.
App not found
Error message:
Cause: Typo in app name, or app is not running.
Solutions:
This lists all running GUI applications. Verify the exact name.
agent-native’s open command launches apps if they’re not running: agent-native open Slack # Launches if needed
If the app name is ambiguous, use the bundle ID: agent-native open com.tinyspeck.slackmacgap
Find bundle IDs with: agent-native apps --format json | jq '.[] | {name, bundleId}'
App names are case-insensitive, but try exact capitalization: agent-native open "System Settings" # Correct
agent-native open "system settings" # Also works
See AXEngine.swift:56-63 for app finding logic.
Ref not resolving
Error message:
Ref @n42 not found. It may be stale. Re-snapshot to get fresh refs.
Cause: The UI structure changed since the last snapshot, invalidating the ref.
Why this happens:
You clicked a navigation element (changed pane/view)
App loaded new content
A modal opened or closed
Window was resized
Solution:
# Re-snapshot to get new refs
agent-native snapshot "System Settings" -i
# Now use the new refs
agent-native click @n5
Always re-snapshot after actions that change the UI structure.
Best practice:
# Take snapshot
agent-native snapshot App -i > /tmp/snap1.txt
# Interact with refs from snap1.txt
agent-native click @n5
# Re-snapshot after UI change
agent-native snapshot App -i > /tmp/snap2.txt
# Use refs from snap2.txt for next interactions
agent-native click @n10
See RefStore.swift for ref storage implementation.
Element not found
Error message:
Element not found: AXButton[@title="Submit"]
Cause: The element doesn’t exist, or is hidden/disabled.
Debugging steps:
Check if element exists
# Full snapshot (no filters)
agent-native snapshot App > full-tree.txt
# Search for your element
grep -i "submit" full-tree.txt
Check interactive-only flag
The -i flag filters to interactive elements. Try without it: agent-native snapshot App # Full tree
Increase depth
Default depth is 8. Some deep elements need more: agent-native snapshot App -i -d 15
Wait for element to appear
Elements may load asynchronously: agent-native wait App --title "Submit" --timeout 5
Use find command
# Search by title
agent-native find App --title "Submit"
# Search by role
agent-native find App --role AXButton
# Combine filters
agent-native find App --title "Submit" --role AXButton
See FindCommand.swift and AXEngine.swift:278-356 for element finding logic.
Element not responding
Error message:
Action 'AXPress' failed on: /AXWindow/AXButton
Cause: Element is disabled, hidden, or doesn’t support the action.
Solutions:
Check if element is enabled
agent-native is enabled @n10
If this returns false, the element can’t be interacted with.
Inspect element attributes
agent-native inspect @n10
Check:
AXEnabled: Should be true
Actions: Should include AXPress or relevant action
AXPosition: Element should be on-screen
# Instead of click
agent-native action @n10 AXPress
# Or focus then press space
agent-native focus @n10
agent-native key App space
Wait for element to be ready
# Wait 1 second
sleep 1
agent-native click @n10
See InspectCommand.swift for element inspection.
Timeout waiting for element
Error message:
Timeout: Element with title="Submit" not found after 5s
Cause: Element never appeared within the timeout period.
Solutions:
agent-native wait App --title "Submit" --timeout 10
Take a snapshot to see what’s actually there: agent-native snapshot App -i
Your filters might be too specific: # Try less specific filter
agent-native wait App --role AXButton --timeout 5
Check page/pane loaded correctly
# Check window title
agent-native get title App
# Take screenshot
agent-native screenshot App /tmp/debug.png
See WaitCommand.swift for wait implementation.
Debugging techniques
Capture full snapshots
When debugging, capture the full tree without filters:
# Save to file
agent-native snapshot App > /tmp/full-tree.txt
# Search for your element
grep -i "keyword" /tmp/full-tree.txt
# View in editor
code /tmp/full-tree.txt
Use JSON output for parsing
# Get JSON snapshot
agent-native snapshot App -i --json > snapshot.json
# Query with jq
cat snapshot.json | jq '.[] | select(.title == "Submit")'
# Extract specific ref
REF = $( cat snapshot.json | jq -r '.[] | select(.title == "Submit") | .ref' )
agent-native click "@ $REF "
Compare snapshots
See how UI changes between actions:
# Before action
agent-native snapshot App -i > /tmp/before.txt
# Perform action
agent-native click @n5
sleep 1
# After action
agent-native snapshot App -i > /tmp/after.txt
# Compare
diff /tmp/before.txt /tmp/after.txt
Use screenshots for visual debugging
# Capture current state
agent-native screenshot App /tmp/state1.png
# Perform actions
agent-native click @n5
# Capture after state
agent-native screenshot App /tmp/state2.png
# Compare visually
open /tmp/state1.png /tmp/state2.png
Inspect element details
# Full element inspection
agent-native inspect @n10
# Or by filter
agent-native inspect App --title "Submit"
Output includes:
Role and subrole
All attributes (title, label, value, etc.)
Available actions
Position and size
Enabled/focused state
Test commands incrementally
Build complex workflows step by step:
# Step 1: Open app
agent-native open App
echo "Opened app"
sleep 1
# Step 2: Snapshot
agent-native snapshot App -i > /tmp/snap.txt
echo "Snapshot captured"
# Step 3: Find element
REF = $( grep -i "submit" /tmp/snap.txt | grep -o 'ref=n[0-9]*' | sed 's/ref=//' | head -1 )
echo "Found ref: @ $REF "
# Step 4: Click
if [[ -n " $REF " ]]; then
agent-native click "@ $REF "
echo "Clicked"
else
echo "Element not found"
exit 1
fi
Enable verbose output
While agent-native doesn’t have a verbose flag, you can wrap commands in debug scripts:
#!/usr/bin/env bash
set -x # Print commands as they execute
set -euo pipefail # Exit on error
agent-native open App
agent-native snapshot App -i
agent-native click @n5
Snapshots are slow
Cause: Deep trees with many elements take time to traverse.
Solutions:
agent-native snapshot App -i -d 5
Remove empty structural elements: agent-native snapshot App -i -c
Use interactive-only flag
Always use -i to filter to interactive elements: agent-native snapshot App -i
Use filter-based commands
Skip snapshots when you know what you’re looking for: agent-native click App --title "Submit"
See SnapshotCommand.swift:26-33 for interactive role filtering.
App becomes unresponsive
Cause: Too many rapid commands can overwhelm the app.
Solution:
Add delays between commands:
agent-native click @n5
sleep 0.5
agent-native fill @n10 "text"
sleep 0.3
agent-native click @n15
macOS version compatibility
Issue: Some features require macOS 13+.
Check version:
Solution: Upgrade to macOS 13 (Ventura) or higher.
Accessibility API changes
Apple occasionally changes Accessibility APIs between macOS versions.
Solution:
Update agent-native to the latest version:
brew upgrade agent-native
Check for known issues on GitHub
Getting help
Check GitHub Issues Search for similar problems and solutions
Run tests Verify your installation: cd /path/to/agent-native
make test-quick
Check accessibility Use macOS Accessibility Inspector:
Open Xcode
Xcode > Open Developer Tool > Accessibility Inspector
Inspect app elements manually
File a bug Include:
agent-native version (agent-native --version)
macOS version (sw_vers)
Full error message
Minimal reproduction steps
Quick reference
Essential debugging commands
# List running apps
agent-native apps
# Full tree snapshot
agent-native snapshot App > debug.txt
# Interactive elements only
agent-native snapshot App -i
# JSON output
agent-native snapshot App -i --json
# Find elements
agent-native find App --title "Submit"
# Inspect element
agent-native inspect @n10
# Check element state
agent-native is enabled @n10
agent-native is focused @n10
# Get element attributes
agent-native get text @n10
agent-native get value @n10
# Screenshot
agent-native screenshot App /tmp/debug.png
# Window title
agent-native get title App
Common error patterns
Error Likely Cause Quick Fix ”access denied” No Accessibility permission Add terminal to System Settings > Accessibility ”App not found” Typo or not running Check agent-native apps ”Ref @nX not found” UI changed Re-snapshot ”Element not found” Wrong filter or not loaded Try snapshot without -i, or increase -d ”Action failed” Element disabled Check is enabled, wait longer ”Timeout” Element never appeared Increase --timeout, verify filter
Next steps
System Settings guide Learn System Settings automation patterns
Electron apps guide Automate Slack, Discord, VS Code
Safari automation Interact with web content
API reference Complete command reference