Skip to main content
agent-native gives AI agents structured control over macOS native applications through the Accessibility (AX) tree.

​
What is agent-native?

agent-native is a macOS CLI tool that exposes native application UI elements to AI agents and automation scripts. Inspired by agent-browser from Vercel Labs, it brings the same level of structured control to desktop apps that agent-browser provides for web pages.
Where agent-browser uses Chrome DevTools Protocol (CDP) and the DOM for web automation, agent-native uses macOS Accessibility APIs and the AX tree for native app automation.

​
Key capabilities

Discovery

List running apps, open applications, and explore their UI structure through the Accessibility tree

Snapshot

Capture interactive elements with persistent references (@n1, @n2) for reliable interaction

Interaction

Click buttons, fill text fields, check toggles, select from dropdowns using simple commands

State inspection

Read text, check element state, get attributes, and verify UI conditions

​
Core workflow

The typical agent-native workflow follows a simple pattern:
1

Launch the app

agent-native open "System Settings"
2

Take a snapshot

agent-native snapshot "System Settings" -i
This generates persistent references (like @n5, @n3) for interactive elements
3

Interact using refs

agent-native click @n5
agent-native fill @n3 "Wi-Fi"
4

Re-snapshot after changes

When the UI changes (new dialog, different pane), take another snapshot to get fresh references

​
When to use agent-native

agent-native is ideal for:
  • AI agent automation: Give LLMs structured access to desktop apps with JSON output
  • GUI testing: Automate macOS app testing through the Accessibility APIs
  • Desktop workflows: Chain together actions across multiple native applications
  • System administration: Automate System Settings and other macOS utilities
  • Accessibility tooling: Build tools that inspect and interact with app UI elements
agent-native requires macOS 13+ (Ventura) and Accessibility permissions for your terminal. See the installation guide for setup instructions.

​
Agent mode

Every command supports --json output for structured data suitable for LLM tool use: 
agent-native snapshot "System Settings" -i --json | jq '.[0].ref'
agent-native get text @n1 --json
agent-native is enabled @n5 --json

​
Next steps

Installation

Install via Homebrew or build from source

Quick start

Complete tutorial from installation to first automation

Build docs developers (and LLMs) love

Get started for freeTalk to us