Capture the accessibility tree of an application window as structured JSON. Each interactive element is assigned a deterministic reference ID (@e1, @e2, etc.) that can be used in subsequent action commands.
Usage agent-desktop snapshot [OPTIONS]
Parameters --app
string
default: "focused app"
Filter to a specific application by name (e.g., “Finder”, “Safari”).
Filter to a specific window ID from list-windows output.
Maximum tree depth to traverse. Prevents infinite recursion on deeply nested UIs.
Include pixel bounds (x, y, width, height) for each element in the tree.
Only include interactive elements (buttons, text fields, etc.) in the tree. Static labels and containers are omitted.
Collapse single-child unnamed nodes to reduce tree depth and improve readability.
Surface type to snapshot. Options: window, focused, menu, menubar, sheet, popover, alert
Response Name of the application that was snapshotted.
Information about the window that was captured. Unique window identifier.
Total number of interactive elements that received ref IDs.
Root accessibility node containing the entire element tree. Reference ID for this element (e.g., @e1, @e2). Only present on interactive elements.
Accessibility role (e.g., button, textfield, checkbox, window, group).
Accessible name or label for the element.
Current value (for text fields, sliders, etc.).
Accessibility description or help text.
Array of state strings: checked, focused, enabled, disabled, expanded, hidden.
Pixel bounds (only if --include-bounds is set). Array of child accessibility nodes (recursive structure).
Examples Basic snapshot with interactive elements only agent-desktop snapshot --app Finder -i
{ "version" : "1.0" , "ok" : true , "command" : "snapshot" , "data" : { "app" : "Finder" , "window" : { "id" : "w-4521" , "title" : "Documents" }, "ref_count" : 14 , "tree" : { "role" : "window" , "name" : "Documents" , "children" : [ { "ref_id" : "@e1" , "role" : "button" , "name" : "Back" , "states" : [ "enabled" ] }, { "ref_id" : "@e2" , "role" : "textfield" , "name" : "Search" , "value" : "" , "states" : [ "enabled" ] }, { "ref_id" : "@e3" , "role" : "button" , "name" : "List View" , "states" : [ "enabled" , "checked" ] } ] } } }
Snapshot with bounds included agent-desktop snapshot --app "System Settings" --include-bounds
{ "version" : "1.0" , "ok" : true , "command" : "snapshot" , "data" : { "app" : "System Settings" , "window" : { "id" : "w-8892" , "title" : "General" }, "ref_count" : 42 , "tree" : { "role" : "window" , "name" : "General" , "bounds" : { "x" : 100 , "y" : 200 , "width" : 800 , "height" : 600 }, "children" : [ { "ref_id" : "@e1" , "role" : "checkbox" , "name" : "Show scroll bars" , "states" : [ "enabled" , "checked" ], "bounds" : { "x" : 150 , "y" : 300 , "width" : 200 , "height" : 20 } } ] } } }
agent-desktop snapshot --surface menu --app TextEdit
{ "version" : "1.0" , "ok" : true , "command" : "snapshot" , "data" : { "app" : "TextEdit" , "window" : { "id" : "menu-1" , "title" : "File" }, "ref_count" : 8 , "tree" : { "role" : "menu" , "name" : "File" , "children" : [ { "ref_id" : "@e1" , "role" : "menuitem" , "name" : "New" , "states" : [ "enabled" ] }, { "ref_id" : "@e2" , "role" : "menuitem" , "name" : "Open…" , "states" : [ "enabled" ] }, { "ref_id" : "@e3" , "role" : "menuitem" , "name" : "Save" , "states" : [ "enabled" ] } ] } } }
Error Cases Application not found { "version" : "1.0" , "ok" : false , "command" : "snapshot" , "error" : { "code" : "APP_NOT_FOUND" , "message" : "No windows found for app 'InvalidApp'" } }
Window not found { "version" : "1.0" , "ok" : false , "command" : "snapshot" , "error" : { "code" : "WINDOW_NOT_FOUND" , "message" : "Window 'w-99999' not found" } }
Permission denied { "version" : "1.0" , "ok" : false , "command" : "snapshot" , "error" : { "code" : "PERM_DENIED" , "message" : "Accessibility permission not granted" , "suggestion" : "Open System Settings > Privacy & Security > Accessibility and add your terminal" } }
Notes
Refs are allocated in depth-first order : @e1, @e2, @e3, etc.
Only interactive roles receive refs: button, textfield, checkbox, link, menuitem, tab, slider, combobox, treeitem, cell, radiobutton, etc.
Static elements (labels, groups, containers) appear in the tree for context but do not receive refs unless --interactive-only is used.
Refs are deterministic within a snapshot but NOT stable across snapshots if the UI changes.
The RefMap is stored at ~/.agent-desktop/last_refmap.json with 0o600 permissions.
Each snapshot replaces the refmap file entirely (atomic write via temp + rename).
See Also
find - Search for elements by role, name, or valueget - Read element propertiesis - Check element stateslist-surfaces - List available surfaces for an app 
