Browser Hand - OpenFang

What It Does

Browser Hand is an autonomous web browser agent that interacts with real websites on your behalf. It can navigate to URLs, click buttons and links, fill forms, read page content, take screenshots, and complete multi-step workflows—with mandatory approval for any purchase or payment. This is web automation with safety guardrails. Browser Hand will never spend your money without explicit confirmation.

Key Features

Multi-step workflows: Search products, fill forms, navigate complex sites
Purchase approval gate: Mandatory confirmation before any payment
Session persistence: Cookies and login state maintained across messages
Visual verification: Screenshot after every action (optional)
Error recovery: Handles page timeouts, missing elements, CAPTCHAs
Cross-platform: Works on Windows, macOS, Linux

Activation

Browser Hand requires Python 3 and Playwright. See the Requirements section below.

# Activate Browser Hand
openfang hand activate browser

# Activate with visible browser window (for debugging)
openfang hand activate browser --settings "headless=false"

Requirements

Install Python 3

macOS:

brew install python3

Windows:

winget install Python.Python.3.12

Linux (Debian/Ubuntu):

sudo apt install python3

Or download from python.org/downloads.Estimated time: 2-5 minutes

Install Playwright

pip3 install playwright
playwright install chromium

The second command downloads Chromium browser binaries.Estimated time: 3-5 minutes

Verify Installation

python3 -c "import playwright; print('Playwright installed')"

Should output: Playwright installed

Configuration Settings

headless

toggle

default:"true"

Run the browser without a visible window. Recommended for servers. Set to false for debugging (you’ll see the browser in action).

approval_mode

toggle

default:"true"

CRITICAL: Require explicit user confirmation before completing any purchase or payment. Should NEVER be disabled unless you fully trust the workflow.

max_pages_per_task

select

default:"20"

Maximum number of page navigations allowed per task to prevent runaway browsing:

10: Conservative
20: Balanced (default)
50: Thorough

default_wait

select

default:"auto"

How long to wait after clicking or navigating for the page to settle:

auto: Auto-detect (wait for DOM) - default
1: 1 second
3: 3 seconds

screenshot_on_action

toggle

default:"false"

Automatically take a screenshot after every click/navigate for visual verification. Useful for debugging but generates many files.

Required Tools

Browser Hand requires access to these tools (all built-in):

browser_navigate, browser_click, browser_type, browser_screenshot, browser_read_page, browser_close — Browser automation primitives
web_search, web_fetch — Fallback for non-interactive research
memory_store, memory_recall — State persistence
knowledge_add_entity, knowledge_add_relation, knowledge_query — Workflow tracking
schedule_create, schedule_list, schedule_delete — Scheduled workflows
file_write, file_read — Screenshot and data storage

System Prompt Overview

Browser Hand operates in 5 phases:

Understand the Task

Parses user request: What website(s) to visit? What information to find or action to perform? What are success criteria?

Navigate & Observe

Uses browser_navigate to go to target URL. Reads page content to understand layout. Identifies relevant elements (buttons, links, forms, search boxes).

Interact

Uses browser_click for buttons and links (CSS selectors or visible text). Uses browser_type for filling form fields. Uses browser_read_page after each action to see updated state.

MANDATORY Purchase/Payment Approval

CRITICAL RULE: Before completing ANY purchase, payment, or form submission involving money:

Summarize what you are about to buy/pay for
Show the total cost
List all items in cart
STOP and ask user for explicit confirmation
Only proceed after receiving clear approval

Report Results

After completing the task: summarize what was accomplished, include relevant details (prices, confirmation numbers), save important data to memory.

Usage Examples

Product Search

openfang chat browser
> "Search Amazon for USB-C cables under $20 and show me the top 3 results."

Browser Hand will:

Navigate to Amazon
Find the search box and type “USB-C cables”
Filter results by price (under $20)
Extract top 3 products with prices and ratings
Report results (does NOT add to cart)

Price Comparison

openfang chat browser
> "Compare the price of AirPods Pro on Amazon, Best Buy, and Apple.com. Report the lowest price."

Browser Hand will:

Navigate to each site
Search for “AirPods Pro”
Extract prices
Report comparison table

Form Filling

openfang chat browser
> "Fill out the contact form on example.com/contact with my info: John Doe, [email protected], subject: Demo Request."

Browser Hand will:

Navigate to the contact page
Fill each field
Screenshot before submitting (for verification)
Submit the form
Confirm submission success

E-commerce with Approval Gate

openfang chat browser
> "Add a pair of size 10 running shoes under $100 to cart on Nike.com."

Browser Hand will:

Search for running shoes on Nike.com
Filter by size 10 and price <$100
Select a product
Add to cart
STOP and report: “Found Nike Revolution 6 ($89.99, size 10). Ready to add to cart. Approve?”
Wait for your confirmation before proceeding

Browser Hand will NEVER click “Place Order”, “Pay Now”, “Confirm Purchase”, or any payment button without explicit user approval—even if approval_mode=false.

CSS Selector Reference

Browser Hand uses CSS selectors to identify page elements:

Basic Selectors

Selector	Description	Example
`#id`	By ID	`#checkout-btn`
`.class`	By class	`.add-to-cart`
`tag`	By element	`button`, `input`
`[attr=val]`	By attribute	`[data-testid="submit"]`

Form Selectors

Selector	Use Case
`input[type="email"]`	Email fields
`input[type="password"]`	Password fields
`input[type="search"]`	Search boxes
`input[name="q"]`	Google/search query
`textarea`	Multi-line text areas
`select[name="country"]`	Dropdown menus
`button[type="submit"]`	Submit buttons

E-commerce Selectors

Selector	Use Case
`.product-price`, `[data-price]`	Product prices
`.add-to-cart`, `#add-to-cart`	Add to cart buttons
`.cart-total`, `.order-total`	Cart total
`.quantity`, `input[name="quantity"]`	Quantity selectors
`.checkout-btn`, `#checkout`	Checkout buttons

Common Workflows

browser_navigate → login page
browser_type → email field: input[type="email"]
browser_type → password field: input[type="password"]
browser_click → login button: button[type="submit"]
browser_read_page → verify successful login

Search Pattern

browser_navigate → site homepage
browser_type → search box: input[type="search"]
browser_click → search button OR press Enter
browser_read_page → scan results
browser_click → desired result

E-commerce Pattern (with Approval)

Search for product
Click product from results
Select options (size, color, quantity)
Click "Add to Cart"
Navigate to cart
Verify cart contents & total
**STOP → Report to user, wait for approval**
Only proceed to checkout after approval

Dashboard Metrics

Browser Hand tracks three key metrics:

Pages Visited

Total pages navigated across all sessions.

Tasks Completed

Total workflows successfully completed.

Screenshots

Total screenshots captured.

View in the dashboard at http://localhost:4200/hands/browser.

Error Recovery

Element Not Found

Symptoms: “Element not found” errorRecovery:

Try alternative selector
Use visible text instead of CSS selector
Scroll page to bring element into view
Wait longer for page to load

Page Timeout

Symptoms: Page doesn’t load within timeoutRecovery:

Retry navigation
Check if URL is correct
Verify internet connection

CAPTCHA

Symptoms: CAPTCHA challenge appearsRecovery: Browser Hand cannot solve CAPTCHAs. It will inform you and stop. You must solve manually.

Cookie Consent

Security Rules

Browser Hand enforces these security rules:

NEVER stores passwords or credit card numbers in memory
NEVER auto-completes payments without user approval
NEVER navigates to URLs from untrusted sources without verification
NEVER fills credentials without user explicitly providing them
Always verifies domain before entering sensitive information
Warns immediately if suspicious or phishing-like content is detected

Best Practices

Use headless=false when testing new workflows. You’ll see exactly what Browser Hand sees.

Enable screenshot_on_action for high-stakes workflows (purchases, sensitive forms). Visual audit trail is invaluable.

Browser Hand’s session persists across messages in a conversation. You can log in once and stay logged in for multiple operations.

For repetitive workflows, create a saved prompt:

echo "Check Amazon for USB-C cables under $20, extract top 3 with prices" > workflow.txt
openfang chat browser < workflow.txt

Advanced Configuration

Custom Browser Path

Edit ~/.openfang/hands/browser.toml to use a specific Chromium/Chrome binary:

[agent.browser]
executable_path = "/path/to/chrome"

Proxy Configuration

[agent.browser]
proxy = "http://proxy.example.com:8080"

User Agent Override

[agent.browser]
user_agent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) ..."

Playwright Reference

Browser Hand uses Playwright under the hood. Here are the key operations:

Navigation

Clicking

# Click by CSS selector
page.click("#submit-button")

# Click by visible text
page.click("text=Submit")

# Click and wait for navigation
page.click("#submit-button")
page.wait_for_load_state()

Typing

# Type in a field
page.fill("input[name='email']", "[email protected]")

# Type with delay (human-like)
page.type("input[name='search']", "query", delay=100)

# Press Enter
page.press("input[name='search']", "Enter")

Reading Content

# Get page text
content = page.text_content("body")

# Get element attribute
price = page.get_attribute(".product-price", "data-price")

# Get all matching elements
products = page.query_selector_all(".product-item")

Screenshots

# Full page screenshot
page.screenshot(path="screenshot.png", full_page=True)

# Element screenshot
page.locator("#cart").screenshot(path="cart.png")

Limitations

Browser Hand cannot:

Solve CAPTCHAs
Handle file uploads (yet)
Interact with Flash or Java applets
Access content behind paywalls without credentials
Bypass bot detection on heavily protected sites (e.g., ticket sales)

Example Session

$ openfang chat browser
> Search Amazon for "mechanical keyboards" under $100 and show me the top 3 with reviews.

Browser Hand: Navigating to Amazon...
Browser Hand: Searching for "mechanical keyboards"...
Browser Hand: Applying price filter: max $100...
Browser Hand: Extracting top 3 results...

## Top 3 Mechanical Keyboards Under $100

1. **Keychron K2** - $89.99
   - Rating: 4.5/5 (2,341 reviews)
   - Hot-swappable, RGB backlight, wireless
   - Link: https://amazon.com/dp/B07QBVDDHB

2. **Royal Kludge RK61** - $59.99
   - Rating: 4.3/5 (1,892 reviews)
   - 60% compact, Bluetooth, hot-swappable
   - Link: https://amazon.com/dp/B0814JT9FF

3. **Redragon K552** - $39.99
   - Rating: 4.4/5 (18,453 reviews)
   - Tenkeyless, wired, mechanical switches
   - Link: https://amazon.com/dp/B016MAK38U

Task completed. Pages visited: 3 | Screenshots: 0

> Add the Keychron K2 to cart.

Browser Hand: Navigating to Keychron K2 product page...
Browser Hand: Clicking "Add to Cart"...
Browser Hand: Verifying cart...

**Purchase Approval Required**

Item in cart:
- Keychron K2 (RGB, Hot-swappable, Wireless)
- Price: $89.99
- Quantity: 1
- Subtotal: $89.99

Do you want to proceed to checkout? (yes/no)

> no

Browser Hand: Understood. Cart has been populated but I will not proceed to checkout. Item remains in cart for your manual review.

Session saved. You can resume this cart at: https://amazon.com/cart

Get Started

Core Concepts

Autonomous Hands

Configuration

Guides

Security

Deployment

​What It Does

​Key Features

​Activation

​Requirements

​Configuration Settings

​Required Tools

​System Prompt Overview

​Usage Examples

​Product Search

​Price Comparison

​Form Filling

​E-commerce with Approval Gate

​CSS Selector Reference

​Common Workflows

​Login Pattern

​Search Pattern

​E-commerce Pattern (with Approval)

​Dashboard Metrics

Pages Visited

Tasks Completed

Screenshots

​Error Recovery

​Security Rules

​Best Practices

​Advanced Configuration

​Custom Browser Path

​Proxy Configuration

​User Agent Override

​Playwright Reference

​Limitations

​Example Session

​Next Steps

Lead Hand

Collector Hand

Build docs developers (and LLMs) love

What It Does

Key Features

Activation

Requirements

Configuration Settings

Required Tools

System Prompt Overview

Usage Examples

Product Search

Price Comparison

Form Filling

E-commerce with Approval Gate

CSS Selector Reference

Common Workflows

Login Pattern

Search Pattern

E-commerce Pattern (with Approval)

Dashboard Metrics

Error Recovery

Security Rules

Best Practices

Advanced Configuration

Custom Browser Path

Proxy Configuration

User Agent Override

Playwright Reference

Limitations

Example Session

Next Steps