veto scan - Veto

The veto scan command discovers tools in your codebase and checks which ones have policy rules defined.

Syntax

veto scan [options]

Description

Scans your codebase to:

Discover tools from source code (TypeScript, Python)
Load policy rules from veto/rules/
Match tools to rules and identify coverage gaps
Suggest policies for uncovered tools (optional)
Report coverage as text or JSON

Options

Fail on Uncovered

--fail-uncovered

Exit with status code 1 if any tools are uncovered (useful for CI). Example:

veto scan --fail-uncovered

Suggest Policies

--suggest

Include AI-generated policy suggestions for uncovered tools. Example:

veto scan --suggest

Include Examples

--include-examples

Include examples/ directory in tool discovery. Example:

veto scan --include-examples

Include Tests

--include-tests

Include test/, tests/, __tests__/ directories in tool discovery. Example:

veto scan --include-tests

Format

--format <text|json>

Output format:

text - Human-readable (default)
json - Machine-readable JSON

Example:

veto scan --format json

Examples

Basic Scan

veto scan

Output:

Veto Scan Coverage Audit
========================

Project directory: /path/to/project
Rules directory: /path/to/project/veto/rules
Rules loaded: 8 (global: 2)
Framework hints: langchain, vercel-ai
Coverage: 8/12 (66.7%)

Discovered tools:
  [COVERED] transfer_funds(amount, recipient, currency)
    locations: src/tools/financial.ts
    sources: source-ts, policy
  [COVERED] approve_invoice(amount, vendor_id)
    locations: src/tools/invoices.ts
    sources: source-ts, policy
  [UNCOVERED] send_email(to, subject, body)
    locations: src/tools/email.ts
    sources: source-ts
  [UNCOVERED] execute_shell_command(command)
    locations: src/tools/shell.ts
    sources: source-ts

Scan with Suggestions

veto scan --suggest

Output:

Veto Scan Coverage Audit
========================

...

Suggested starter rules:

  send_email (@veto/communication)
  Rationale: Tool name matches communication keywords (e.g. email/message/notify).
  Snippet:
    rules:
      - id: guard-send-email
        name: Guard send_email
        description: Restrict sensitive outbound communication
        enabled: true
        severity: high
        action: block
        tools:
          - send_email
        conditions:
          - field: arguments.to
            operator: not_contains
            value: '@company.com'

  execute_shell_command (@veto/coding-agent)
  Rationale: Tool name matches coding-agent keywords (e.g. shell/exec/write_file).
  Snippet:
    rules:
      - id: guard-execute-shell-command
        name: Guard execute_shell_command
        description: Prevent destructive filesystem or shell operations
        enabled: true
        severity: critical
        action: block
        tools:
          - execute_shell_command
        conditions:
          - field: arguments.command
            operator: contains
            value: 'rm -rf'

JSON Output

veto scan --format json

Output:

{
  "timestamp": "2024-03-04T12:00:00.000Z",
  "projectDir": "/path/to/project",
  "policy": {
    "vetoDir": "/path/to/project/veto",
    "rulesDirectory": "/path/to/project/veto/rules",
    "recursiveRules": true,
    "rulesLoaded": 8,
    "globalRules": 2,
    "sourceFiles": [
      "veto/rules/defaults.yaml",
      "veto/rules/financial.yaml"
    ],
    "toolsReferenced": [
      "transfer_funds",
      "approve_invoice"
    ]
  },
  "manifest": {
    "packageJsonFound": true,
    "pyprojectFound": false,
    "jsDependencies": ["@langchain/core", "ai"],
    "pythonDependencies": [],
    "frameworks": ["langchain", "vercel-ai"]
  },
  "discoveredTools": [
    {
      "name": "transfer_funds",
      "parameters": ["amount", "recipient", "currency"],
      "locations": ["src/tools/financial.ts"],
      "sources": ["source-ts", "policy"],
      "covered": true,
      "coverageReason": "tool-rule",
      "matchedRuleIds": ["block-large-transfers"]
    },
    {
      "name": "send_email",
      "parameters": ["to", "subject", "body"],
      "locations": ["src/tools/email.ts"],
      "sources": ["source-ts"],
      "covered": false,
      "coverageReason": "none",
      "matchedRuleIds": []
    }
  ],
  "summary": {
    "total": 12,
    "covered": 8,
    "uncovered": 4,
    "coveragePercent": 66.67
  },
  "suggestions": []
}

CI/CD Gate

veto scan --fail-uncovered

Output (if uncovered tools exist):

Veto Scan Coverage Audit
========================

...

Coverage: 8/12 (66.7%)

Error: 4 uncovered tools found

(exits with code 1)

Scan Specific Directory

veto scan --directory ./packages/backend

How It Works

1. Tool Discovery

Scans source code for tool definitions: TypeScript:

// Detected patterns
tool({ name: "transfer_funds", ... })
const myTool = tool(...)
new DynamicTool({ name: "send_email", ... })
function_tool("execute_code", ...)

Python:

# Detected patterns
@tool
def transfer_funds(amount: int):
    pass

class TransferTool(BaseTool):
    name = "transfer_funds"

2. Policy Loading

Loads rules from veto/rules/:

rules:
  - id: block-large-transfers
    tools:
      - transfer_funds

3. Coverage Matching

Matches tools to rules:

✅ COVERED: Tool has specific rule or global rule applies
❌ UNCOVERED: Tool has no rules

4. Suggestions (Optional)

Generates policy suggestions based on tool name heuristics:

transfer, payment → @veto/financial
email, message → @veto/communication
navigate, click → @veto/browser-automation
query, database → @veto/data-access
shell, exec → @veto/coding-agent
deploy, release → @veto/deployment

Use Cases

Pre-Commit Hook

#!/bin/bash
# .git/hooks/pre-commit

veto scan --fail-uncovered --format text

if [ $? -ne 0 ]; then
  echo "Error: Uncovered tools found. Add policies before committing."
  exit 1
fi

CI Pipeline

# .github/workflows/veto.yml
name: Veto Policy Check

on: [push, pull_request]

jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-node@v3
        with:
          node-version: 20
      - run: npm install -g veto-cli
      - run: veto scan --fail-uncovered --format json > scan-report.json
      - uses: actions/upload-artifact@v3
        with:
          name: veto-scan-report
          path: scan-report.json

Weekly Coverage Report

#!/bin/bash
# cron: 0 9 * * 1 (Every Monday at 9am)

veto scan --suggest --format text > weekly-coverage-report.txt
cat weekly-coverage-report.txt | mail -s "Veto Coverage Report" [email protected]

Find Uncovered Tools

veto scan --format json | jq '.discoveredTools[] | select(.covered == false) | .name'

Output:

"send_email"
"execute_shell_command"
"browse_web"
"query_database"

Coverage Reasons

tool-rule

Tool has a specific rule targeting it:

rules:
  - id: block-transfers
    tools:
      - transfer_funds  # Specific match

global-rule

Tool is covered by a global rule (no tools: field):

rules:
  - id: log-all-tools
    action: log
    # No tools field = applies to all tools

none

Tool has no matching rules:

[UNCOVERED] send_email(to, subject, body)

Troubleshooting

No Tools Discovered

No tools discovered from policy files or source heuristics.

Solution:

Check that you’re using supported tool libraries (LangChain, Vercel AI, etc.)
Verify tool definitions follow standard patterns
Use --include-tests or --include-examples if tools are there

Low Coverage

Coverage: 2/20 (10.0%)

Solution:

# Generate suggestions
veto scan --suggest > suggestions.txt

# Generate policies for uncovered tools
veto policy generate --tool <name> --prompt "..."

# Or add global rule
echo 'rules:\n  - id: require-approval-all\n    action: require_approval' > veto/rules/global.yaml

False Positives

Scan detects non-tool functions: Solution:

Exclude directories: Don’t use --include-tests or --include-examples
Tools must follow standard naming patterns
Report false positives as bugs

Best Practices

1. Scan Regularly

# Add to package.json
{
  "scripts": {
    "veto:scan": "veto scan",
    "veto:check": "veto scan --fail-uncovered"
  }
}

# Run before commits
npm run veto:check

2. Use in CI

# Fail builds if coverage drops
veto scan --fail-uncovered

3. Track Coverage Over Time

# Save scan results
veto scan --format json > coverage-$(date +%Y%m%d).json

# Compare over time
jq '.summary.coveragePercent' coverage-*.json

4. Generate Suggestions

# Get policy suggestions
veto scan --suggest > suggestions.md

# Review and implement
vim suggestions.md

Next Steps

Generate Policies - Create rules for uncovered tools
Apply Policies - Deploy generated policies
Launch Studio - Interactive coverage view
Guard Check - Test individual tools

Getting Started

Commands

​Syntax

​Description

​Options

​Fail on Uncovered

​Suggest Policies

​Include Examples

​Include Tests

​Format

​Directory

​Examples

​Basic Scan

​Scan with Suggestions

​JSON Output

​CI/CD Gate

​Scan Specific Directory

​How It Works

​1. Tool Discovery

​2. Policy Loading

​3. Coverage Matching

​4. Suggestions (Optional)

​Use Cases

​Pre-Commit Hook

​CI Pipeline

​Weekly Coverage Report

​Find Uncovered Tools

​Coverage Reasons

​tool-rule

​global-rule

​none

​Troubleshooting

​No Tools Discovered

​Low Coverage

​False Positives

​Best Practices

​1. Scan Regularly

​2. Use in CI

​3. Track Coverage Over Time

​4. Generate Suggestions

​Next Steps

Build docs developers (and LLMs) love

Syntax

Description

Options

Fail on Uncovered

Suggest Policies

Include Examples

Include Tests

Format

Directory

Examples

Basic Scan

Scan with Suggestions

JSON Output

CI/CD Gate

Scan Specific Directory

How It Works

1. Tool Discovery

2. Policy Loading

3. Coverage Matching

4. Suggestions (Optional)

Use Cases

Pre-Commit Hook

CI Pipeline

Weekly Coverage Report

Find Uncovered Tools

Coverage Reasons

tool-rule

global-rule

none

Troubleshooting

No Tools Discovered

Low Coverage

False Positives

Best Practices

1. Scan Regularly

2. Use in CI

3. Track Coverage Over Time

4. Generate Suggestions

Next Steps