Skip to main content
The Web Browser tool gives agents the ability to visit websites, extract text content, and answer questions based on the information found on web pages.

​
Overview

The Web Browser tool combines web page fetching with text processing and question-answering capabilities. It uses embeddings to create a searchable index of the page content, allowing the agent to efficiently find relevant information to answer queries.
Unlike simple web scrapers, the Web Browser tool intelligently extracts and indexes content, making it ideal for agents that need to understand and reason about web page information.

​
How It Works

1

Agent Determines Need

The agent recognizes that it needs to visit a website to answer a user’s question.
2

Fetch Page Content

The tool fetches the HTML content from the specified URL.
3

Extract & Embed Text

Text is extracted from the page and converted into embeddings for semantic search.
4

Answer Question

The language model queries the embedded content to find relevant information and formulates an answer.

​
Configuration

​
model
BaseLanguageModel
required
The language model used to process the extracted content and answer questions.Compatible Models:
  • ChatOpenAI (GPT-3.5, GPT-4)
  • ChatAnthropic (Claude)
  • Any BaseChatModel or BaseLanguageModel
Use the same model as your agent for consistency, or a faster/cheaper model for efficiency.
​
embeddings
Embeddings
required
The embedding model used to create vector representations of the web page content.Compatible Embeddings:
  • OpenAI Embeddings
  • HuggingFace Embeddings
  • Cohere Embeddings
  • Any LangChain Embeddings implementation
The embeddings enable semantic search over the page content, allowing the tool to find relevant information even when exact keywords don’t match.

​
Usage

​
Adding to an Agent

1

Add Web Browser Tool

Drag the Web Browser tool from the Tools category onto your canvas.
2

Configure Language Model

Connect a language model (e.g., ChatOpenAI) to the model input.
3

Configure Embeddings

Connect an embeddings model (e.g., OpenAI Embeddings) to the embeddings input.
4

Connect to Agent

Connect the Web Browser tool to your agent’s tools input.
5

Test

Ask the agent questions that require visiting websites:
  • “What are the latest features on the Flowise homepage?”
  • “Visit example.com and tell me what they do”
  • “Browse to docs.example.com and find information about installation”

​
Example Queries

User: Visit flowise.ai and tell me what Flowise is used for.

Agent: Let me browse the Flowise website.
[Uses Web Browser: https://flowise.ai]

Response: Flowise is an open-source low-code tool for developers 
to build customized LLM orchestration flows and AI agents. It provides 
a drag-and-drop UI to visualize and build AI workflows...

​
Common Use Cases

Research Assistant

Browse websites to gather information and answer research questions

Competitor Analysis

Visit competitor websites to extract product information and features

Documentation Helper

Navigate documentation sites to find specific technical information

News Aggregation

Visit news sites to gather current information on topics

​
Capabilities and Limitations

  • Visit any publicly accessible URL
  • Extract text content from web pages
  • Answer questions based on page content
  • Handle multiple URLs in a conversation
  • Understand context and semantics through embeddings

​
Performance Considerations

Optimization Tips
  • Model Selection: Use a faster, cheaper model for the Web Browser tool if budget is a concern
  • Embedding Model: Choose efficient embeddings (e.g., OpenAI text-embedding-ada-002)
  • Caching: Results are not automatically cached; consider implementing caching for frequently accessed pages
  • Token Costs: Each page visit consumes tokens for both embedding and question-answering

​
Example Workflows

​
Research Agent with Web Browser

// Workflow Components:
// 1. ChatOpenAI (GPT-4) - Main agent model
// 2. ChatOpenAI (GPT-3.5-Turbo) - Web Browser model (cheaper)
// 3. OpenAI Embeddings - For content indexing
// 4. Web Browser Tool - Configured with models above
// 5. Tool Agent - With Web Browser in tools array
// 6. Buffer Memory - To maintain conversation context

// The agent can now:
// - Visit websites based on user requests
// - Answer questions about web page content
// - Compare information across multiple pages
// - Maintain context across multiple web visits

​
Multi-Tool Research Agent

// Combine Web Browser with other tools:
// - Calculator: For numerical analysis of found data
// - Retriever: For internal knowledge base
// - Web Browser: For external information

// Example conversation:
// User: "Find the current price of Bitcoin and calculate what 5 BTC would be worth"
// Agent: 
//   1. [Web Browser: Visit crypto price site]
//   2. [Calculator: price * 5]
//   3. Return combined result

​
Troubleshooting

Possible Causes:
  • Page uses JavaScript to load content
  • Page blocks automated access (bot detection)
  • URL is incorrect or inaccessible
Solutions:
  • For JS-heavy sites, use a Puppeteer or Playwright-based approach instead
  • Verify the URL is correct and publicly accessible
  • Check if the site has an API that provides the data
  • Test the URL in a regular browser first
Possible Causes:
  • Question doesn’t clearly indicate need for web browsing
  • Agent tries to answer from its training data
  • Tool not properly connected
Solutions:
  • Use explicit language: “Browse to…”, “Visit…”, “Check the website…”
  • Update system message to encourage web browsing for current information
  • Verify tool connections in the workflow
Possible Causes:
  • LLM generates answer beyond what’s on the page
  • Embeddings didn’t capture relevant information
  • Content on page is ambiguous
Solutions:
  • Use a more capable language model for the Web Browser
  • Improve system message to emphasize accuracy
  • Ask agent to cite specific parts of the page
  • Cross-reference with multiple sources
Possible Causes:
  • Large web pages take time to process
  • Embedding generation is slow
  • Multiple tool calls in sequence
Solutions:
  • Use faster embedding models
  • Implement caching for frequently accessed pages
  • Consider using a dedicated web scraper for specific sites

​
Best Practices

Recommended Practices
  1. Specific URLs: Encourage users to provide or agents to construct specific URLs rather than generic domains
  2. System Instructions: Guide the agent on when to use web browsing vs. its own knowledge
  3. Citation: Ask the agent to cite sources and URLs in its responses
  4. Verification: For critical information, cross-reference multiple sources
  5. Respect Limits: Honor robots.txt and implement rate limiting

​
Example System Message

You are a research assistant with access to web browsing capabilities.
When users ask about current events, specific websites, or information 
that may have changed recently, use the Web Browser tool to visit relevant 
websites and extract accurate information.

Always cite the URL you visited when providing information from websites.
If you cannot find information on a page, clearly state that rather than 
guessing or using your training data.

​
Security Considerations

Security Best Practices
  • URL Validation: Validate URLs before visiting to prevent SSRF attacks
  • Content Filtering: Be cautious with user-provided URLs
  • Rate Limiting: Implement rate limits to prevent abuse
  • Sensitive Data: Don’t visit URLs containing sensitive credentials
  • Cost Control: Monitor usage to prevent excessive API costs
For different web access needs, consider:

Cheerio Scraper

For document loading and bulk scraping (not real-time agent use)

Puppeteer Scraper

For JavaScript-heavy websites requiring browser execution

Web Search APIs

Google, Brave, or Serper for search-based information retrieval

API Tools

Direct API access for structured data from specific services

Tool Agent

Learn how to build agents that use multiple tools

Custom Tools

Create specialized web scraping tools

Search Tools

Explore web search alternatives

Agent Overview

Understand agent fundamentals

Build docs developers (and LLMs) love

Get started for freeTalk to us