Skip to main content
The Web Search tool enables agents to search the internet for current information, news, and general knowledge using the Serper API (Google search).

Overview

Web Search provides:
  • Google Search Integration: Powered by Serper.dev API
  • Organic Results: Returns relevant search results with snippets
  • Date Information: Includes publication dates when available
  • Formatted Output: Clean markdown format for easy parsing
  • Configurable Endpoint: Support for custom Serper URLs

Registration

@register_tool('web_search', allow_overwrite=True)
class WebSearch(BaseTool):
    ...
Tool Name: web_search

Parameters

query
string
required
The search query to submit to the search engine. Use natural language or keywords.

Parameter Schema

{
  "type": "object",
  "properties": {
    "query": {
      "type": "string"
    }
  },
  "required": ["query"]
}

Configuration

Environment Variables

SERPER_API_KEY
string
required
Your Serper API key. Get one from serper.dev
export SERPER_API_KEY="your-api-key-here"
SERPER_URL
string
default:"https://google.serper.dev/search"
Custom Serper API endpoint (optional).
export SERPER_URL="https://google.serper.dev/search"

Setup

1. Get API Key

  1. Visit serper.dev
  2. Sign up for an account
  3. Get your API key from the dashboard
  4. Free tier includes 2,500 searches

2. Set Environment Variable

# Linux/Mac
export SERPER_API_KEY="your-api-key-here"

# Windows (Command Prompt)
set SERPER_API_KEY=your-api-key-here

# Windows (PowerShell)
$env:SERPER_API_KEY="your-api-key-here"
Or in Python:
import os
os.environ['SERPER_API_KEY'] = 'your-api-key-here'

Usage

from qwen_agent.tools import WebSearch
import json

# Initialize the tool
web_search = WebSearch()

# Perform a search
result = web_search.call(params=json.dumps({'query': 'latest AI news'}))
print(result)

Output Format

[1]“Latest breakthrough in AI research Researchers announce new language model…“2024-03-15 [2]“AI companies announce partnerships Major tech firms collaborate on AI safety…“2024-03-14 [3]“New AI regulations proposed Government proposes framework for AI governance…”

Using with Agents

from qwen_agent.agents import Assistant

bot = Assistant(
    llm={'model': 'qwen-max'},
    function_list=['web_search']
)

messages = [
    {
        'role': 'user',
        'content': 'What are the latest developments in quantum computing?'
    }
]

for response in bot.run(messages=messages):
    print(response)

Multiple Searches

web_search = WebSearch()

queries = [
    'Python best practices 2024',
    'machine learning tutorials',
    'latest tech trends'
]

for query in queries:
    print(f"\n=== Searching: {query} ===")
    result = web_search.call(params=json.dumps({'query': query}))
    print(result)

Return Format

The tool returns search results in a structured markdown format:
[1]"Title of first result
Snippet text from the page..."Date

[2]"Title of second result
Snippet text from the page..."Date

[3]"Title of third result
Snippet text from the page..."
Each result includes:
  • Index: Sequential number [1], [2], etc.
  • Title: Page title in quotes
  • Snippet: Brief excerpt from the page
  • Date: Publication date (if available)

Example: Research Assistant

from qwen_agent.agents import Assistant

def create_research_assistant():
    """Create an agent that can search the web for information."""
    bot = Assistant(
        llm={'model': 'qwen-max'},
        name='Research Assistant',
        description='I can search the web to find current information and help answer your questions.',
        system_message=(
            'You are a helpful research assistant. When users ask questions, '
            'use web search to find current information. Always cite your sources '
            'and provide the most recent and relevant information.'
        ),
        function_list=['web_search']
    )
    return bot

# Create and use the assistant
bot = create_research_assistant()

messages = []
while True:
    query = input('\nAsk a question (or "quit"): ')
    if query.lower() in ['quit', 'exit', 'q']:
        break
    
    messages.append({'role': 'user', 'content': query})
    
    response = []
    for response in bot.run(messages=messages):
        pass  # Wait for complete response
    
    # Print assistant's response
    print(f"\nAssistant: {response[-1]['content']}")
    messages.extend(response)

API Response Structure

The Serper API returns JSON with this structure:
{
    'organic': [
        {
            'title': 'Page title',
            'snippet': 'Page description or excerpt',
            'link': 'https://example.com/page',
            'date': '2024-03-15',  # Optional
            'position': 1
        },
        # ... more results
    ],
    'searchParameters': {
        'q': 'search query',
        'type': 'search'
    }
}
The tool extracts and formats the organic results.

Advanced Usage

Direct API Access

from qwen_agent.tools import WebSearch

# Use the static search method directly
raw_results = WebSearch.search('AI research papers')

# raw_results is a list of dicts
for result in raw_results:
    print(f"Title: {result['title']}")
    print(f"Link: {result['link']}")
    print(f"Snippet: {result['snippet']}")
    if 'date' in result:
        print(f"Date: {result['date']}")
    print()

Custom Formatting

from qwen_agent.tools import WebSearch

class CustomWebSearch(WebSearch):
    @staticmethod
    def _format_results(search_results):
        """Custom result formatting."""
        formatted = []
        for i, doc in enumerate(search_results, 1):
            formatted.append(
                f"{i}. {doc['title']}\n"
                f"   {doc.get('snippet', 'No description')}\n"
                f"   {doc.get('link', '')}\n"
            )
        return '\n'.join(formatted)

# Use custom formatter
custom_search = CustomWebSearch()
result = custom_search.call(params='{"query": "Python tips"}')
print(result)

Error Handling

from qwen_agent.tools import WebSearch
import json

web_search = WebSearch()

try:
    result = web_search.call(
        params=json.dumps({'query': 'test search'})
    )
    print(result)
except ValueError as e:
    if 'SERPER_API_KEY' in str(e):
        print("Error: API key not set. Please set SERPER_API_KEY environment variable.")
    else:
        print(f"Error: {e}")
except Exception as e:
    print(f"Search failed: {e}")

Best Practices

Good queries:
  • “latest developments in renewable energy 2024”
  • “Python async programming best practices”
  • “COVID-19 vaccine statistics”
Avoid:
  • Single words (too broad)
  • Very long queries (over 200 chars)
  • Special characters that might break JSON
  • Never hardcode API keys in source code
  • Use environment variables
  • Keep keys in .env files (add to .gitignore)
  • Rotate keys periodically
  • Monitor usage in Serper dashboard
  • Free tier: 2,500 searches/month
  • Add delays between searches if making many requests
  • Consider caching results for common queries
  • Monitor your usage at serper.dev/dashboard
  • Check for empty results before processing
  • Handle missing dates gracefully
  • Validate URLs before using
  • Consider result position for relevance

Example: News Monitor

from qwen_agent.tools import WebSearch
import json
import time
from datetime import datetime

def monitor_news_topic(topic, interval_minutes=60):
    """Monitor a topic and report new results periodically."""
    web_search = WebSearch()
    seen_titles = set()
    
    print(f"Monitoring: {topic}")
    print(f"Checking every {interval_minutes} minutes...\n")
    
    while True:
        try:
            result = web_search.call(
                params=json.dumps({'query': f'{topic} latest news'})
            )
            
            # Parse results (simplified)
            lines = result.split('\n')
            current_titles = [line for line in lines if line.startswith('[')]
            
            # Check for new articles
            new_articles = [t for t in current_titles if t not in seen_titles]
            
            if new_articles:
                print(f"\n[{datetime.now().strftime('%Y-%m-%d %H:%M')}] New articles:")
                for article in new_articles:
                    print(article)
                seen_titles.update(new_articles)
            
            time.sleep(interval_minutes * 60)
            
        except KeyboardInterrupt:
            print("\nStopping monitor...")
            break
        except Exception as e:
            print(f"Error: {e}")
            time.sleep(60)

# Monitor AI news
if __name__ == '__main__':
    monitor_news_topic('artificial intelligence', interval_minutes=30)

Troubleshooting

ValueError: SERPER_API_KEY is None!
Solution: Set the environment variable:
export SERPER_API_KEY="your-key"
Causes:
  • Invalid API key
  • Expired API key
  • Key not activated
Solution: Check your key at serper.dev/dashboard
Cause: Rate limit exceededSolution:
  • Wait before retrying
  • Upgrade your Serper plan
  • Implement request throttling
Causes:
  • Very specific query with no matches
  • Typos in query
  • Network issues
Solution:
  • Broaden the search query
  • Check spelling
  • Verify internet connection

Limitations

  • Requires internet connection
  • API key needed (free tier available)
  • Rate limits apply based on plan
  • Search results depend on Google’s index
  • No control over result ranking

Alternatives

If Web Search doesn’t meet your needs:
  • Web Extractor: Extract content from specific URLs
  • Custom Search: Implement your own search using other APIs
  • RAG with Retrieval: Search pre-indexed documents

Web Extractor

Extract content from web pages

Image Search

Search for images on the web

Assistant Agent

Agent that can use web search

Build docs developers (and LLMs) love