Web Research

​

Overview

​

Tool

​

web_research

result = await web_research(
    topic="How does OAuth 2.0 authorization code flow work?",
    depth=3,
    max_sources=8
)

​

Research Process

​

1. Query Decomposition

• Original query
• “what is [topic]” variant

• All depth 1 queries
• Splits “and” and “vs” comparisons
• Adds “[topic] explained”

• All depth 2 queries
• “how does [topic] work”
• “why [topic]”
• “[topic] advantages disadvantages”

Depth 1:
- OAuth 2.0 vs JWT
- what is OAuth 2.0 vs JWT

Depth 2:
- OAuth 2.0
- JWT
- OAuth 2.0 vs JWT comparison
- OAuth 2.0 vs JWT explained

Depth 3:
- how does OAuth 2.0 vs JWT work
- why OAuth 2.0 vs JWT
- OAuth 2.0 vs JWT advantages disadvantages

​

2. Parallel Search

​

3. Source Ranking

• Frequency: Sources appearing in multiple search results score higher
• Domain diversity: Only one result per domain to ensure broad coverage
• Quality scoring: Primary sources (docs, official APIs) ranked above secondary sources

• PRIMARY (score 4-5): Official docs, GitHub, developer portals, .gov sites, Mozilla
• SECONDARY (score 2-3): Stack Overflow, Medium, Dev.to, Reddit
• UNVERIFIED (score 2): Other domains

​

4. Content Fetching

• HTML tag stripping (script, style, nav, footer)
• Content truncation at 30,000 characters per source
• Whitespace normalization
• Graceful handling of fetch failures

​

5. Cited Summary Generation

• Key findings from each source with quality labels
• Confidence assessment based on source count and quality
• Numbered citations for easy reference

​

Output Format

# Research: [Topic]

## Key Findings

**[1] OAuth 2.0 Specification (PRIMARY)**
OAuth 2.0 is an authorization framework that enables applications
to obtain limited access to user accounts on an HTTP service...

**[2] JWT Introduction (PRIMARY)**
JSON Web Tokens are an open, industry standard RFC 7519 method
for representing claims securely between two parties...

**[3] OAuth vs JWT Comparison (SECONDARY)**
OAuth 2.0 is an authorization protocol, while JWT is a token format.
They can be used together: OAuth 2.0 for authorization flow...

## Confidence: HIGH — Multiple authoritative sources agree

## Sources

[1] [PRIMARY] OAuth 2.0 Specification — https://oauth.net/2/
[2] [PRIMARY] JWT Introduction — https://jwt.io/introduction
[3] [SECONDARY] OAuth vs JWT on Stack Overflow — https://stackoverflow.com/...

​

Confidence Levels

​

HIGH

• 3 or more valid sources
• Average quality score greater than or equal to 3.5
• Multiple authoritative sources agree

​

MEDIUM

• 2 or more valid sources OR average quality greater than or equal to 3.0
• Limited sources or mixed authority

​

LOW

• Fewer than 2 sources OR low average quality
• Few sources or only unofficial references

​

Best Practices

1. Use depth 2 for most research — good balance of speed and thoroughness
2. Increase depth to 3 for complex topics — generates more comprehensive queries
3. Set max_sources to 8-10 for thorough research — ensures broad coverage
4. Check confidence level before acting on findings
5. Verify PRIMARY sources for critical decisions

​

Limitations

• No LLM summarization: Findings are raw excerpts, not synthesized summaries
• DuckDuckGo only: Does not use Brave Search API (may add in future)
• Text-only: Cannot extract content from PDFs, videos, or images
• Fetch failures: Some sources may fail to fetch due to rate limits or paywalls

​

Implementation

• Rule-based query decomposition (no LLM call)
• asyncio.gather() for parallel search and fetch operations
• Domain-based deduplication to ensure diversity
• Quality scoring with predefined patterns (docs., github.com, .gov, etc.)
• Frequency-based ranking for multi-query overlap