Skip to main content

​
Overview

PentAGI integrates with multiple search engines to gather intelligence during penetration testing. While optional, enabling search engines significantly enhances PentAGI’s ability to research vulnerabilities, exploit techniques, and security tools.
Search engine configuration is optional but recommended for better results. You can enable one or multiple search providers.

​
DuckDuckGo

DuckDuckGo provides privacy-focused web search without requiring an API key.
​
DUCKDUCKGO_ENABLED
boolean
default:"false"
Enable DuckDuckGo search engine integration
DUCKDUCKGO_ENABLED=true

​
Features

  • No API key required
  • Privacy-focused search
  • No rate limits
  • Free to use

​
Example Configuration

.env
# DuckDuckGo Configuration
DUCKDUCKGO_ENABLED=true
Google Custom Search provides comprehensive web search results with programmable access.
​
GOOGLE_API_KEY
string
Your Google API key from Google Cloud Console
GOOGLE_API_KEY=AIzaSyD...
​
GOOGLE_CX_KEY
string
Custom Search Engine ID from Programmable Search Engine
GOOGLE_CX_KEY=0123456789abcdef0:abcdefg123
​
GOOGLE_LR_KEY
string
Language restriction code (optional)
GOOGLE_LR_KEY=lang_en

​
Setup Instructions

  1. Create a Google Cloud Project at console.cloud.google.com
  2. Enable Custom Search API in your project
  3. Create API credentials (API Key)
  4. Create a Custom Search Engine at programmablesearchengine.google.com
  5. Get your Search Engine ID (CX key)

​
Example Configuration

.env
# Google Custom Search Configuration
GOOGLE_API_KEY=AIzaSyDExampleKey123456789
GOOGLE_CX_KEY=0123456789abcdef0:examplecxkey
GOOGLE_LR_KEY=lang_en

​
Tavily

Tavily provides AI-optimized search specifically designed for LLM applications.
​
TAVILY_API_KEY
string
Your Tavily API key from tavily.com
TAVILY_API_KEY=tvly-...

​
Features

  • Optimized for AI/LLM use cases
  • Clean, structured results
  • Fast response times
  • Focused on developer documentation and technical content

​
Setup Instructions

  1. Visit tavily.com
  2. Sign up for an account
  3. Generate an API key from your dashboard
  4. Add the key to your .env file

​
Example Configuration

.env
# Tavily Configuration
TAVILY_API_KEY=tvly-ExampleKey123456789

​
Traversaal

Traversaal provides advanced web search with AI-powered result ranking.
​
TRAVERSAAL_API_KEY
string
Your Traversaal API key from traversaal.ai
TRAVERSAAL_API_KEY=tr_...

​
Features

  • AI-powered result ranking
  • Developer-focused search
  • Technical documentation prioritization
  • Clean JSON responses

​
Setup Instructions

  1. Visit traversaal.ai
  2. Create an account
  3. Generate an API key
  4. Add the key to your .env file

​
Example Configuration

.env
# Traversaal Configuration
TRAVERSAAL_API_KEY=tr_ExampleKey123456789

​
Perplexity

Perplexity combines search with AI-powered answer generation for comprehensive results.
​
PERPLEXITY_API_KEY
string
Your Perplexity API key from perplexity.ai
PERPLEXITY_API_KEY=pplx-...
​
PERPLEXITY_MODEL
string
default:"sonar"
Perplexity model to use for search
PERPLEXITY_MODEL=sonar-pro
Available models:
  • sonar - Standard model
  • sonar-pro - Enhanced model with better accuracy
​
PERPLEXITY_CONTEXT_SIZE
string
default:"low"
Context size for search results
PERPLEXITY_CONTEXT_SIZE=medium
Available sizes:
  • low - Minimal context, faster responses
  • medium - Balanced context and speed
  • high - Maximum context, slower responses

​
Features

  • AI-powered answer synthesis
  • Multiple model options
  • Configurable context size
  • Real-time web search combined with AI analysis

​
Setup Instructions

  1. Visit perplexity.ai
  2. Sign up for API access
  3. Generate an API key from your account settings
  4. Add the key to your .env file

​
Example Configuration

.env
# Perplexity Configuration
PERPLEXITY_API_KEY=pplx-ExampleKey123456789
PERPLEXITY_MODEL=sonar-pro
PERPLEXITY_CONTEXT_SIZE=medium

​
Searxng

Searxng is a privacy-respecting metasearch engine that aggregates results from multiple sources.
​
SEARXNG_URL
string
URL of your Searxng instance
SEARXNG_URL=http://searxng.example.com:8080
​
SEARXNG_CATEGORIES
string
default:"general"
Search categories to query (comma-separated)
SEARXNG_CATEGORIES=general,it
Available categories:
  • general - General web search
  • it - IT and technology
  • science - Scientific content
  • news - News articles
  • social media - Social media content
​
SEARXNG_LANGUAGE
string
Language code for search results
SEARXNG_LANGUAGE=en-US
​
SEARXNG_SAFESEARCH
number
default:"0"
Safe search level
SEARXNG_SAFESEARCH=0
Values:
  • 0 - Disabled
  • 1 - Moderate
  • 2 - Strict
​
SEARXNG_TIME_RANGE
string
Time range for search results
SEARXNG_TIME_RANGE=year
Available ranges:
  • day - Last 24 hours
  • week - Last week
  • month - Last month
  • year - Last year

​
Features

  • Privacy-respecting metasearch
  • Aggregates results from 70+ search engines
  • Self-hosted option for complete control
  • No tracking or profiling
  • Highly configurable

​
Setup Instructions

Option 1: Use Public Instance
  1. Find a public Searxng instance at searx.space
  2. Add the instance URL to your .env file
Option 2: Self-Host Searxng
  1. Follow the Searxng installation guide
  2. Deploy using Docker: 
    docker run -d -p 8080:8080 searxng/searxng
  3. Add your instance URL to .env file

​
Example Configuration

.env
# Searxng Configuration
SEARXNG_URL=http://searxng.example.com:8080
SEARXNG_CATEGORIES=general,it
SEARXNG_LANGUAGE=en-US
SEARXNG_SAFESEARCH=0
SEARXNG_TIME_RANGE=year

​
Complete Example Configuration

Here’s a complete example with all search engines configured:
.env
## Search Engine Configuration

# DuckDuckGo (no API key required)
DUCKDUCKGO_ENABLED=true

# Google Custom Search
GOOGLE_API_KEY=AIzaSyDExampleKey123456789
GOOGLE_CX_KEY=0123456789abcdef0:examplecxkey
GOOGLE_LR_KEY=lang_en

# Tavily
TAVILY_API_KEY=tvly-ExampleKey123456789

# Traversaal
TRAVERSAAL_API_KEY=tr_ExampleKey123456789

# Perplexity
PERPLEXITY_API_KEY=pplx-ExampleKey123456789
PERPLEXITY_MODEL=sonar-pro
PERPLEXITY_CONTEXT_SIZE=medium

# Searxng
SEARXNG_URL=http://searxng.example.com:8080
SEARXNG_CATEGORIES=general,it
SEARXNG_LANGUAGE=en-US
SEARXNG_SAFESEARCH=0
SEARXNG_TIME_RANGE=year

# Optional: Global proxy for all search engines
PROXY_URL=http://proxy.example.com:8080

​
Search Engine Selection Strategy

PentAGI automatically selects the most appropriate search engine based on:
  • Availability: Only configured search engines are used
  • Query Type: Different engines excel at different query types
  • Rate Limits: Automatically rotates between engines if rate limited
  • Response Quality: Prioritizes engines with better historical results
Minimal Setup: 
DUCKDUCKGO_ENABLED=true
Recommended Setup: 
DUCKDUCKGO_ENABLED=true
TAVILY_API_KEY=your_key
Advanced Setup: 
DUCKDUCKGO_ENABLED=true
TAVILY_API_KEY=your_key
PERPLEXITY_API_KEY=your_key
PERPLEXITY_MODEL=sonar-pro
SEARXNG_URL=http://your-searxng:8080

​
Proxy Configuration

All search engines support routing through a proxy:
​
PROXY_URL
string
Global HTTP proxy URL for all search engines and external systems
PROXY_URL=http://proxy.example.com:8080
The proxy is particularly useful for:
  • Network isolation in secure environments
  • Routing through VPNs
  • Corporate proxy requirements
  • Testing from specific geographic locations

​
Next Steps

LLM Providers

Configure OpenAI, Anthropic, Gemini, and more

Security Settings

Configure SSL, authentication, and secrets

Observability

Set up monitoring and analytics

Build docs developers (and LLMs) love

Get started for freeTalk to us