Overview
TheRAGAgent class orchestrates the complete RAG (Retrieval-Augmented Generation) flow including question analysis, multi-source search, response generation, and iterative improvement.
Class Definition
RAGAgent
Constructor
__init__(openai_api_key=None)
Initialize the RAG Agent with all necessary components.
OpenAI API key for LLM operations. If None, will attempt to load from environment variables.
ValueError- If OpenAI API key is not provided and not found in environment
Core Methods
set_database_connection(db_connection)
Set the database connection for retrieving conversation history.Database connection object for conversation memory
RAG Flow Pipeline
The RAG agent executes a multi-stage pipeline:Stage 1: Setup
- Initialize context and logging
- Set maximum iterations
Stage 2: Reasoning
- Analyze question with LLM
- Extract metadata (tickers, quarters)
- Explain research approach
Stage 3: Search Planning
- Resolve temporal references to specific quarters
- Generate queries for each data source
Stage 4: Parallel Multi-Source Search
Runs searches in parallel for maximum performance:- News Search
- 10-K Search
- Transcript Search
Optional Tavily news search if included in search plan
Stage 5: Context Preparation
- Build news/10-K context strings
- Combine citations from all sources
Stage 6: Improvement Loop
- Generate initial answer
- Evaluate quality and confidence
- Perform follow-up searches if needed
- Re-generate answer with additional context
Stage 7: Finalization
- Deduplicate citations
- Build final result structure
- Update conversation memory
Internal Methods
Question Analysis
The user’s question to analyze
Unique conversation identifier
tuple of (question_analysis, target_quarters, error_dict)
- question_analysis (Dict): Analysis results including status, tickers, question type, processed question, and quarter context
- target_quarters (List[str]): Quarters to search (e.g.,
['2025_q1', '2025_q2']) - error_dict (Dict): Error information if validation fails
Search Execution
- question
- question_analysis
- target_quarters
- individual_results
- all_chunks
- all_citations
- search_time
- is_general_question
- is_multi_ticker
- tickers_to_process
- target_quarter
Ticker Processing
Ticker symbol to process
Original user question
Cleaned/normalized question
Whether this is part of multi-ticker query
Quarters to search
Parallel Processing
List of ticker symbols to process
Configuration
The agent uses aConfig object with these key settings:
Enable/disable hybrid search (vector + keyword)
Weight for vector search results in hybrid search
Weight for keyword search results in hybrid search
Maximum number of quarters to search (3 years)
Maximum number of tickers to process in parallel
Number of chunks to retrieve per quarter
Usage Examples
- Basic Usage
- With Database
- Custom Config
Components
The RAG Agent initializes and orchestrates several components:DatabaseManager
Manages database connections and queries
QuestionAnalyzer
Analyzes questions to extract intent and entities
ReasoningPlanner
Plans research approach and strategy
SearchPlanner
Plans which data sources to query
SearchEngine
Executes hybrid search operations
ResponseGenerator
Generates natural language responses
TavilyService
Fetches recent news articles
SECFilingsService
Retrieves and processes 10-K filings
Logging
The RAG Agent provides detailed logging at multiple levels:Error Handling
The agent handles various error scenarios:- Missing API Key: Raises
ValueErroron initialization - Database Errors: Logs warning and continues with default config
- Quarter Unavailable: Returns user-friendly error with available quarters
- No Results: Returns empty context response when no data is found
- Search Failures: Falls back gracefully and logs errors
Performance Optimization
The RAG Agent includes several performance optimizations:- Parallel Search: Multiple data sources searched concurrently
- Thread Pool: CPU-bound operations executed in thread pool
- Multi-Ticker Parallel: Tickers processed simultaneously
- Multi-Quarter Parallel: Quarters searched in parallel
- Async Operations: Non-blocking I/O operations
Related
SearchEngine
Search operations and hybrid search implementation
Agent
Main agent interface and aliases