RAG System

What is RAG?

RAG (Retrieval-Augmented Generation) enhances AI responses by providing relevant domain knowledge as additional context. The Lead Intelligence Engine uses RAG to inject expertise about lead qualification, strategy frameworks, and service definitions into the evaluation process.

Think of RAG as giving the AI a “cheat sheet” - instead of relying only on its training data, it has access to your specific business knowledge.

How It Works

Knowledge Base Loading

On initialization, the RAG system loads all .md files from the knowledge/ directory into memory.

Query Processing

When evaluating a business, the extracted website content is used as a search query.

Relevance Scoring

Each knowledge document is scored based on keyword overlap with the query.

Context Injection

The top 3 most relevant documents are prepended to the AI prompt as “Advisory Context”.

Knowledge Base Structure

The knowledge/ directory contains domain expertise:

knowledge/
├── Lead_q_criteria.md        # Lead qualification rules
└── strategy_efficiency.md    # Conversation angle framework

Lead Qualification Criteria

File: Lead_q_criteria.md Contents:

Geography: Target markets (Yangon, Mandalay, Bangkok)
Industry Types: Medical, Education, F&B, Construction, etc.
Business Size: Team size, revenue proxies, growth stage
Digital Maturity Framework: 5-layer evaluation (Visibility, Website, Measurement, Operational, Asset Ownership)
Industry Exclusions: Don’t sell marketing to marketing agencies, don’t sell dev to dev shops

When Retrieved: Business content mentions specific industries, locations, or maturity signals.

Strategy Framework

File: strategy_efficiency.md Contents:

Conversation Angle Prediction: 6 strategic entry points
1. Revenue Visibility Gap
2. Conversion Infrastructure Weakness
3. Operational Bottleneck
4. Platform Dependency Risk
5. Underutilized Attention
6. Competitive Positioning Gap
Trigger Conditions: What patterns indicate each angle
Framing Language: Analytical, neutral, structured tone

When Retrieved: Business shows specific gaps (no analytics, Facebook-only, manual booking, etc.)

Strategy Framework

Full documentation of the 6 conversation angles

Retrieval Algorithm

The RAG system uses keyword-based retrieval with scoring:

rag.py (lines 21-69)

def retrieve(self, query_text, limit=3):
    """Simple keyword-based retrieval with basic ranking."""
    scored_docs = []
    query_text = query_text.lower()
    query_words = set(query_text.split())

    for doc in self.corpus:
        doc_content = doc["content"].lower()
        doc_words = set(doc_content.split())
        
        # Intersection score
        intersection = query_words.intersection(doc_words)
        score = len(intersection)
        
        # Boost for long words (> 4 chars)
        for word in query_words:
            if len(word) > 4 and word in doc_content:
                score += 0.5

        if score > 0:
            scored_docs.append((score, doc))

    # Sort by score descending
    scored_docs.sort(key=lambda x: x[0], reverse=True)
    
    # Return top 3 unique documents
    return [doc["content"] for score, doc in scored_docs[:limit]]

Scoring Mechanics

Base Score: Word Intersection

Counts how many words appear in both the query and the document.Example:

Query: “medical clinic bangkok”
Document contains: “medical”, “bangkok”, “clinic”
Score: 3

Boost: Long Words

Adds +0.5 for each word longer than 4 characters that appears in the document.Reasoning: Long words are more specific and meaningful (e.g., “marketing” vs “to”).Example:

Query: “restaurant automation system”
“automation” (10 chars) found → +0.5
“restaurant” (10 chars) found → +0.5
“system” (6 chars) found → +0.5
Total boost: +1.5

Fallback: High-Value Terms

If no matches found, searches for these domain-specific keywords:

Geographic: yangon, mandalay, bangkok
Industry: medical, education, marketing, digital

Assigns score of 1 if any found.

Why This Works

This simple algorithm is effective because:

Business websites repeat key terms (“dental”, “clinic”, “Bangkok”)
Knowledge base is small (2 files), so computation is fast (<0.1s)
No training or embedding models needed - zero setup

Context Injection

Retrieved documents are injected into the AI prompt:

evaluator.py (lines 69-73)

user_content = f"Analyze this website content and provide the evaluation in JSON:\n\n{content}"
if rag_context:
    rag_text = "\n\n".join(rag_context)
    user_content = f"Additional Advisory Context (RAG):\n{rag_text}\n\n---\n\n{user_content}"

Prompt Structure

Without RAG:

Analyze this website content and provide the evaluation in JSON:

[Website text...]

With RAG:

Additional Advisory Context (RAG):
[Lead qualification criteria]

[Strategy framework]

---

Analyze this website content and provide the evaluation in JSON:

[Website text...]

The --- separator helps the AI distinguish between reference material and the actual content to analyze.

Impact on Evaluation

RAG improves AI evaluation accuracy by:

1. Industry Exclusion Enforcement

Without RAG, the AI might suggest marketing services to a marketing agency. With RAG, the qualification criteria explicitly states:

“Digital Marketing Agencies: Do NOT sell ‘Marketing Services’ or ‘Marketing Packages’. Focus on ‘Technology Services’.“

2. Geography-Aware Scoring

RAG provides context about target markets:

“Geography: Yangon, Mandalay, Bangkok”

Businesses in these locations get higher fit scores.

3. Strategic Outreach Angles

The strategy framework guides the AI to generate specific outreach angles:

“Platform Dependency Risk - Current growth relies on rented Facebook platform.”

Instead of generic:

“They need a website.”

4. Digital Maturity Assessment

The 5-layer framework helps the AI identify structural gaps:

Visibility Layer → “Low posting frequency”
Website Evaluation → “No lead capture form”
Measurement Signals → “No tracking scripts detected”

Performance Characteristics

Latency

Loading corpus: ~10ms (on initialization)
Retrieval per query: ~50ms (keyword matching)
Total overhead: <0.1s per URL

RAG adds negligible latency because it uses in-memory search with simple string matching.

Token Consumption

RAG adds ~500-1,000 tokens to each prompt:

Lead criteria: ~300 tokens
Strategy framework: ~400 tokens
Separator and formatting: ~50 tokens

Cost: ~$0.0001 per analysis (at Groq pricing)

Customizing the Knowledge Base

You can add your own knowledge files:

Create Markdown File

touch knowledge/my_custom_criteria.md

Add Content

Write your domain expertise in plain markdown:

# Custom Industry Rules

## E-commerce Signals
- Product catalogs
- Shopping cart functionality
- Payment gateway integration

## Recommended Services
- Foundation Package if no cart
- Custom Digital Solutions if scaling

Restart Engine

RAG loads files on initialization:

python main.py https://example.com

Keep knowledge files focused and concise. Large files dilute relevance scores and increase token costs.

Limitations

1. No Semantic Understanding

Keyword matching doesn’t understand meaning:

“bank” (financial) vs “river bank” → same score
“clinic” and “healthcare” → no match (despite similarity)

Mitigation: Use synonyms and variations in knowledge files.

2. Fixed Limit (Top 3)

Only 3 documents are retrieved, regardless of relevance. Mitigation: Keep knowledge base small and well-organized.

3. No Re-Ranking

Once scored, results are not re-evaluated based on AI feedback. Future: Could implement feedback loop where AI rates context usefulness.

Comparison: RAG vs No RAG

With RAG
Without RAG

Analysis of a Bangkok dental clinic:

{
  "business_name": "Smile Bright Dental",
  "business_type": "Dental Healthcare Provider",
  "primary_service": "Foundation Package",
  "fit_score": 88,
  "reasoning": "Bangkok-based clinic (target geography) with Facebook-only presence. High fit for Foundation Package per qualification criteria.",
  "outreach_angle": "Platform dependency risk - current growth relies on rented Facebook platform. Consider owned digital asset for long-term stability."
}

✅ Recognizes target geography
✅ Applies strategic framework
✅ Uses domain-specific language

Same clinic, no RAG context:

{
  "business_name": "Smile Bright Dental",
  "business_type": "Dental Clinic",
  "primary_service": "Foundation Package",
  "fit_score": 72,
  "reasoning": "Small dental clinic needs a basic website for online presence.",
  "outreach_angle": "Having a website would help attract more patients."
}

❌ Generic reasoning
❌ Lower confidence score
❌ Weak outreach angle

Next Steps

Evaluation Pipeline

Learn how RAG context is used in AI evaluation

Knowledge Base

Explore the lead qualification criteria

RAG API Reference

Programmatic usage of the RAG system

Configuration

Customize your knowledge base

Get Started

Core Concepts

Usage Guides

Knowledge Base

What is RAG?

How It Works

Knowledge Base Structure

Lead Qualification Criteria

Strategy Framework

Strategy Framework

Retrieval Algorithm

Scoring Mechanics

Why This Works

Context Injection

Prompt Structure

Impact on Evaluation

1. Industry Exclusion Enforcement

2. Geography-Aware Scoring

3. Strategic Outreach Angles

4. Digital Maturity Assessment

Performance Characteristics

Latency

Token Consumption

Customizing the Knowledge Base

Limitations

1. No Semantic Understanding

2. Fixed Limit (Top 3)

3. No Re-Ranking

Comparison: RAG vs No RAG

Next Steps

Evaluation Pipeline

Knowledge Base

RAG API Reference

Configuration

Build docs developers (and LLMs) love

Get Started

Core Concepts

Usage Guides

Knowledge Base

​What is RAG?

​How It Works

​Knowledge Base Structure

​Lead Qualification Criteria

​Strategy Framework

Strategy Framework

​Retrieval Algorithm

​Scoring Mechanics

​Why This Works

​Context Injection

​Prompt Structure

​Impact on Evaluation

​1. Industry Exclusion Enforcement

​2. Geography-Aware Scoring

​3. Strategic Outreach Angles

​4. Digital Maturity Assessment

​Performance Characteristics

​Latency

​Token Consumption

​Customizing the Knowledge Base

​Limitations

​1. No Semantic Understanding

​2. Fixed Limit (Top 3)

​3. No Re-Ranking

​Comparison: RAG vs No RAG

​Next Steps

Evaluation Pipeline

Knowledge Base

RAG API Reference

Configuration

Build docs developers (and LLMs) love

What is RAG?

How It Works

Knowledge Base Structure

Lead Qualification Criteria

Strategy Framework

Retrieval Algorithm

Scoring Mechanics

Why This Works

Context Injection

Prompt Structure

Impact on Evaluation

1. Industry Exclusion Enforcement

2. Geography-Aware Scoring

3. Strategic Outreach Angles

4. Digital Maturity Assessment

Performance Characteristics

Latency

Token Consumption

Customizing the Knowledge Base

Limitations

1. No Semantic Understanding

2. Fixed Limit (Top 3)

3. No Re-Ranking

Comparison: RAG vs No RAG

Next Steps