Structured RAG

TypeAgent implements a structured RAG (Retrieval-Augmented Generation) system that combines semantic similarity search with structured knowledge queries. This hybrid approach provides both the precision of structured queries and the recall of semantic search.

Traditional RAG vs Structured RAG

Traditional RAG
TypeAgent's Structured RAG

Traditional RAG systems rely primarily on embedding similarity:

User Query → Embedding → Vector Search → Top-K Chunks → LLM → Answer

Limitations:

No understanding of structure (entities, relationships, actions)
Cannot filter by properties or attributes
Difficult to scope searches to specific subjects or time ranges
Limited ability to combine multiple search criteria
Relies entirely on semantic similarity scores

TypeAgent’s structured RAG uses multiple query strategies:

User Query → Query Translation → Multi-Index Search → Result Fusion → Answer
                    ↓                    ↓
              SearchQuery          [SemanticRefIndex,
              (entities,            PropertyIndex,
               actions,             MessageTextIndex,
               filters)             TimestampIndex]

Advantages:

Understands entities, actions, topics, and relationships
Supports property-based filtering (entity types, facets)
Enables temporal scoping (time ranges)
Combines multiple search criteria with boolean logic
Falls back to semantic search when needed

Query Translation Pipeline

TypeAgent translates natural language queries into structured search expressions using LLMs:

Stage 1: Language to Structure

from typeagent.knowpro import searchlang
import typechat

# Create query translator
model = create_chat_model()
query_translator = utils.create_translator(
    model, 
    search_query_schema.SearchQuery
)

# Translate natural language
query_text = "What did John discuss about the project last week?"
result = await searchlang.search_query_from_language(
    conversation,
    query_translator,
    query_text
)

if isinstance(result, typechat.Success):
    search_query = result.value
    # search_query.search_expressions - list of SearchExpr
    # Each SearchExpr contains filters for entities, actions, time ranges

Structured Search Query Schema

The LLM translates queries into this structure:

type SearchQuery = {
  search_expressions: SearchExpr[];
};

type SearchExpr = {
  filters: SearchFilter[];
  rewritten_query?: string;
};

type SearchFilter = {
  entity_search_terms?: EntityTerm[];
  action_search_term?: ActionTerm;
  search_terms?: string[];
  time_range?: DateTimeRange;
};

type EntityTerm = {
  name: string;
  type?: string[];
  facets?: Facet[];
  is_name_pronoun?: boolean;
};

type ActionTerm = {
  action_verbs?: VerbsTerm;
  actor_entities?: EntityTerm[] | "*";
  target_entities?: EntityTerm[] | "*";
  additional_entities?: EntityTerm[] | "*";
  is_informational?: boolean;
};

The LLM uses TypeChat to ensure the output matches this schema exactly, providing type-safe structured queries.

Multi-Index Search Strategy

Once the query is structured, TypeAgent executes parallel searches across multiple indexes:

Index Selection

from typeagent.knowpro.searchlang import SearchQueryCompiler

compiler = SearchQueryCompiler(
    conversation,
    options=LanguageQueryCompileOptions(
        exact_scope=False,  # Enable fuzzy matching
        verb_scope=True,    # Match action verbs
        apply_scope=True    # Apply scoping constraints
    )
)

# Compile to low-level search expressions
query_expressions = compiler.compile_query(search_query)

# Each query_expression targets specific indexes:
# - SemanticRefIndex for term lookups
# - PropertyIndex for entity.name, entity.type, facets
# - TimestampIndex for time_range filters
# - MessageTextIndex for semantic similarity

Property-Based Queries

The PropertyIndex enables precise structured queries:

from typeagent.storage.memory.propindex import PropertyNames

# Query by entity name
scored_refs = await property_index.lookup_property(
    PropertyNames.EntityName.value,
    "John"
)

# Query by entity type
scored_refs = await property_index.lookup_property(
    PropertyNames.EntityType.value,
    "person"
)

# Query by action verb
scored_refs = await property_index.lookup_property(
    PropertyNames.Verb.value,
    "discuss"
)

# Query by action subject
scored_refs = await property_index.lookup_property(
    PropertyNames.Subject.value,
    "John"
)

Semantic Term Queries

The SemanticRefIndex provides term-based lookups:

# Lookup entities by name
scored_refs = await semantic_ref_index.lookup_term("project")

# Each scored_ref contains:
# - semantic_ref_ordinal: index into SemanticRefCollection
# - score: relevance score (1.0 for exact matches)

for scored_ref in scored_refs:
    semantic_ref = await semantic_refs.get_item(
        scored_ref.semantic_ref_ordinal
    )
    # semantic_ref.knowledge - ConcreteEntity, Action, or Topic
    # semantic_ref.range - TextRange pointing to message location

Query Scoping

TypeAgent implements sophisticated scoping to narrow results:

Action Scoping

When a query involves an action, TypeAgent scopes results to messages containing that specific subject-verb-object pattern:

# Query: "What did John discuss?"
# Compiled scope:
scope_defining_terms = SearchTermGroup("and", [
    PropertyTerm("subject", "John"),  # Actor
    PropertyTerm("verb", "discuss")   # Action verb
])

Only messages containing actions with John as subject AND discuss as verb are considered.

Temporal Scoping

Time range filters narrow results to specific periods:

# Query: "What happened last week?"
# Compiled time range:
when = WhenFilter(
    date_range=DateRange(
        start=datetime(2024, 1, 1),
        end=datetime(2024, 1, 7)
    )
)

# TimestampIndex maps timestamps to message ordinals
message_ordinals = await timestamp_index.get_messages_in_range(
    when.date_range.start,
    when.date_range.end
)

Entity Facet Scoping

Facets enable property-based filtering:

# Query: "Which blue cars were mentioned?"
# Compiled entity term:
entity = EntityTerm(
    name="car",
    type=["vehicle"],
    facets=[Facet(name="color", value="blue")]
)

# PropertyIndex lookups:
# 1. lookup_property("facet.name", "color")
# 2. lookup_property("facet.value", "blue")
# 3. Intersect with lookup_property("name", "car")

Result Fusion and Ranking

Results from multiple indexes are merged and ranked:

from typeagent.knowpro.search import run_search_query

# Execute search across all relevant indexes
query_results = await run_search_query(
    conversation,
    search_query_expr,
    search_options
)

# Results contain:
# - ConversationSearchResult objects
# - Each with scored semantic references
# - Ranked by relevance and combined scores

Scoring Strategy

Term Matches
Fuzzy Matches
Semantic Search

Exact term matches receive score 1.0:

ScoredSemanticRefOrdinal(
    semantic_ref_ordinal=42,
    score=1.0  # Exact match
)

Related terms use embedding similarity:

# RelatedTermsIndex with fuzzy matching
related = await related_terms_index.find_related(
    "discuss",
    max_distance=0.3
)
# Returns: [("talk", 0.85), ("speak", 0.78), ...]

ScoredSemanticRefOrdinal(
    semantic_ref_ordinal=42,
    score=0.85  # Similarity score
)

Message embeddings provide similarity scores:

# MessageTextIndex semantic search
similar_messages = await message_index.search(
    query_text="project discussion",
    top_k=10
)
# Returns messages sorted by cosine similarity

Fallback Mechanisms

TypeAgent implements multiple fallback strategies for robustness:

Verb Scope Fallback

# Primary query with verb matching
primary_query = SearchQueryExpr(
    select_expressions=[{
        "search_term_group": ...,
        "when": WhenFilter(
            scope_defining_terms=SearchTermGroup("and", [
                PropertyTerm("subject", "John"),
                PropertyTerm("verb", "discuss")  # Strict verb match
            ])
        )
    }]
)

results = await run_search_query(conversation, primary_query, options)

if not has_conversation_results(results):
    # Fallback: remove verb requirement
    fallback_query = SearchQueryExpr(
        select_expressions=[{
            "search_term_group": ...,
            "when": WhenFilter(
                scope_defining_terms=SearchTermGroup("and", [
                    PropertyTerm("subject", "John")
                    # No verb constraint - more permissive
                ])
            )
        }]
    )
    results = await run_search_query(conversation, fallback_query, options)

Semantic Similarity Fallback

When structured search finds no results, the system can fall back to pure semantic similarity using the MessageTextIndex. This ensures queries always return relevant results, even if the structured approach fails.

Answer Generation with Context

The final stage synthesizes answers using retrieved knowledge:

from typeagent.knowpro import answers

# Configure context options
answer_options = answers.AnswerContextOptions(
    entities_top_k=50,   # Include top 50 entities
    topics_top_k=50,     # Include top 50 topics
    messages_top_k=None  # Include all matched messages
)

# Generate answer from search results
_, combined_answer = await answers.generate_answers(
    answer_translator,
    search_results,
    conversation,
    question,
    options=answer_options
)

match combined_answer.type:
    case "Answered":
        return combined_answer.answer
    case "NoAnswer":
        return combined_answer.why_no_answer

The answer generator receives:

Entities: Structured entity objects with types and facets
Actions: Subject-verb-object triples with relationships
Topics: Relevant topics and keywords
Messages: Original message text for context

This rich context enables the LLM to generate accurate, grounded answers with proper citations.

Benefits of Structured RAG

Precision

Property-based queries enable exact filtering by entity types, facets, action subjects, and verbs.

Recall

Semantic search fallback ensures relevant results even when structured queries fail.

Composability

Combine multiple search criteria with boolean logic for complex queries.

Explainability

Structured queries make it clear why results were returned and what was matched.

Get Started

Core Concepts

Guides

Traditional RAG vs Structured RAG

Query Translation Pipeline

Stage 1: Language to Structure

Structured Search Query Schema

Multi-Index Search Strategy

Index Selection

Property-Based Queries

Semantic Term Queries

Query Scoping

Result Fusion and Ranking

Scoring Strategy

Fallback Mechanisms

Verb Scope Fallback

Semantic Similarity Fallback

Answer Generation with Context

Benefits of Structured RAG

Precision

Recall

Composability

Explainability

Next Steps

Knowledge Extraction

Indexing

Build docs developers (and LLMs) love

Get Started

Core Concepts

Guides

​Traditional RAG vs Structured RAG

​Query Translation Pipeline

​Stage 1: Language to Structure

​Structured Search Query Schema

​Multi-Index Search Strategy

​Index Selection

​Property-Based Queries

​Semantic Term Queries

​Query Scoping

​Result Fusion and Ranking

​Scoring Strategy

​Fallback Mechanisms

​Verb Scope Fallback

​Semantic Similarity Fallback

​Answer Generation with Context

​Benefits of Structured RAG

Precision

Recall

Composability

Explainability

​Next Steps

Knowledge Extraction

Indexing

Build docs developers (and LLMs) love

Traditional RAG vs Structured RAG

Query Translation Pipeline

Stage 1: Language to Structure

Structured Search Query Schema

Multi-Index Search Strategy

Index Selection

Property-Based Queries

Semantic Term Queries

Query Scoping

Result Fusion and Ranking

Scoring Strategy

Fallback Mechanisms

Verb Scope Fallback

Semantic Similarity Fallback

Answer Generation with Context

Benefits of Structured RAG

Next Steps