System Architecture

TypeAgent implements a conversation-based knowledge processing system with dual storage backends and structured retrieval. The architecture consists of four main layers that work together to transform raw conversation data into searchable, structured knowledge.

Architectural Overview

The system follows a clear data flow pipeline:

Input → Storage Layer → Knowledge Extractor → Query Pipeline → Response
  ↓          ↓               ↓                    ↓             ↓
Text    Collections+    Entities/Topics      Parallel      Generated
Audio   6 Indexes       Actions/Relations    Index         Answer +
Images                                        Queries       Citations

Four Core Layers

Storage Layer
Knowledge Extractor
Query Pipeline
Integration Layer

The storage layer provides a dual-implementation pattern with identical interfaces but different backing stores:

Collections (2 types)

MessageCollection - Stores conversation messages with ordinal indexing
SemanticRefCollection - Stores semantic references and knowledge artifacts

Indexes (6 types)

Each index serves a specific query pattern:

SemanticRefIndex - Term → SemanticRef mappings for content discovery
PropertyIndex - Property name → SemanticRef mappings for structured queries
TimestampToTextRangeIndex - Temporal navigation and filtering
MessageTextIndex - Embedding-based semantic similarity search
RelatedTermsIndex - Term expansion and alias resolution
ConversationThreads - Thread organization and context grouping

Storage Providers

# Memory provider - fast, no persistence
from typeagent.storage.memory import MemoryStorageProvider

provider = MemoryStorageProvider(
    message_text_settings=message_settings,
    related_terms_settings=related_settings
)

# SQLite provider - persistent with identical API
from typeagent.storage.sqlite import SqliteStorageProvider

provider = SqliteStorageProvider(
    db_path="conversation.db",
    message_text_index_settings=message_settings
)

Both providers expose identical APIs through the IStorageProvider interface, enabling seamless switching between in-memory and persistent storage.

The knowledge extractor transforms raw conversation text into structured knowledge artifacts.

Extraction Modes

Basic Mode: Rule-based extraction from titles, headings, metadata
AI Mode: LLM-powered entity, topic, and relationship extraction
Hybrid Enhancement: Combines basic extraction with AI summarization
Batch Processing: Handles multiple content items with progress tracking

Core Components

from typeagent.knowpro.convknowledge import KnowledgeExtractor
from typeagent.aitools.model_adapters import create_chat_model

# Create extractor with custom model
extractor = KnowledgeExtractor(
    model=create_chat_model(),
    max_chars_per_chunk=2048
)

# Extract knowledge from text
result = await extractor.extract(message_text)
if isinstance(result, Success):
    knowledge = result.value
    # knowledge.entities - list of ConcreteEntity
    # knowledge.actions - list of Action
    # knowledge.topics - list of topic strings

The extractor uses TypeChat to translate natural language into structured KnowledgeResponse objects with entities, actions, and topics.

The query pipeline implements a structured RAG system that translates natural language queries into multi-stage retrieval operations.

Pipeline Stages

Language Processing: Parse user intent and extract search terms
Index Querying: Execute parallel searches across relevant indexes
Score Fusion: Merge and rank results from multiple sources
Context Building: Prepare enriched context for answer generation
Answer Generation: LLM-powered response synthesis with citations

Query Translation

from typeagent.knowpro import searchlang

# Search with natural language
search_options = searchlang.LanguageSearchOptions(
    compile_options=searchlang.LanguageQueryCompileOptions(
        exact_scope=False,
        verb_scope=True,
        apply_scope=True
    ),
    max_message_matches=25
)

result = await searchlang.search_conversation_with_language(
    conversation,
    query_translator,
    "What topics were discussed?",
    search_options
)

Fallback Strategy

When structured search fails to find matches, the system automatically falls back to raw text similarity search, ensuring graceful degradation.

The integration layer coordinates all components through a unified conversation interface.

Key Classes

from typeagent.knowpro.conversation_base import ConversationBase

# Create conversation with storage and indexing
conversation = await ConversationBase.create(
    settings=conversation_settings,
    name="my_conversation",
    tags=["project", "meeting"]
)

# Add messages with automatic indexing
result = await conversation.add_messages_with_indexing(
    messages=[msg1, msg2, msg3]
)

# Query end-to-end
answer = await conversation.query(
    "What decisions were made?"
)

Secondary Index Coordination

The ConversationSecondaryIndexes class provides unified access to all six indexes:

from typeagent.knowpro.secindex import ConversationSecondaryIndexes

# Automatically initialized by conversation
secondary_indexes = conversation.secondary_indexes

# Access individual indexes
property_index = secondary_indexes.property_to_semantic_ref_index
timestamp_index = secondary_indexes.timestamp_index
message_index = secondary_indexes.message_index

Key Design Principles

Storage Abstraction

Identical APIs for memory vs persistent storage enable seamless switching. Both MemoryStorageProvider and SqliteStorageProvider implement the same IStorageProvider interface, so your code works with either backend without changes.

# Transaction support works identically
async with storage_provider:
    await conversation.add_messages_with_indexing(messages)
    # SQLite: committed on success, rolled back on error
    # Memory: no rollback support

Parallel Indexing

Six specialized indexes support different query patterns and access paths:

SemanticRefIndex - Fast term lookup
PropertyIndex - Structured property queries (name, type, facets)
TimestampIndex - Temporal filtering
MessageTextIndex - Embedding similarity
RelatedTermsIndex - Fuzzy matching and synonyms
ConversationThreads - Context grouping

All indexes are updated incrementally during message ingestion.

Graceful Degradation

The system operates with basic extraction when AI models are unavailable:

Metadata extraction always works (titles, timestamps, basic fields)
LLM extraction is optional (controlled by auto_extract_knowledge setting)
Query fallback to text similarity when structured search finds nothing

This ensures the system remains functional even with limited resources.

Structured RAG

Combines semantic similarity with structured knowledge for precision and recall:

Structured first: Query specific properties (entities, actions, topics)
Semantic fallback: Use embeddings when structured search fails
Score fusion: Merge results from multiple indexes
Context building: Assemble relevant knowledge for LLM
Answer synthesis: Generate natural language response

Implementation Status

The architecture is fully implemented with production-ready components:

✅ Dual storage providers with full API parity
✅ Six-index architecture with unified testing
✅ Multi-mode knowledge extraction pipeline
✅ Natural language query processing with fallbacks
✅ Thread-aware search and context building
✅ Incremental indexing with transaction support
✅ Embedding consistency validation

Next Steps

Structured RAG

Learn how TypeAgent’s structured RAG differs from traditional RAG

Knowledge Extraction

Understand how AI models extract structured knowledge

Indexing

Deep dive into the six specialized indexes

API Reference

Explore the complete API documentation

Get Started

Core Concepts

Guides

System Architecture

Architectural Overview

Four Core Layers

Collections (2 types)

Indexes (6 types)

Storage Providers

Extraction Modes

Core Components

Pipeline Stages

Query Translation

Fallback Strategy

Key Classes

Secondary Index Coordination

Key Design Principles

Implementation Status

Next Steps

Structured RAG

Knowledge Extraction

Indexing

API Reference

Build docs developers (and LLMs) love

Get Started

Core Concepts

Guides

​Architectural Overview

​Four Core Layers

​Collections (2 types)

​Indexes (6 types)

​Storage Providers

​Extraction Modes

​Core Components

​Pipeline Stages

​Query Translation

​Fallback Strategy

​Key Classes

​Secondary Index Coordination

​Key Design Principles

​Implementation Status

​Next Steps

Structured RAG

Knowledge Extraction

Indexing

API Reference

Build docs developers (and LLMs) love

Architectural Overview

Four Core Layers

Collections (2 types)

Indexes (6 types)

Storage Providers

Extraction Modes

Core Components

Pipeline Stages

Query Translation

Fallback Strategy

Key Classes

Secondary Index Coordination

Key Design Principles

Implementation Status

Next Steps