PostToolUse Hook → Save raw tool outputs → Retrieve everything on startup

SessionStart loaded:
- 150 file read operations
- 80 grep searches
- 45 bash commands
Total: ~35,000 tokens loaded
Relevant to current task: ~500 tokens (1.4%)

PostToolUse Hook → Queue observation → SDK Worker → AI compression → Store insights

❌ Old: Load 50 observations upfront    → 8,500 tokens
✅ New: Show index of 50 observations   →   800 tokens
        Agent fetches 2–3 relevant ones →   300 tokens
        Total: 1,100 tokens (87% savings)

// ❌ Wrong assumption
UserPromptSubmit → Capture session ID once → Use forever

// ✅ Actual behavior
Turn 1: session_abc123
Turn 2: session_def456
Turn 3: session_ghi789

// Kills worker immediately — summaries get interrupted
SessionEnd → DELETE /worker/session → Worker stops

// Mark complete, let worker finish on its own
SessionEnd → UPDATE sdk_sessions SET completed_at = NOW()
Worker sees completion → Finishes processing → Exits naturally

// ✅ Streaming input mode
async function* messageGenerator(): AsyncIterable<UserMessage> {
  yield { role: "user", content: "You are a memory assistant..." };

  while (session.status === 'active') {
    const observations = await pollQueue();
    for (const obs of observations) {
      yield { role: "user", content: formatObservation(obs) };
    }
    await sleep(1000);
  }
}

const response = query({
  prompt: messageGenerator(),
  options: { maxTurns: 1000 }
});

┌─────────────────────────────────────────────────────────┐
│              CLAUDE CODE SESSION                         │
│  User → Claude → Tools (Read, Edit, Write, Bash)        │
│                    ↓                                     │
│              PostToolUse Hook                            │
│              (queues observation)                        │
└─────────────────────────────────────────────────────────┘
                     ↓ SQLite queue
┌─────────────────────────────────────────────────────────┐
│              SDK WORKER PROCESS                          │
│  ONE streaming session per Claude Code session          │
│                                                          │
│  AsyncIterable<UserMessage>                             │
│    → Yields observations from queue                     │
│    → SDK compresses via AI                              │
│    → Parses XML responses                               │
│    → Stores in database                                 │
└─────────────────────────────────────────────────────────┘
                     ↓ SQLite storage
┌─────────────────────────────────────────────────────────┐
│              NEXT SESSION                                │
│  SessionStart Hook                                       │
│    → Queries database                                    │
│    → Returns progressive disclosure index               │
│    → Agent fetches details via MCP                      │
└─────────────────────────────────────────────────────────┘

CREATE TABLE observations (
  id         INTEGER PRIMARY KEY,
  session_id TEXT,
  text       TEXT,
  created_at INTEGER
);

CREATE TABLE observations (
  id              INTEGER PRIMARY KEY AUTOINCREMENT,
  session_id      TEXT NOT NULL,
  project         TEXT NOT NULL,
  title           TEXT NOT NULL,         -- Progressive disclosure metadata
  subtitle        TEXT,
  type            TEXT NOT NULL,         -- decision, bugfix, feature, etc.
  narrative       TEXT NOT NULL,
  facts           TEXT,                  -- JSON array
  concepts        TEXT,                  -- JSON array of tags
  files_read      TEXT,                  -- JSON array
  files_modified  TEXT,                  -- JSON array
  created_at      TEXT NOT NULL,
  created_at_epoch INTEGER NOT NULL,
  FOREIGN KEY(session_id) REFERENCES sdk_sessions(id)
);

-- FTS5 for full-text search (100x faster than LIKE queries)
CREATE VIRTUAL TABLE observations_fts USING fts5(
  title, subtitle, narrative, facts, concepts,
  content=observations
);

# Broken output (v4.3.0)
npm WARN deprecated [email protected]...
npm WARN deprecated ...
{"hookSpecificOutput": {"additionalContext": "..."}}

// ❌ Vulnerable
const results = db.query(
  `SELECT * FROM observations_fts WHERE observations_fts MATCH '${userQuery}'`
);

// ✅ Safe
const results = db.query(
  'SELECT * FROM observations_fts WHERE observations_fts MATCH ?',
  [userQuery]
);

Text query → SQLite FTS5 (keyword matching)
                   ↓
         Chroma vector search (semantic)
                   ↓
           Merge + re-rank results

async function ensureWorkerHealthy() {
  const healthy = await isWorkerHealthy(1000);
  if (!healthy) {
    await startWorker();
    await waitForWorkerHealth(10000);
  }
}

const currentVersion = getPackageVersion();
const installedVersion = readFileSync('.install-version', 'utf-8');

if (currentVersion !== installedVersion) {
  await runNpmInstall();
  writeFileSync('.install-version', currentVersion);
}

GET /              # Viewer HTML
GET /stream        # SSE real-time events
GET /api/prompts   # Paginated user prompts
GET /api/observations  # Paginated observations
GET /api/summaries # Paginated session summaries
GET /api/stats     # Database statistics

9+ MCP tools registered at session start:
- search_observations
- find_by_type
- find_by_file
- find_by_concept
- get_recent_context
- get_observation
- get_session
- get_prompt
- help

Problems:
- Overlapping operations (search_observations vs find_by_type)
- Complex parameter schemas (~2,500 tokens in tool definitions)
- No built-in workflow guidance
- High cognitive load — which tool to use?
- Code size: ~2,718 lines in mcp-server.ts

4 MCP tools following 3-layer progressive disclosure:

1. __IMPORTANT — Workflow documentation (always visible)
   "3-LAYER WORKFLOW (ALWAYS FOLLOW):
    1. search(query)        → Get index with IDs
    2. timeline(anchor=ID)  → Get context around that point
    3. get_observations([IDs]) → Fetch full details only if needed
    NEVER fetch full details without filtering first."

2. search         — Layer 1: Index (~50–100 tokens/result)
3. timeline       — Layer 2: Chronological context
4. get_observations — Layer 3: Full details (~500–1,000 tokens/result)

Traditional approach:
  Fetch 20 observations upfront → 10,000–20,000 tokens
  Only 2 relevant (90% waste)

3-layer workflow:
  search (20 results)         → 1,000–2,000 tokens
  Identify 3 relevant IDs     →     0 tokens
  get_observations (3 IDs)    → 1,500–3,000 tokens
  Total: 2,500–5,000 tokens   (50–75% savings)