Decagon Assistant Example

​

Project Overview

​

Key Features

• Real-time streaming responses via Claude API with token-by-token delivery
• Multi-turn context retention with sliding window + summarization
• Cross-session memory - assistant remembers user preferences and context
• Persistent conversation history stored in PostgreSQL
• Production-ready deployment on Vercel with proper error handling

​

Product Requirements

​

Product Statement

We are building a developer workflow copilot — a conversational AI assistant that feels humanlike and natural — for the Decagon AI challenge. The assistant is a full-stack Next.js application powered by the Anthropic Claude API, deployed publicly, featuring real-time streaming responses, persistent cross-session memory, and deep multi-turn context understanding.

​

Success Criteria (Ranked)

1. Naturalness: Conversations feel indistinguishable from chatting with a sharp, friendly senior engineer
2. Context retention: Assistant remembers everything within a session and key facts across sessions
3. Utility: Genuinely helps developers with code review, debugging, architecture guidance

​

Hard Limits

• Time budget: 24 hours from first commit to deployed demo
• Resources: Vercel hobby tier, Neon/Supabase free tier Postgres
• External services: Anthropic Claude API only (no other paid APIs)
• Runtime mode: Deployed publicly on Vercel with HTTPS

​

Project Structure

src/
  app/                    # Next.js App Router
    page.tsx              # Main chat page
    layout.tsx            # Root layout with providers
    api/
      chat/route.ts       # Streaming chat completion endpoint
      conversations/route.ts  # Conversation CRUD
      memory/route.ts     # User memory endpoints
  components/             # React UI components  
    ChatWindow.tsx        # Main chat container
    MessageBubble.tsx     # Individual message display
    InputBar.tsx          # Message input with send button
    Sidebar.tsx           # Conversation list sidebar
    CodeBlock.tsx         # Syntax-highlighted code rendering
  lib/                    # Core business logic
    anthropic.ts          # Claude API client + streaming
    conversation.ts       # Conversation CRUD + context windowing  
    memory.ts             # Cross-session memory extraction
    system-prompt.ts      # Dynamic prompt assembly
  types/
    index.ts              # Message, Conversation, Memory types
prisma/
  schema.prisma           # Database schema
__tests__/                # Tests mirroring src structure

​

Acceptance Tests

# Build and type checking
npm run build  # Zero errors and TypeScript warnings
npm run test   # All unit and integration tests pass
npm run lint   # Zero ESLint issues

• Opening deployed URL shows chat interface within 2 seconds
• Sending “Hello, I’m working on a React project” then “Can you help debug useEffect?” produces response referencing React context without re-asking
• Refreshing page and asking “What were we talking about?” returns response referencing previous conversation
• Streaming tokens appear within 500ms (no full-response delay)
• 10-message conversation maintains context - 10th response references details from messages 1-3
• Sending code snippet with bug produces response identifying bug, explaining why, and providing correction

​

Specification Documents

​

SPEC.md - Complete Product Specification

• Real-time streaming chat responses with incremental token delivery
• Multi-turn conversation with context window management
• Persistent conversation history in Postgres
• Cross-session user memory
• Dynamic system prompt with memory injection
• Polished chat UI with markdown and code highlighting
• Deployed to Vercel

• Conversation title auto-generation
• Code block copy button
• Keyboard shortcuts (Enter, Shift+Enter, Ctrl+N)
• Dark mode toggle
• Conversation search/filter

• User authentication / multi-user (single-user demo)
• Voice input/output
• File upload or image analysis
• Tool use / function calling
• Rate limiting (demo context)
• Analytics
• Mobile native app

​

AGENTS.md - Execution Policies

## Code Style
- Next.js App Router: page.tsx for pages, route.ts for API handlers
- TypeScript strict mode - explicit parameter and return types
- React functional components with hooks (no classes)
- Named exports (except Next.js pages requiring default)
- File naming: kebab-case.ts for modules, PascalCase.tsx for components
- Functions under 50 lines - extract helpers
- async/await everywhere (no .then() chains)
- Tailwind utility classes only

## Dependencies  
Allowed runtime: next, react, @anthropic-ai/sdk, @prisma/client, 
                 zod, nanoid, date-fns
Allowed dev: typescript, prisma, tailwindcss, jest, 
             @testing-library/react, eslint, prettier
             
Banned: CSS-in-JS, state libraries (Redux/Zustand), LangChain, 
        real-time frameworks (socket.io)

## Testing Policy
- Test files in __tests__/ mirroring src/ structure  
- Unit tests for all src/lib/ modules (core logic)
- Component tests for ChatWindow, MessageBubble
- API route tests with mocked request/response
- Minimum coverage: every exported function has happy + error path test

• Chat endpoint uses ReadableStream and TransformStream
• Response content type: text/event-stream or application/octet-stream
• Client uses fetch with response.body.getReader() (no polling/WebSocket)
• Handle backpressure - client disconnect closes stream cleanly
• Each chunk must be valid parseable unit

• Messages stored in Postgres immediately (not batched)
• Sliding window: last 20 messages verbatim + summary of older messages
• Memory extraction runs after each assistant response (async)
• Memory retrieval before system prompt assembly
• Memory storage: simple key-value { userId, key, value, updatedAt }

​

Architecture Decisions

​

Direct Anthropic SDK over Vercel AI SDK

• Date: 2026-02-14
• Decision: Use @anthropic-ai/sdk directly instead of Vercel AI SDK or LangChain
• Why: Full control over streaming, system prompts, and context window management. Vercel AI SDK abstracts away details needed for natural conversation tuning
• Alternatives: Vercel AI SDK (simpler but less control), LangChain (too heavy), raw fetch (too low-level)
• Status: active

​

Next.js App Router over Pages Router

• Date: 2026-02-14
• Decision: Use Next.js 14 App Router with server components and route handlers
• Why: Route handlers provide native ReadableStream streaming. Server components reduce bundle size. Current standard for modern Next.js
• Alternatives: Pages Router (stable but older), Express + React SPA (more infra), Remix (less Vercel-native)
• Status: active

​

Postgres with Prisma over In-Memory

• Date: 2026-02-14
• Decision: PostgreSQL (Neon) with Prisma ORM for all persistence
• Why: Cross-session memory requires durability. localStorage doesn’t survive incognito. In-memory doesn’t survive serverless cold starts. Prisma provides type-safe database access
• Alternatives: SQLite (no Vercel serverless), Supabase (heavier), Redis (not primary storage), Vercel KV (limited queries)
• Status: active

​

SSE-Style Streaming over WebSockets

• Date: 2026-02-14
• Decision: Server-sent events pattern (ReadableStream) instead of WebSockets
• Why: Vercel serverless doesn’t support persistent WebSocket connections. SSE works with native fetch API, streams through Vercel edge cleanly. Request-response pattern doesn’t need full duplex
• Alternatives: WebSocket via socket.io (Vercel incompatible), polling (bad UX), Edge Runtime WebSocket (experimental)
• Status: active

​

Sliding Window + Summary for Context

• Date: 2026-02-14
• Decision: Keep last 20 messages verbatim, summarize older with Claude Haiku
• Why: Preserves conversational flow while managing context window. Haiku summarization is fast and cheap. 20 messages provides enough recency
• Alternatives: Full truncation (loses context), embedding retrieval (over-engineered), no summarization (loses early context)
• Status: active

​

Key-Value Memory over Vector Store

• Date: 2026-02-14
• Decision: Store memories as simple key-value pairs in Postgres
• Why: Memory needs are structured (name, tech stack, preferences). Can inject all memories into system prompt. Vector search is overkill
• Alternatives: Pinecone/Weaviate (unnecessary), JSON blob (harder to query), pgvector (adds extension dependency)
• Status: active

​

Setup and Execution

​

Prerequisites

Node.js 20.x or later
npm 10.x or later  
PostgreSQL 15+ (local) OR Neon/Supabase connection string
Anthropic API key with claude-sonnet-4-5-20250929 access

​

Quick Start

git clone <repo-url> decagon-assistant
cd decagon-assistant
npm install

cp .env.example .env

ANTHROPIC_API_KEY=sk-ant-...
DATABASE_URL=postgresql://user:password@localhost:5432/decagon_assistant
NEXT_PUBLIC_APP_URL=http://localhost:3000

npx prisma generate
npx prisma db push  

npm run dev

​

Verification

npm run build    # TypeScript compilation, zero errors
npm run test     # All tests pass
npm run lint     # Zero ESLint issues

1. Open http://localhost:3000, send message, verify streaming response within 500ms
2. Send 3 messages in sequence, verify 3rd response references context from 1st
3. Refresh page, send “What were we talking about?” - verify it recalls prior conversation

​

Deployment

​

Vercel Deployment

git push origin main

ANTHROPIC_API_KEY=sk-ant-...
DATABASE_URL=postgresql://...

​

Operational Runbook

​

Monitoring

• Vercel function logs: Dashboard > Project > Functions > Logs
• Local dev: stdout from npm run dev
• Database queries: Set DEBUG=prisma:query in .env

• Anthropic API response latency (first token) - target < 500ms
• Database query latency - watch for connection pool exhaustion
• Vercel function duration - must stay under 10s (streaming keeps alive)
• Error rate on /api/chat - any 500s indicate streaming/API issues

• “Something went wrong” error → Check Vercel logs for Anthropic API errors
• Messages load but no streaming → Check Content-Type header and ReadableStream
• Empty conversation history → Database connection issue, verify DATABASE_URL
• Slow first message after idle → Vercel cold start + Prisma connection (expected 2-3s)

​

Recovery Procedures

1. Check Neon/Supabase dashboard for database status
2. Verify DATABASE_URL in Vercel environment variables
3. If connection pool exhausted: redeploy on Vercel to reset serverless instances
4. If schema drift: run npx prisma db push against production database
5. Verify recovery: open URL, send message, confirm persistence after refresh

# Drop and recreate database (destroys all data)
npx prisma db push --force-reset

# Redeploy on Vercel  
# Dashboard > Deployments > Redeploy

# Verify streaming works

​

Key Learnings

​

What Worked Well

1. Direct SDK control: Using @anthropic-ai/sdk directly provided precise control over streaming and prompts without framework abstractions
2. Sliding window + summary: Balances context retention with token efficiency
3. Prisma + Postgres: Type-safe database access that survives serverless cold starts
4. SSE over WebSocket: Works seamlessly with Vercel serverless

​

Challenges Addressed

1. Vercel serverless cold starts: Mitigated with Prisma connection pooling and streaming to keep functions alive
2. Context window management: Solved with 20-message verbatim window + Haiku summarization of older messages
3. Memory persistence: Key-value model simpler and more maintainable than vector store for structured preferences

​

Next Steps