AI Insights System
The AI Insights system uses Anthropic’s Claude API to generate strategic SEO recommendations based on report data. It provides intelligent, data-driven insights with fallback logic for reliability.Architecture Overview
Core Component
Class:AIInsightsGeneratorLocation:
src/lib/ai/insights-generator.ts:15
Data Flow
Claude API Integration
Initialization
Location:src/lib/ai/insights-generator.ts:11
ANTHROPIC_API_KEY
Model Configuration
Current Model:claude-sonnet-4-20250514
Location: src/lib/ai/insights-generator.ts:40
- Input: ~1500 tokens (report data + prompt)
- Output: ~500 tokens (5 insights in JSON)
- Total: ~2000 tokens per request
Prompt Engineering
Prompt Structure
Location:src/lib/ai/insights-generator.ts:87
Prompt Components
- Instruction Header - Defines output requirements
- Quality Criteria - Ensures actionable, specific insights
- Output Schema - JSON structure specification
- Example Insight - Shows ideal format
- Data Context - Client’s actual metrics
- Format Enforcement - Requests valid JSON
Full Prompt Template
Prompt Best Practices
- Specificity Enforcement - Requires referencing actual data
- Format Constraints - Demands JSON output for reliable parsing
- Quality Examples - Shows ideal insight structure
- Context Loading - Includes all relevant metrics
- Action Orientation - Forces actionable recommendations
Response Processing
Parsing Claude’s Response
Location:src/lib/ai/insights-generator.ts:156
Response Validation
Checks:- Valid JSON format
- Contains
insightsarray - Each insight has required fields
- Priority values are valid (
high|medium|low) - Category values are valid (
keyword|technical|content|performance)
Fallback System
Rule-Based Insights
Location:src/lib/ai/insights-generator.ts:182
Provides reliable insights when AI fails or API is unavailable.
Fallback Logic
Fallback Triggers
Conditions that trigger fallback:- API Error - Claude API unreachable or returns error
- Invalid JSON - Response cannot be parsed
- Missing Fields - Required insight properties missing
- Timeout - API request exceeds timeout limit
- Rate Limit - Anthropic API quota exceeded
Fallback Insight Categories
1. Mobile Performance
Trigger:mobileScore < 75Priority: High if score < 50, Medium otherwise
2. Keyword Position
Trigger:averagePosition > 8Priority: High
3. Content Expansion
Trigger: Always includedPriority: Medium
4. Technical SEO
Trigger: Always includedPriority: Medium
5. Performance Tracking
Trigger: Always includedPriority: Low
Insight Data Structure
AIInsight Type
Location: src/types/google-api.ts:123
Example Insight Object
Integration Points
Report Generation Pipeline
Used in:src/lib/reports/generate-report.ts (planned)
PDF Report Display
Component:src/lib/pdf/components/StrategicRecommendationsPage.tsx
Insights are rendered as a dedicated page in PDF reports with:
- Priority badges (High/Medium/Low)
- Category icons
- Expandable action items
- Expected impact callouts
Database Storage
Insights can be stored in the database for historical tracking:Error Handling
Comprehensive Logging
Location: Throughoutsrc/lib/ai/insights-generator.ts
Error Recovery Flow
- Catch API Error - Log full error details
- Log Fallback - Indicate switch to rule-based system
- Generate Fallback - Create 5 rule-based insights
- Return Successfully - Never throw error to report generation
Graceful Degradation
Philosophy: Reports should always succeed, even if AI fails.Performance Considerations
Timing
Typical Generation Time:- Claude API call: 2-5 seconds
- Parsing: < 100ms
- Total: ~3-5 seconds per report
Cost Optimization
Token Usage
- 100 reports/month = 200K tokens
- Cost: $0.60/month
- Per report: $0.006
Caching Strategy
Recommended: Cache insights for duplicate report requestsTesting
Test Cases
1. Successful AI Generation
2. Fallback on API Error
3. Invalid JSON Response
Best Practices
Prompt Optimization
- Include Examples - Show Claude ideal output format
- Reference Data - Require specific metric references
- Constrain Output - Request exact JSON structure
- Action-Oriented - Demand actionable recommendations
- Quantify Impact - Ask for measurable expected outcomes
Error Handling
- Never Fail Reports - Always return insights (AI or fallback)
- Log Extensively - Track AI performance and failures
- Monitor Costs - Track token usage and API costs
- Fallback Quality - Ensure rule-based insights are valuable
Data Preparation
- Clean Input - Sanitize data before sending to Claude
- Format Numbers - Present metrics clearly
- Prioritize Context - Include most relevant data first
- Limit Length - Keep prompts under 2000 tokens
Monitoring
Metrics to Track
- AI Success Rate - % of reports using AI vs fallback
- Generation Time - Average time to generate insights
- Token Usage - Total tokens consumed per report
- Cost per Report - Track actual API costs
- Error Types - Categorize failures (API, parsing, timeout)