Skip to main content

​
Overview

The analyze-topic function is the main orchestrator that:
  1. Creates or retrieves a topic from the database
  2. Calls fetch-twitter, fetch-reddit, and fetch-youtube in parallel
  3. Runs analyze-sentiment on collected posts
  4. Returns a comprehensive status report
This is the primary entry point for triggering a complete sentiment analysis workflow.

​
Endpoint

POST https://your-project.supabase.co/functions/v1/analyze-topic

​
Request

​
query
string
required
The search query to analyze (e.g., “iPhone 16”, “climate change”)
​
title
string
Display title for the topic. Defaults to the query value if not provided.
​
hashtag
string
Associated hashtag for the topic. Auto-generated from query if not provided (e.g., “#iPhone16”).

​
Example Request

curl -X POST https://your-project.supabase.co/functions/v1/analyze-topic \
  -H "Authorization: Bearer YOUR_ANON_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "iPhone 16",
    "title": "iPhone 16 Launch",
    "hashtag": "#iPhone16"
  }'

​
Response

​
success
boolean
required
Whether the orchestration completed successfully
​
topic_id
string
required
UUID of the created or existing topic
​
twitter
object
Result from the Twitter/X fetch operation
​
reddit
object
Result from the Reddit fetch operation
​
youtube
object
Result from the YouTube fetch operation
​
analysis
object
Result from the sentiment analysis operation

​
Success Response Example

{
  "success": true,
  "topic_id": "a3f5e8b1-4c2d-4e9f-8a1b-3c5d6e7f8a9b",
  "twitter": {
    "success": true,
    "info": "Scrape.do (X: 18, Reddit: 12)",
    "scrape_status": "ok"
  },
  "reddit": {
    "success": true,
    "info": "Scrape.do (Reddit)",
    "scrape_status": "ok"
  },
  "youtube": {
    "success": true,
    "info": "YouTube Search API"
  },
  "analysis": {
    "success": true,
    "error": ""
  }
}

​
Partial Success Example

{
  "success": true,
  "topic_id": "a3f5e8b1-4c2d-4e9f-8a1b-3c5d6e7f8a9b",
  "twitter": {
    "success": false,
    "info": "fetch-twitter returned HTTP 402",
    "scrape_status": "quota"
  },
  "reddit": {
    "success": false,
    "info": "Reddit unavailable: blocked",
    "scrape_status": "blocked"
  },
  "youtube": {
    "success": true,
    "info": "YouTube Search API (fallback)"
  },
  "analysis": {
    "success": true,
    "error": ""
  }
}
The orchestrator is designed for graceful degradation. Even if some data sources fail, the function will continue with available sources and return success: true as long as at least one source provides data.

​
Error Response

{
  "error": "query is required"
}

​
Behavior

​
Topic Deduplication

The function checks if a topic with the same query already exists. If found, it reuses the existing topic ID instead of creating a duplicate. 
const { data: existingTopic } = await supabase
  .from('topics')
  .select('*')
  .eq('query', query)
  .limit(1);

if (existingTopic && existingTopic.length > 0) {
  topicId = existingTopic[0].id;
} else {
  // Create new topic
}

​
Fallback Chain

Twitter/X Data Collection:
  1. Scrape.do (X/Twitter + Reddit) - Primary
  2. Parallel AI Social Search - Secondary
  3. YouTube API - Tertiary
  4. Algorithmic generation - Final fallback
Reddit Data Collection:
  1. Scrape.do (Reddit HTML scraping)
YouTube Data Collection:
  1. YouTube Data API v3 (official)

​
Parallel Execution

All data fetching functions are called in parallel to minimize latency: 
// Calls happen simultaneously, not sequentially
await Promise.all([
  fetch(`${supabaseUrl}/functions/v1/fetch-twitter`, {...}),
  fetch(`${supabaseUrl}/functions/v1/fetch-reddit`, {...}),
  fetch(`${supabaseUrl}/functions/v1/fetch-youtube`, {...})
]);

​
Error Handling

The orchestrator uses soft error handling - it logs errors but continues execution. Only critical errors (like missing query parameter or database connection failures) return HTTP 500.

​
Common Errors

ErrorCauseHTTP Status
query is requiredMissing query parameter500
Topic not foundInvalid topic_id in database500
Database connection errorsSupabase credentials invalid500
Sub-function failuresScraping/API errors200 (soft fail)

​
Rate Limits

This function triggers multiple downstream API calls:
  • Scrape.do: 2-3 requests per call (X, Reddit, possible retries)
  • YouTube API: ~100-200 quota units per call
  • Gemini API: 1-2 requests per call (sentiment analysis + summary)
Avoid calling this endpoint more than 5 times per minute to stay within typical API quotas.

​
Performance

Typical execution time: 15-45 seconds
  • Fast path (all sources available): ~15-20s
  • Fallback path (scraping blocked): ~25-35s
  • Full fallback chain: ~40-45s

​
Best Practices

Use specific, focused queries for better data quality (“iPhone 16 Pro Max battery” vs “phones”)
Provide custom title and hashtag for better topic organization
Handle partial success scenarios in your client code
Store the returned topic_id to query results later via the database or API

Build docs developers (and LLMs) love

Get started for freeTalk to us