Social Pulse - Argument Cartographer

Overview

Social Pulse brings the voice of public discourse into every analysis. By integrating Twitter/X data, Argument Cartographer shows not just what experts and journalists say, but what everyday people are discussing, debating, and feeling about each topic.

Social Pulse uses the official Twitter API v2 to fetch recent, relevant tweets sorted by engagement and relevance.

Real-Time Data

Tweets from the last 7 days, refreshed with each analysis

AI Summary

Gemini generates neutral summaries of sentiment and discussion themes

Top Tweets

Most engaging tweets (likes, retweets) surface key viewpoints

Rich Metadata

Engagement metrics, timestamps, and author information

How It Works

Query Generation

When analyzing a topic, the AI generates a focused 2-4 word search query optimized for Twitter.Example: “Should AI be regulated?” → “AI regulation”

Twitter API Search

System searches Twitter using official API v2 with filters:

Language: English
Exclude: Retweets (original content only)
Sort: Relevancy (engagement-weighted)
Limit: 20 tweets

Data Enrichment

API returns:

Full tweet text
Author info (name, handle, profile image)
Engagement metrics (likes, retweets, replies, impressions)
Timestamp (UTC)

AI Analysis

Gemini reads all 20 tweets and generates:

Sentiment summary (positive/negative/mixed)
Key discussion points
Emerging themes
Notable perspectives

Ranking & Display

Top 5 tweets (by likes) are displayed in the Social Pulse panel

Twitter API Integration

Search Parameters

const searchParams = new URLSearchParams({
  'query': `${topic} lang:en -is:retweet`,
  'tweet.fields': 'created_at,author_id,public_metrics',
  'expansions': 'author_id',
  'user.fields': 'profile_image_url,username,name',
  'max_results': '20',
  'sort_order': 'relevancy' // Engagement-weighted
});

const response = await fetch(
  `https://api.twitter.com/2/tweets/search/recent?${searchParams}`,
  {
    headers: {
      'Authorization': `Bearer ${process.env.TWITTER_BEARER_TOKEN}`,
    },
  }
);

Query Filters

Language Filter
Retweet Exclusion
Relevancy Sort

lang:en - English tweets onlyWhy: Keeps analysis focused and enables accurate AI sentiment analysisFuture: Multi-language support planned

-is:retweet - Original content onlyWhy: Avoids duplicate content and surfaces original thoughts rather than amplificationResult: More diverse perspectives

sort_order: relevancy - Engagement-weightedWhy: Twitter’s algorithm surfaces tweets that are both topically relevant AND gaining tractionAlternative: recency for breaking news

Rate Limits

Twitter API Limits:

Free tier: 500,000 tweets/month read
Rate: 450 requests/15 minutes

At 1 search per analysis, you can run ~450 analyses per 15 minutes before hitting rate limits.

Data Schema

interface Tweet {
  id: string;
  text: string;
  author: {
    name: string; // Display name
    username: string; // @handle
    profile_image_url: string; // Avatar URL
  };
  created_at: string; // ISO 8601 timestamp
  public_metrics: {
    retweet_count: number;
    reply_count: number;
    like_count: number;
    impression_count: number; // How many saw it
  };
}

Example Tweet Object

{
  "id": "1234567890123456789",
  "text": "AI regulation is crucial for preventing misuse, but we need to avoid stifling innovation. Balance is key. #AIethics",
  "author": {
    "name": "Dr. Sarah Chen",
    "username": "sarahchen_ai",
    "profile_image_url": "https://pbs.twimg.com/profile_images/..."
  },
  "created_at": "2024-03-08T14:32:10.000Z",
  "public_metrics": {
    "retweet_count": 142,
    "reply_count": 38,
    "like_count": 891,
    "impression_count": 45230
  }
}

AI-Generated Summary

The “Social Pulse” summary is generated by Gemini analyzing all retrieved tweets:

const socialPulsePrompt = ai.definePrompt({
  name: 'socialPulsePrompt',
  input: { schema: z.object({ tweets: z.array(z.string()) }) },
  output: { 
    schema: z.object({ 
      socialPulse: z.string().describe(
        "A brief, neutral summary of public sentiment and key discussion points"
      ) 
    }) 
  },
  prompt: `You are an AI analyst. Analyze these tweets about a specific topic.
  Write a brief, neutral summary of:
  - Overall sentiment (positive/negative/mixed)
  - Key discussion points
  - Main themes of conversation
  
  Similar to the style of "Grok" on X/Twitter - objective and theme-focused.
  
  Tweets:
  - {{{tweets.join("\n- ")}}}`
});

Example Summaries

Mixed Sentiment
Polarized
Emerging Consensus

Public discourse on AI regulation shows mixed sentiment. 
Supporters emphasize safety concerns and ethical guardrails, 
citing risks of autonomous weapons and deepfakes. Critics 
worry about innovation constraints and definitional challenges 
around "AI." A common middle ground advocates for targeted 
regulation of high-risk applications rather than blanket rules.

The universal basic income debate is highly polarized on X. 
Proponents celebrate Finland's trial results and frame UBI as 
inevitable automation response. Opponents focus on cost concerns, 
work disincentives, and inflation risks. Little middle ground 
visible - most tweets express strong support or opposition.

Climate policy discourse shows emerging consensus on renewable 
investment, but sharp divides on timelines and carbon pricing. 
Youth voices demand urgency, while skeptics question cost-benefit 
ratios. Extreme weather events frequently cited as catalyst. 
Surprisingly, even critics now rarely deny climate change itself.

UI Components

Tweet Card Display

Tweets are rendered in visually rich cards:

export function TweetCard({ tweet }: { tweet: Tweet }) {
  return (
    <Card className="tweet-card">
      {/* Author Header */}
      <div className="flex items-start gap-3 p-4">
        <Avatar>
          <AvatarImage src={tweet.author.profile_image_url} />
          <AvatarFallback>{tweet.author.name[0]}</AvatarFallback>
        </Avatar>
        
        <div className="flex-1">
          <div className="font-semibold">{tweet.author.name}</div>
          <div className="text-sm text-muted-foreground">
            @{tweet.author.username}
          </div>
        </div>
        
        <a 
          href={`https://twitter.com/${tweet.author.username}/status/${tweet.id}`}
          target="_blank"
          rel="noopener"
          className="text-blue-500 hover:text-blue-600"
        >
          <ExternalLink className="h-4 w-4" />
        </a>
      </div>
      
      {/* Tweet Text */}
      <div className="px-4 pb-3">
        <p className="text-base whitespace-pre-wrap">{tweet.text}</p>
      </div>
      
      {/* Engagement Metrics */}
      <div className="flex items-center gap-4 px-4 pb-4 text-sm text-muted-foreground">
        <Metric icon={Heart} count={tweet.public_metrics.like_count} />
        <Metric icon={Repeat2} count={tweet.public_metrics.retweet_count} />
        <Metric icon={MessageCircle} count={tweet.public_metrics.reply_count} />
      </div>
      
      {/* Timestamp */}
      <div className="px-4 pb-4 text-xs text-muted-foreground">
        {formatRelativeTime(tweet.created_at)}
      </div>
    </Card>
  );
}

const Metric = ({ icon: Icon, count }: { icon: any; count: number }) => (
  <div className="flex items-center gap-1">
    <Icon className="h-4 w-4" />
    <span>{formatNumber(count)}</span>
  </div>
);

The side panel layout:

export function SocialView({ data }: { data: AnalysisResult }) {
  return (
    <div className="social-pulse-panel">
      {/* AI Summary */}
      <div className="mb-6">
        <h3 className="flex items-center gap-2 mb-3">
          <Rss className="h-5 w-5" />
          Social Pulse
        </h3>
        <p className="text-muted-foreground leading-relaxed">
          {data.socialPulse}
        </p>
      </div>
      
      {/* Tweet Feed */}
      <div className="space-y-4">
        <h4 className="text-sm font-semibold text-muted-foreground uppercase tracking-wide">
          Top Tweets
        </h4>
        
        {data.tweets.slice(0, 5).map(tweet => (
          <TweetCard key={tweet.id} tweet={tweet} />
        ))}
        
        {data.tweets.length === 0 && (
          <EmptyState message="No tweets found for this topic" />
        )}
      </div>
    </div>
  );
}

Benefits & Use Cases

Ground Truth Check

Compare expert analysis with grassroots public opinion

Sentiment Gauge

Quickly assess if topic is polarizing or consensus-building

Emerging Narratives

Spot new talking points before they reach mainstream media

Diverse Voices

Access perspectives beyond traditional news sources

Example Scenarios

Policy Debate

Scenario: Analyzing a proposed healthcare reform billExpert sources show policy details, cost estimates, impact studiesSocial Pulse reveals:

Personal stories of insurance struggles (emotional dimension)
Specific provisions people care about (vs. media focus)
Regional differences in sentiment
Misinformation spreading in real-time

Tech Controversy

Scenario: Debate over AI chatbot safetyExpert sources discuss technical specifications, risk assessmentsSocial Pulse reveals:

User experience stories (positive and negative)
Competing priorities (safety vs. accessibility)
Emerging use cases experts haven’t considered
Community-driven solutions

Cultural Issue

Scenario: Book banning in schoolsExpert sources provide legal analysis, educational researchSocial Pulse reveals:

Parent concerns (specific passages, age appropriateness)
Student perspectives (often missing from news)
Librarian professional insights
Community organizing efforts

Limitations

Twitter/X Data Isn’t RepresentativeSocial media users are:

Younger and more urban than general population
More politically engaged
More extreme in views (silent majority doesn’t tweet)
Subject to echo chambers and algorithmic amplification

Use Social Pulse as one data point, not gospel.

Known Issues

Low-Volume Topics

Problem: Niche topics may have < 20 tweetsResult: Empty state or sparse feedWorkaround: System gracefully handles empty results with informative message

Spam & Bots

Problem: Twitter has bot accounts and spamMitigation: Relevancy sort tends to filter these out (low engagement)Future: Implement bot detection scoring

Language Mixing

Problem: lang:en filter isn’t perfect - code-switching and multilingual tweets slip throughImpact: Occasionally irrelevant tweets in non-EnglishFuture: Secondary language detection

Recency Bias

Problem: API only returns last 7 days of tweets (free tier)Impact: Historical topics have no social dataWorkaround: “No recent tweets” message for older topics

Privacy & Ethics

All tweets displayed are public - we only access data users have chosen to share publicly on Twitter. No private DMs, protected accounts, or deleted content.

Ethical Considerations

Attribution: Every tweet links back to original with author credit
Context: We show full tweet text, not cherry-picked quotes
No amplification of hate: Future: content moderation filters
Transparency: Users see the AI’s summary AND raw tweets

Configuration

To run without Twitter integration:

# Remove or leave empty
TWITTER_BEARER_TOKEN=

The system will gracefully skip Twitter search and show “Social Pulse unavailable.”

Adjust Tweet Count

// src/ai/tools/twitter-search.ts
const searchParams = new URLSearchParams({
  // ...
  'max_results': '50', // Increase from 20 (max: 100)
});

// src/ai/flows/generate-argument-blueprint.ts
tweets: tweets.slice(0, 10), // Show top 10 instead of 5

Roadmap for additional social listening:

Community discussions, long-form debates, niche subreddits

YouTube Comments

Video-specific discourse, expert creator communities

News Comments

Direct reactions to journalism, contextual discussion

Mastodon/Fediverse

Decentralized social media, different demographics

Next Steps

Narrative Radar

See how Social Pulse integrates with curated topics

Creating Analyses

Tips for topics that generate rich social data

External Integrations

Technical details of Twitter API integration

Configuration

Customize social pulse behavior

Overview

Getting Started

Core Features

User Guide

Architecture

​Overview

Real-Time Data

AI Summary

Top Tweets

Rich Metadata

​How It Works

​Twitter API Integration

​Search Parameters

​Query Filters

​Rate Limits

​Data Schema

​Example Tweet Object

​AI-Generated Summary

​Example Summaries

​UI Components

​Tweet Card Display

​Social Pulse Panel

​Benefits & Use Cases

Ground Truth Check

Sentiment Gauge

Emerging Narratives

Diverse Voices

​Example Scenarios

​Limitations

​Known Issues

​Privacy & Ethics

​Ethical Considerations

​Configuration

​Disable Social Pulse

​Adjust Tweet Count

​Alternative Social Sources (Future)

Reddit

YouTube Comments

News Comments

Mastodon/Fediverse

​Next Steps

Narrative Radar

Creating Analyses

External Integrations

Configuration

Build docs developers (and LLMs) love

Overview

How It Works

Twitter API Integration

Search Parameters

Query Filters

Rate Limits

Data Schema

Example Tweet Object

AI-Generated Summary

Example Summaries

UI Components

Tweet Card Display

Social Pulse Panel

Benefits & Use Cases

Example Scenarios

Limitations

Known Issues

Privacy & Ethics

Ethical Considerations

Configuration

Disable Social Pulse

Adjust Tweet Count

Alternative Social Sources (Future)

Next Steps