Skip to main content

​
Overview

Home Mixer is the primary orchestration service that powers the For You feed on X. It coordinates multiple subsystems to retrieve, rank, and filter posts, returning a personalized feed for each user. The service leverages the Candidate Pipeline framework to compose a multi-stage recommendation pipeline, combining both in-network content (from Thunder) and out-of-network content (from Phoenix Retrieval).

​
Architecture

Home Mixer exposes a gRPC endpoint (ScoredPostsService) that processes feed requests through a series of pipeline stages: 
pub struct HomeMixerServer {
    phx_candidate_pipeline: Arc<PhoenixCandidatePipeline>,
}

​
Pipeline Stages

The PhoenixCandidatePipeline implements the complete feed generation flow:
1

Query Hydration

Fetch user context before retrieving candidates:
  • User Action Sequence: Recent engagement history (likes, replies, reposts)
  • User Features: Following list, preferences, language, country
2

Candidate Sourcing

Retrieve candidates from multiple sources in parallel:
  • Thunder: In-network posts from followed accounts
  • Phoenix Retrieval: Out-of-network posts discovered via ML-based similarity search
3

Hydration

Enrich candidates with additional metadata:
  • Core post data (text, media, timestamps)
  • Author information (username, verification status)
  • Video duration for video posts
  • Subscription eligibility
4

Pre-Scoring Filters

Remove ineligible candidates:
  • Duplicates and reposts of same content
  • Posts older than retention threshold
  • Self-posts (user’s own tweets)
  • Blocked/muted authors
  • Muted keywords
  • Previously seen/served posts
5

Scoring

Apply multiple scorers sequentially:
  • Phoenix Scorer: ML predictions from transformer model
  • Weighted Scorer: Combine predictions into relevance score
  • Author Diversity Scorer: Attenuate repeated authors
  • OON Scorer: Adjust out-of-network content scores
6

Selection

Sort by final score and select top K candidates
7

Post-Selection Processing

Final validation and cleanup:
  • Visibility filtering (spam, violence, gore detection)
  • Conversation deduplication
8

Side Effects

Cache request information for future optimizations

​
Implementation

The pipeline is constructed using the builder pattern, assembling all components:
home-mixer/candidate_pipeline/phoenix_candidate_pipeline.rs
impl PhoenixCandidatePipeline {
    async fn build_with_clients(
        uas_fetcher: Arc<UserActionSequenceFetcher>,
        phoenix_client: Arc<dyn PhoenixPredictionClient>,
        phoenix_retrieval_client: Arc<dyn PhoenixRetrievalClient>,
        thunder_client: Arc<ThunderClient>,
        // ... other clients
    ) -> PhoenixCandidatePipeline {
        // Query Hydrators
        let query_hydrators = vec![
            Box::new(UserActionSeqQueryHydrator::new(uas_fetcher)),
            Box::new(UserFeaturesQueryHydrator { strato_client }),
        ];

        // Sources
        let sources = vec![
            Box::new(PhoenixSource { phoenix_retrieval_client }),
            Box::new(ThunderSource { thunder_client }),
        ];

        // ... filters, scorers, selectors
    }
}

​
Request Flow

When a user requests their For You feed:
home-mixer/server.rs
#[tonic::async_trait]
impl ScoredPostsService for HomeMixerServer {
    async fn get_scored_posts(
        &self,
        request: Request<ScoredPostsQuery>,
    ) -> Result<Response<ScoredPostsResponse>, Status> {
        let query = ScoredPostsQuery::new(
            proto_query.viewer_id,
            proto_query.client_app_id,
            proto_query.country_code,
            proto_query.language_code,
            proto_query.seen_ids,
            proto_query.served_ids,
            proto_query.in_network_only,
            // ...
        );
        
        let pipeline_result = self.phx_candidate_pipeline.execute(query).await;
        
        let scored_posts: Vec<ScoredPost> = pipeline_result
            .selected_candidates
            .into_iter()
            .map(|candidate| ScoredPost { /* ... */ })
            .collect();
            
        Ok(Response::new(ScoredPostsResponse { scored_posts }))
    }
}

​
Key Components

​
Sources

Two primary candidate sources run in parallel:

Thunder Source

Retrieves recent posts from accounts the user follows (in-network)

Phoenix Source

ML-based retrieval of relevant posts from the global corpus (out-of-network)

​
Filters

Home Mixer applies extensive filtering to ensure feed quality:
FilterPurpose
DropDuplicatesFilterRemove duplicate post IDs
CoreDataHydrationFilterRemove posts missing core metadata
AgeFilterRemove posts older than threshold
SelfTweetFilterRemove user’s own posts
RetweetDeduplicationFilterDedupe reposts of same content
IneligibleSubscriptionFilterRemove paywalled content user can’t access
PreviouslySeenPostsFilterRemove already seen posts
PreviouslyServedPostsFilterRemove posts served in current session
MutedKeywordFilterRemove posts with muted keywords
AuthorSocialgraphFilterRemove blocked/muted authors
VFFilterRemove spam/violence/gore
DedupConversationFilterDeduplicate conversation threads

​
Scorers

Multiple scorers combine to produce the final relevance score:
1

Phoenix Scorer

Calls the Phoenix ranking model to get ML predictions for each engagement type (like, reply, repost, etc.)
2

Weighted Scorer

Combines multiple predictions into a single relevance score using weighted sum
3

Author Diversity Scorer

Attenuates scores for posts from the same author to ensure feed diversity
4

OON Scorer

Adjusts scores for out-of-network content based on user preferences

​
Configuration

Key parameters controlling pipeline behavior: 
// Maximum results from each source
pub const THUNDER_MAX_RESULTS: u32 = 1000;
pub const PHOENIX_MAX_RESULTS: usize = 500;

// Final result size
pub const RESULT_SIZE: usize = 100;

// Post age limits
pub const MAX_POST_AGE: u64 = 86400 * 2; // 2 days

​
Performance Characteristics

The pipeline is highly optimized for low latency:
  • Parallel execution: Sources and hydrators run concurrently
  • Sub-millisecond lookups: Thunder provides in-memory access to recent posts
  • Batch prediction: Phoenix scores all candidates in a single inference call
  • Streaming responses: gRPC with Zstd compression for efficient transport
Typical request latency: 100-300ms end-to-end

​
Monitoring

Home Mixer emits detailed metrics at each pipeline stage: 
#[xai_stats_macro::receive_stats]
async fn get_scored_posts(...) -> Result<...> {
    // Automatic instrumentation tracks:
    // - Request count and latency
    // - Candidates from each source
    // - Filtered vs kept candidates
    // - Final result size
}

Candidate Pipeline

Framework that powers Home Mixer’s orchestration

Phoenix

ML component for retrieval and ranking

Thunder

In-memory post store for in-network content

​
Example Request

grpcurl -d '{
  "viewer_id": 12345,
  "client_app_id": "web",
  "country_code": "US",
  "language_code": "en",
  "seen_ids": [],
  "served_ids": [],
  "in_network_only": false
}' \
  home-mixer.example.com:443 \
  xai.ScoredPostsService/GetScoredPosts
Set in_network_only: true to disable Phoenix retrieval and only return posts from followed accounts.

Build docs developers (and LLMs) love

Get started for freeTalk to us