Skip to main content
Sources are the first stage of the Home Mixer pipeline, responsible for retrieving candidate posts from various backend services. The Home Mixer implements two primary sources: Thunder (in-network) and Phoenix (out-of-network).

Overview

Sources implement the Source trait which provides an asynchronous get_candidates method that returns a list of PostCandidate objects. Each source can optionally implement an enable method to conditionally activate based on query parameters.

ThunderSource

The Thunder source retrieves in-network posts from users that the viewer follows. It queries the Thunder service’s AMP cluster to fetch posts from the user’s following list.

Implementation

pub struct ThunderSource {
    pub thunder_client: Arc<ThunderClient>,
}

Configuration

thunder_client
Arc<ThunderClient>
required
Client for communicating with the Thunder service clusters
cluster
ThunderCluster
default:"Amp"
The Thunder cluster to query (defaults to Amp cluster)
max_results
usize
Maximum number of posts to retrieve from Thunder service (configured via THUNDER_MAX_RESULTS)

Request Parameters

When querying Thunder, the source constructs a request with:
  • user_id: The viewer’s user ID
  • following_user_ids: List of user IDs the viewer follows
  • max_results: Maximum posts to retrieve
  • algorithm: Retrieval algorithm (defaults to “default”)
  • exclude_tweet_ids: Posts to exclude from results

Output Fields

tweet_id
i64
The unique identifier of the post
author_id
u64
The user ID of the post author
in_reply_to_tweet_id
Option<u64>
If the post is a reply, the ID of the parent post
ancestors
Vec<u64>
List of ancestor post IDs in the conversation thread. Includes the immediate parent and conversation root if different.
served_type
ServedType
Set to ForYouInNetwork to indicate this candidate came from the user’s network

Code Example

home-mixer/sources/thunder_source.rs
let request = GetInNetworkPostsRequest {
    user_id: query.user_id as u64,
    following_user_ids: following_list.iter().map(|&id| id as u64).collect(),
    max_results: p::THUNDER_MAX_RESULTS,
    exclude_tweet_ids: vec![],
    algorithm: "default".to_string(),
    debug: false,
    is_video_request: false,
};

let response = client
    .get_in_network_posts(request)
    .await
    .map_err(|e| format!("ThunderSource: {}", e))?;

let candidates: Vec<PostCandidate> = response
    .into_inner()
    .posts
    .into_iter()
    .map(|post| PostCandidate {
        tweet_id: post.post_id,
        author_id: post.author_id as u64,
        in_reply_to_tweet_id: post.in_reply_to_post_id,
        ancestors,
        served_type: Some(pb::ServedType::ForYouInNetwork),
        ..Default::default()
    })
    .collect();

PhoenixSource

The Phoenix source retrieves out-of-network posts using the Phoenix retrieval service. It leverages the user’s action sequence to find relevant posts from outside their following graph.

Implementation

pub struct PhoenixSource {
    pub phoenix_retrieval_client: Arc<dyn PhoenixRetrievalClient + Send + Sync>,
}

Configuration

phoenix_retrieval_client
Arc<dyn PhoenixRetrievalClient>
required
Client for communicating with the Phoenix retrieval service
max_results
usize
Maximum number of posts to retrieve (configured via PHOENIX_MAX_RESULTS)

Conditional Activation

Phoenix source includes an enable method that checks query parameters:
home-mixer/sources/phoenix_source.rs
fn enable(&self, query: &ScoredPostsQuery) -> bool {
    !query.in_network_only
}
The source is disabled when in_network_only is true, ensuring only Thunder results are used.

Requirements

Phoenix source requires the user_action_sequence to be present in the query. This sequence captures the user’s recent interactions and is used by Phoenix to understand user preferences.

Output Fields

tweet_id
i64
The unique identifier of the post
author_id
u64
The user ID of the post author
in_reply_to_tweet_id
Option<u64>
If the post is a reply, the ID of the parent post
served_type
ServedType
Set to ForYouPhoenixRetrieval to indicate this candidate came from Phoenix

Code Example

home-mixer/sources/phoenix_source.rs
let sequence = query
    .user_action_sequence
    .as_ref()
    .ok_or_else(|| "PhoenixSource: missing user_action_sequence".to_string())?;

let response = self
    .phoenix_retrieval_client
    .retrieve(user_id, sequence.clone(), p::PHOENIX_MAX_RESULTS)
    .await
    .map_err(|e| format!("PhoenixSource: {}", e))?;

let candidates: Vec<PostCandidate> = response
    .top_k_candidates
    .into_iter()
    .flat_map(|scored_candidates| scored_candidates.candidates)
    .filter_map(|scored_candidate| scored_candidate.candidate)
    .map(|tweet_info| PostCandidate {
        tweet_id: tweet_info.tweet_id as i64,
        author_id: tweet_info.author_id,
        in_reply_to_tweet_id: Some(tweet_info.in_reply_to_tweet_id),
        served_type: Some(pb::ServedType::ForYouPhoenixRetrieval),
        ..Default::default()
    })
    .collect();

Source Pipeline Integration

Both sources are typically used together in the Phoenix candidate pipeline. Thunder provides in-network candidates while Phoenix provides out-of-network candidates, creating a diverse feed that balances familiar content with discovery.

Typical Flow

  1. Thunder Source queries the user’s following graph
  2. Phoenix Source (if enabled) retrieves out-of-network recommendations
  3. Candidates from both sources are merged
  4. Subsequent pipeline stages apply filters and scoring

Error Handling

Both sources return Result<Vec<PostCandidate>, String>, allowing the pipeline to handle failures gracefully. Common error scenarios include:
  • No available service channels
  • Network timeouts
  • Missing required query fields
  • Service-level errors
  • Filters - Filter candidates after source retrieval
  • Scorers - Score candidates for ranking
  • Phoenix Scorer - Scores Phoenix candidates using ML predictions

Build docs developers (and LLMs) love