Skip to main content

​
Overview

The SideEffect trait defines asynchronous actions that run after candidate selection without affecting the pipeline result. Side effects are used for logging, caching, analytics, and other non-blocking operations.
Side effects run asynchronously and do not block the pipeline response. Failures in side effects do not affect the returned results.

​
Trait Definition

pub trait SideEffect<Q, C>: Send + Sync
where
    Q: Clone + Send + Sync + 'static,
    C: Clone + Send + Sync + 'static,
{
    fn enable(&self, query: Arc<Q>) -> bool;
    async fn run(&self, input: Arc<SideEffectInput<Q, C>>) -> Result<(), String>;
    fn name(&self) -> &'static str;
}

​
Type Parameters

​
Q
generic
The query type that contains request context and parametersConstraints: Clone + Send + Sync + 'static
​
C
generic
The candidate type representing selected resultsConstraints: Clone + Send + Sync + 'static

​
SideEffectInput

#[derive(Clone)]
pub struct SideEffectInput<Q, C> {
    pub query: Arc<Q>,
    pub selected_candidates: Vec<C>,
}
​
query
Arc<Q>
Shared reference to the original query that was processed
​
selected_candidates
Vec<C>
The final candidates that were selected and will be returned to the user

​
Methods

​
enable

​
enable
fn
Determines whether this side effect should run for the given query
fn enable(&self, query: Arc<Q>) -> bool
​
query
Arc<Q>
Shared reference to the query object
​
return
bool
Returns true if this side effect should run, false to skip. Default implementation returns true.

​
run

​
run
async fn
Performs the side effect operation asynchronously
async fn run(&self, input: Arc<SideEffectInput<Q, C>>) -> Result<(), String>
​
input
Arc<SideEffectInput<Q, C>>
Shared reference to the side effect input containing query and selected candidates
​
return
Result<(), String>
Returns Ok(()) on success, or an error message on failure. Errors are logged but do not affect the pipeline result.

​
name

​
name
fn
Returns a stable name for logging and metrics
fn name(&self) -> &'static str
​
return
&'static str
A short type name derived from the implementing struct

​
Example Implementation

Here’s a real example that caches request information for offline analysis: 
use xai_candidate_pipeline::side_effect::{SideEffect, SideEffectInput};
use tonic::async_trait;
use std::sync::Arc;

pub struct CacheRequestInfoSideEffect {
    pub strato_client: Arc<dyn StratoClient + Send + Sync>,
}

#[async_trait]
impl SideEffect<ScoredPostsQuery, PostCandidate> for CacheRequestInfoSideEffect {
    fn enable(&self, query: Arc<ScoredPostsQuery>) -> bool {
        // Only run in production and for For You requests
        env::var("APP_ENV").unwrap_or_default() == "prod" 
            && !query.in_network_only
    }

    async fn run(
        &self,
        input: Arc<SideEffectInput<ScoredPostsQuery, PostCandidate>>,
    ) -> Result<(), String> {
        let user_id: i64 = input.query.user_id;

        // Extract tweet IDs from selected candidates
        let post_ids: Vec<i64> = input
            .selected_candidates
            .iter()
            .map(|c| c.tweet_id)
            .collect();

        // Store the request info for later analysis
        let res = self.strato_client
            .store_request_info(user_id, post_ids)
            .await
            .map_err(|e| e.to_string())?;

        // Decode and verify the response
        let decoded: StratoResult<StratoValue<()>> = decode(&res);
        match decoded {
            StratoResult::Ok(_) => Ok(()),
            StratoResult::Err(_) => Err("error received from strato".to_string()),
        }
    }
}

​
Usage Notes

  • Side effects run asynchronously after the pipeline completes
  • They do not block the response being returned to the client
  • Failures are logged but do not affect the pipeline result
  • Multiple side effects can be configured and will run concurrently
  • Use Arc for sharing data efficiently between side effects
  • Side effects receive the final selected candidates, not intermediate results

​
Common Side Effect Types

​
Analytics & Logging

  • Log selected candidates for offline analysis
  • Track candidate source distributions
  • Record feature values for ML training

​
Caching

  • Cache selected results for fast re-serving
  • Pre-warm caches for related requests
  • Store impressions for deduplication

​
External Updates

  • Notify downstream systems of selections
  • Update user profiles based on served content
  • Trigger recommendation model updates

​
Monitoring

  • Emit metrics about selected candidates
  • Track content diversity
  • Monitor score distributions

​
Best Practices

  1. Non-Blocking: Keep operations fast; avoid expensive synchronous work
  2. Error Handling: Handle errors gracefully; don’t let side effects affect the response
  3. Idempotent: Design side effects to be safely retried if needed
  4. Conditional Execution: Use enable() to control when side effects run
  5. Resource Sharing: Use Arc to share clients and avoid cloning heavy objects
  6. Monitoring: Log side effect failures for debugging
  7. Timeouts: Implement timeouts for external service calls

​
Performance Considerations

  • Side effects run asynchronously and don’t add latency to the response
  • However, they still consume resources (CPU, memory, network)
  • Monitor side effect execution time and failure rates
  • Consider rate limiting side effects if they call external services
  • Use batching for efficiency when possible

​
Error Handling

impl SideEffect<Q, C> for MySideEffect {
    async fn run(&self, input: Arc<SideEffectInput<Q, C>>) -> Result<(), String> {
        // Try the operation, but don't panic on failure
        match self.client.send_data(&input).await {
            Ok(_) => Ok(()),
            Err(e) => {
                // Log the error for debugging
                eprintln!("Side effect failed: {}", e);
                // Return error but pipeline continues
                Err(e.to_string())
            }
        }
    }
}

​
Testing Side Effects

  • Test that side effects are enabled/disabled correctly
  • Verify data is formatted correctly for external systems
  • Mock external clients to test error handling
  • Ensure side effect failures don’t crash the pipeline

​
See Also

Build docs developers (and LLMs) love

Get started for freeTalk to us