Overview
The RetryPolicy trait defines how the driver decides whether and how to retry failed requests. Policies can retry on the same node, retry on a different node, fail immediately, or ignore write errors.
Source: scylla/src/policies/retry/retry_policy.rs:37
Trait Definition
pub trait RetryPolicy: std::fmt::Debug + Send + Sync {
fn new_session(&self) -> Box<dyn RetrySession>;
}
Method
new_session
Creates a new retry session for a single request.
fn new_session(&self) -> Box<dyn RetrySession>
A new retry session that will decide retry behavior for a request
RetrySession to track retry state.
RetrySession
Tracks retry state for a single request and makes retry decisions.
pub trait RetrySession: Send + Sync {
fn decide_should_retry(&mut self, request_info: RequestInfo) -> RetryDecision;
fn reset(&mut self);
}
Methods
decide_should_retry
Decides what to do after a request failure.
fn decide_should_retry(&mut self, request_info: RequestInfo) -> RetryDecision
Information about the failed request
How to handle the failure
reset
Resets the session state before reusing it for a new request.
Supporting Types
RequestInfo
Information about a failed request.
pub struct RequestInfo<'a> {
pub error: &'a RequestAttemptError,
pub is_idempotent: bool,
pub consistency: Consistency,
}
The error that caused the request to fail
Whether the request is safe to retry (can be applied multiple times)
The consistency level of the failed request
RetryDecision
Instructs the driver how to handle a failed request.
pub enum RetryDecision {
RetrySameTarget(Option<Consistency>),
RetryNextTarget(Option<Consistency>),
DontRetry,
IgnoreWriteError,
}
Retry on the same node and shard. Optional consistency overrides the original.
Retry on the next node from the load balancing plan. Optional consistency overrides the original.
Stop retrying and return the error to the user.
Return an empty successful response, ignoring the error.
Some(consistency) overrides the request consistency. None keeps the same consistency.
DefaultRetryPolicy
The default retry policy with sensible retry behavior.
pub struct DefaultRetryPolicy {
// ... internal fields
}
Creation
use scylla::policies::retry::DefaultRetryPolicy;
use std::sync::Arc;
let policy = Arc::new(DefaultRetryPolicy::new());
Behavior
- Read Timeout: Retry if enough replicas responded
- Write Timeout:
- Retry unlogged batch on same node
- Retry other writes on next node
- Unavailable: Retry on next node
- Idempotence: Only retry idempotent requests on network errors
Example:
use scylla::{SessionBuilder, policies::retry::DefaultRetryPolicy};
use std::sync::Arc;
let policy = Arc::new(DefaultRetryPolicy::new());
let session = SessionBuilder::new()
.known_node("127.0.0.1:9042")
.retry_policy(policy)
.build()
.await?;
FallthroughRetryPolicy
Never retries - immediately returns errors to the user.
pub struct FallthroughRetryPolicy;
Creation
use scylla::policies::retry::FallthroughRetryPolicy;
use std::sync::Arc;
let policy = Arc::new(FallthroughRetryPolicy);
let session = SessionBuilder::new()
.known_node("127.0.0.1:9042")
.retry_policy(Arc::new(FallthroughRetryPolicy))
.build()
.await?;
DowngradingConsistencyRetryPolicy
Retries with lower consistency levels when possible.
pub struct DowngradingConsistencyRetryPolicy;
This policy can return partial results. Use with caution and only when appropriate for your application.
Creation
use scylla::policies::retry::DowngradingConsistencyRetryPolicy;
use std::sync::Arc;
let policy = Arc::new(DowngradingConsistencyRetryPolicy);
Behavior
- Read Timeout: Retry with lower consistency if some replicas responded
- Write Timeout: Retry with lower consistency for batch log writes
- Unavailable: Retry with lower consistency on next node
Example:
let session = SessionBuilder::new()
.known_node("127.0.0.1:9042")
.retry_policy(Arc::new(DowngradingConsistencyRetryPolicy))
.build()
.await?;
Examples
Using DefaultRetryPolicy
use scylla::{Session, SessionBuilder};
use scylla::policies::retry::DefaultRetryPolicy;
use std::sync::Arc;
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let policy = Arc::new(DefaultRetryPolicy::new());
let session = SessionBuilder::new()
.known_node("127.0.0.1:9042")
.retry_policy(policy)
.build()
.await?;
// Requests will be retried according to DefaultRetryPolicy
let result = session
.query_unpaged("SELECT * FROM my_table", &[])
.await?;
Ok(())
}
Per-Statement Retry Policy
use scylla::statement::Statement;
use scylla::policies::retry::FallthroughRetryPolicy;
use std::sync::Arc;
let mut stmt = Statement::new("SELECT * FROM my_table");
stmt.set_retry_policy(Some(Arc::new(FallthroughRetryPolicy)));
// This query will not retry on failure
let result = session.query_unpaged(stmt, &[]).await?;
Custom Retry Policy
use scylla::policies::retry::{
RetryPolicy, RetrySession, RetryDecision, RequestInfo
};
use scylla::frame::types::Consistency;
use scylla::errors::RequestAttemptError;
#[derive(Debug)]
struct LimitedRetryPolicy {
max_retries: usize,
}
impl LimitedRetryPolicy {
fn new(max_retries: usize) -> Self {
Self { max_retries }
}
}
impl RetryPolicy for LimitedRetryPolicy {
fn new_session(&self) -> Box<dyn RetrySession> {
Box::new(LimitedRetrySession {
max_retries: self.max_retries,
retry_count: 0,
})
}
}
struct LimitedRetrySession {
max_retries: usize,
retry_count: usize,
}
impl RetrySession for LimitedRetrySession {
fn decide_should_retry(&mut self, request_info: RequestInfo) -> RetryDecision {
if self.retry_count >= self.max_retries {
return RetryDecision::DontRetry;
}
self.retry_count += 1;
// Only retry on network errors for idempotent requests
match request_info.error {
RequestAttemptError::BrokenConnectionError(_)
| RequestAttemptError::UnableToAllocStreamId => {
if request_info.is_idempotent {
RetryDecision::RetryNextTarget(None)
} else {
RetryDecision::DontRetry
}
}
_ => RetryDecision::DontRetry,
}
}
fn reset(&mut self) {
self.retry_count = 0;
}
}
// Usage
let policy = Arc::new(LimitedRetryPolicy::new(3));
let session = SessionBuilder::new()
.known_node("127.0.0.1:9042")
.retry_policy(policy)
.build()
.await?;
Idempotence and Retries
use scylla::statement::Statement;
// Non-idempotent operation (won't retry on network errors)
let mut insert = Statement::new(
"INSERT INTO counters (id) VALUES (?)"
);
insert.set_is_idempotent(false);
// Idempotent operation (safe to retry)
let mut select = Statement::new(
"SELECT * FROM users WHERE id = ?"
);
select.set_is_idempotent(true);
// UPDATE with idempotent values
let mut update = Statement::new(
"UPDATE users SET name = ? WHERE id = ?"
);
update.set_is_idempotent(true); // Safe because we're setting to specific value
// Counter UPDATE is NOT idempotent
let mut counter = Statement::new(
"UPDATE counters SET count = count + 1 WHERE id = ?"
);
counter.set_is_idempotent(false); // Not safe to retry
Best Practices
- Use DefaultRetryPolicy for most applications
- Mark statements as idempotent when safe to retry
- Be cautious with DowngradingConsistencyRetryPolicy - it can return partial results
- Set per-statement policies for operations with special requirements
- Test retry behavior under failure scenarios
Idempotence is critical: Only retry idempotent operations on network errors. Non-idempotent operations (like counter increments) should not be retried automatically.
Retry Decision Guide
| Error Type | DefaultRetryPolicy | Recommendation |
|---|
| Read timeout (enough replicas) | Retry same node | ✅ Safe |
| Read timeout (not enough) | Don’t retry | ✅ Correct |
| Write timeout (batch log) | Retry next node | ✅ Safe |
| Write timeout (unlogged) | Retry same node | ✅ Safe |
| Unavailable | Retry next node | ✅ Safe |
| Network error (idempotent) | Retry next node | ✅ Safe |
| Network error (non-idempotent) | Don’t retry | ✅ Correct |
See Also
