Paging allows you to retrieve large result sets in smaller chunks (pages), reducing memory usage and improving performance.
Why Use Paging?
Without paging, all query results are returned in a single response, which can cause:
- High memory usage: Entire result set loaded into memory
- Increased latency: Waiting for all results before processing any
- Potential timeouts: Very large result sets may exceed timeout limits
With paging, you receive results incrementally, processing each page as it arrives.
Page Size
The page size determines how many rows are returned per page. The default is 5000 rows.
Setting Page Size on Unprepared Statements
use scylla::statement::unprepared::Statement;
let mut statement = Statement::new("SELECT * FROM ks.table");
statement.set_page_size(100);
// Or use builder pattern
let statement = Statement::new("SELECT * FROM ks.table")
.with_page_size(100);
Setting Page Size on Prepared Statements
use scylla::statement::unprepared::Statement;
// Set before preparing
let statement = Statement::new("SELECT * FROM ks.table")
.with_page_size(100);
let mut prepared = session.prepare(statement).await?;
// Or after preparing
prepared.set_page_size(100);
Choose page size based on your needs: smaller pages reduce memory usage but increase overhead; larger pages are more efficient but use more memory.
Automatic Paging
The easiest way to handle paging is with automatic iterators that fetch pages transparently.
query_iter - Unprepared Statements
use futures::stream::StreamExt;
use futures::stream::TryStreamExt;
let mut rows_stream = session
.query_iter("SELECT a, b, c FROM examples_ks.basic", &[])
.await?
.rows_stream::<(i32, i32, String)>()?;
while let Some(next_row_res) = rows_stream.next().await {
let (a, b, c) = next_row_res?;
println!("a, b, c: {}, {}, {}", a, b, c);
}
// Or use try_next for easier error handling
let mut rows_stream = session
.query_iter("SELECT a, b, c FROM examples_ks.basic", &[])
.await?
.rows_stream::<(i32, i32, String)>()?;
while let Some((a, b, c)) = rows_stream.try_next().await? {
println!("a, b, c: {}, {}, {}", a, b, c);
}
execute_iter - Prepared Statements
use futures::stream::StreamExt;
let prepared = session
.prepare("SELECT a, b FROM ks.t")
.await?;
let mut rows_stream = session
.execute_iter(prepared, &[])
.await?
.rows_stream::<(i32, i32)>()?;
while let Some(next_row_res) = rows_stream.next().await {
let (a, b): (i32, i32) = next_row_res?;
println!("a, b: {}, {}", a, b);
}
Working with Untyped Rows
use scylla::value::Row;
use futures::StreamExt;
let mut rows_stream = session
.query_iter("SELECT a, b, c FROM examples_ks.basic", &[])
.await?
.rows_stream::<Row>()?;
while let Some(row) = rows_stream.next().await.transpose()? {
let a = row.columns[0].as_ref().unwrap().as_int().unwrap();
let b = row.columns[1].as_ref().unwrap().as_int().unwrap();
let c = row.columns[2].as_ref().unwrap().as_text().unwrap();
println!("a, b, c: {}, {}, {}", a, b, c);
}
Custom Structs with DeserializeRow
use scylla::DeserializeRow;
use futures::StreamExt;
#[derive(Debug, DeserializeRow)]
struct RowData {
a: i32,
b: Option<i32>,
c: String,
}
let mut rows_stream = session
.query_iter("SELECT a, b, c FROM examples_ks.basic", &[])
.await?
.rows_stream::<RowData>()?;
while let Some(row_data) = rows_stream.try_next().await? {
println!("row_data: {:?}", row_data);
}
Manual Paging
For more control over paging, use manual methods that return one page at a time.
query_single_page - Unprepared Statements
use scylla::statement::unprepared::Statement;
use scylla::response::PagingState;
use std::ops::ControlFlow;
let paged_query = Statement::new("SELECT a, b, c FROM examples_ks.select_paging")
.with_page_size(6);
let mut paging_state = PagingState::start();
loop {
let (res, paging_state_response) = session
.query_single_page(paged_query.clone(), &[], paging_state)
.await?;
let res = res.into_rows_result()?;
println!("Fetched {} rows", res.rows_num());
// Process rows in this page
for row in res.rows::<(i32, i32, String)>()? {
let (a, b, c) = row?;
println!("a: {}, b: {}, c: {}", a, b, c);
}
match paging_state_response.into_paging_control_flow() {
ControlFlow::Break(()) => {
// No more pages to fetch
break;
}
ControlFlow::Continue(new_paging_state) => {
// Update paging state to continue
paging_state = new_paging_state;
}
}
}
execute_single_page - Prepared Statements
use scylla::response::PagingState;
use std::ops::ControlFlow;
let paged_prepared = session
.prepare(
Statement::new("SELECT a, b, c FROM examples_ks.select_paging")
.with_page_size(7)
)
.await?;
let mut paging_state = PagingState::start();
loop {
let (res, paging_state_response) = session
.execute_single_page(&paged_prepared, &[], paging_state)
.await?;
let res = res.into_rows_result()?;
println!("Fetched {} rows", res.rows_num());
// Process rows
for row in res.rows::<(i32, i32, String)>()? {
let (a, b, c) = row?;
println!("a: {}, b: {}, c: {}", a, b, c);
}
match paging_state_response.into_paging_control_flow() {
ControlFlow::Break(()) => break,
ControlFlow::Continue(new_paging_state) => {
paging_state = new_paging_state;
}
}
}
Paging State
The PagingState tracks position in the result set:
use scylla::response::{PagingState, PagingStateResponse};
use std::ops::ControlFlow;
// Start paging from the beginning
let mut paging_state = PagingState::start();
// Alternative: use default
let mut paging_state = PagingState::default();
// After each query, check the response
let (res, paging_state_response) = session
.query_single_page(query, &[], paging_state)
.await?;
// Convert response to control flow
match paging_state_response.into_paging_control_flow() {
ControlFlow::Break(()) => {
// No more pages - query is complete
}
ControlFlow::Continue(new_paging_state) => {
// More pages available - update state
paging_state = new_paging_state;
}
}
Paging state is opaque and version-specific. Don’t try to parse or manually construct paging states - always use values returned by the server.
Complete Example
use scylla::client::session::{Session, SessionBuilder};
use scylla::statement::unprepared::Statement;
use scylla::response::PagingState;
use futures::stream::StreamExt;
use futures::stream::TryStreamExt;
use std::ops::ControlFlow;
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let session: Session = SessionBuilder::new()
.known_node("127.0.0.1:9042")
.build()
.await?;
// Create schema and insert data
session
.query_unpaged(
"CREATE KEYSPACE IF NOT EXISTS examples_ks \
WITH REPLICATION = {'class': 'NetworkTopologyStrategy', 'replication_factor': 1}",
&[],
)
.await?;
session
.query_unpaged(
"CREATE TABLE IF NOT EXISTS examples_ks.select_paging \
(a int, b int, c text, primary key (a, b))",
&[],
)
.await?;
for i in 0..16_i32 {
session
.query_unpaged(
"INSERT INTO examples_ks.select_paging (a, b, c) VALUES (?, ?, 'abc')",
(i, 2 * i),
)
.await?;
}
// Example 1: Automatic paging with iterator
println!("\n=== Automatic Paging ===");
let mut rows_stream = session
.query_iter("SELECT a, b, c FROM examples_ks.select_paging", &[])
.await?
.rows_stream::<(i32, i32, String)>()?;
while let Some((a, b, c)) = rows_stream.try_next().await? {
println!("a: {}, b: {}, c: {}", a, b, c);
}
// Example 2: Manual paging
println!("\n=== Manual Paging ===");
let paged_query = Statement::new("SELECT a, b, c FROM examples_ks.select_paging")
.with_page_size(6);
let mut paging_state = PagingState::start();
let mut page_num = 1;
loop {
let (res, paging_state_response) = session
.query_single_page(paged_query.clone(), &[], paging_state)
.await?;
let res = res.into_rows_result()?;
println!("Page {}: {} rows", page_num, res.rows_num());
match paging_state_response.into_paging_control_flow() {
ControlFlow::Break(()) => break,
ControlFlow::Continue(new_paging_state) => {
paging_state = new_paging_state;
page_num += 1;
}
}
}
Ok(())
}
Choose Appropriate Page Size
Balance between memory usage (smaller pages) and efficiency (larger pages). For most use cases, 1000-5000 rows per page works well.
Use Prepared Statements
Always use execute_iter instead of query_iter when possible. Prepared statements are significantly faster.
Stream Processing
Process rows as they arrive instead of collecting all pages into memory. Use streaming APIs (rows_stream()) for best memory efficiency.
Consider Query Design
If you frequently need to page through large result sets, consider redesigning your data model to enable more targeted queries.
See Also