Load Balancing

Overview

Sol RPC Router uses a weighted random algorithm to distribute requests across healthy backends. The selection logic filters out unhealthy backends and respects optional method-specific routing overrides.

Selection Algorithm

Weighted Random Selection

Implementation in src/state.rs:45-105:

src/state.rs

pub fn select_backend(&self, rpc_method: Option<&str>) -> Option<(String, String)> {
    let state = self.state.load();

    // Check method-specific routing first
    if let Some(method) = rpc_method {
        if let Some(backend_label) = state.method_routes.get(method) {
            if let Some(backend) = state
                .backends
                .iter()
                .find(|b| b.config.label == *backend_label)
            {
                if backend.healthy.load(Ordering::Relaxed) {
                    debug!("Method {} routed to label={}", method, backend_label);
                    return Some((backend.config.label.clone(), backend.config.url.clone()));
                } else {
                    info!(
                        "Method {} target label={} is unhealthy, falling back to weighted selection",
                        method, backend_label
                    );
                }
            }
        }
    }

    // Filter out unhealthy backends (lock-free)
    let healthy_backends: Vec<&RuntimeBackend> = state
        .backends
        .iter()
        .filter(|b| b.healthy.load(Ordering::Relaxed))
        .collect();

    if healthy_backends.is_empty() {
        return None; // No healthy backends available
    }

    // Calculate total weight of healthy backends
    let healthy_total_weight: u32 = healthy_backends.iter().map(|b| b.config.weight).sum();

    if healthy_total_weight == 0 {
        return healthy_backends
            .first()
            .map(|b| (b.config.label.clone(), b.config.url.clone()));
    }

    // Weighted random selection among healthy backends
    let mut rng = rand::thread_rng();
    let mut random_weight = rng.gen_range(0..healthy_total_weight);

    for backend in &healthy_backends {
        if random_weight < backend.config.weight {
            return Some((backend.config.label.clone(), backend.config.url.clone()));
        }
        random_weight -= backend.config.weight;
    }

    // Fallback (should never reach here if weights are valid)
    healthy_backends
        .first()
        .map(|b| (b.config.label.clone(), b.config.url.clone()))
}

How It Works

Method Routing Check

If a method-specific route is configured and the backend is healthy, use it immediately:

config.toml

[method_routes]
getSlot = "mainnet-primary"

If the target backend is unhealthy, fall through to weighted selection.

Filter Healthy Backends

Remove unhealthy backends from consideration:

let healthy_backends: Vec<&RuntimeBackend> = state
    .backends
    .iter()
    .filter(|b| b.healthy.load(Ordering::Relaxed))
    .collect();

Health status is checked atomically via AtomicBool without locks.

Calculate Total Weight

Sum weights of all healthy backends:

let healthy_total_weight: u32 = healthy_backends
    .iter()
    .map(|b| b.config.weight)
    .sum();

If total weight is 0, return the first healthy backend.

Random Selection

Generate a random number in [0, total_weight) and walk through backends:

let mut rng = rand::thread_rng();
let mut random_weight = rng.gen_range(0..healthy_total_weight);

for backend in &healthy_backends {
    if random_weight < backend.config.weight {
        return Some((backend.config.label.clone(), backend.config.url.clone()));
    }
    random_weight -= backend.config.weight;
}

Each backend has a probability of weight / total_weight of being selected.

Weight Distribution

Example Configuration

config.toml

[[backends]]
label = "mainnet-primary"
url = "https://api.mainnet-beta.solana.com"
weight = 10

[[backends]]
label = "backup-rpc"
url = "https://solana-api.com"
weight = 5

[[backends]]
label = "local-node"
url = "http://127.0.0.1:8899"
weight = 2

Traffic Distribution

With all backends healthy, the expected traffic split is:

Backend	Weight	Probability	Approximate Traffic
mainnet-primary	10	10/17	~58.8%
backup-rpc	5	5/17	~29.4%
local-node	2	2/17	~11.8%

The weighted random algorithm provides probabilistic distribution. Over time, traffic converges to the expected ratios, but individual requests are randomly distributed.

Dynamic Rebalancing

When a backend becomes unhealthy, traffic automatically redistributes:

Total weight: 10 + 5 + 2 = 17

mainnet-primary: 10/17 = 58.8%
backup-rpc:       5/17 = 29.4%
local-node:       2/17 = 11.8%

Method-Specific Routing

Configuration

Pin specific RPC methods to designated backends:

config.toml

[method_routes]
getSlot = "mainnet-primary"
getBlockHeight = "mainnet-primary"
getTransaction = "archive-node"
getSignaturesForAddress = "archive-node"

Routing Behavior

Exact Match: If the RPC method matches a route and the target backend is healthy, route directly
Fallback: If the target backend is unhealthy, fall back to weighted selection
Unknown Methods: Methods not in the routing table use weighted selection

Example from src/state.rs:49-67:

src/state.rs

if let Some(method) = rpc_method {
    if let Some(backend_label) = state.method_routes.get(method) {
        // Find the backend by label to check its atomic health
        if let Some(backend) = state
            .backends
            .iter()
            .find(|b| b.config.label == *backend_label)
        {
            if backend.healthy.load(Ordering::Relaxed) {
                debug!("Method {} routed to label={}", method, backend_label);
                return Some((backend.config.label.clone(), backend.config.url.clone()));
            } else {
                info!(
                    "Method {} target label={} is unhealthy, falling back to weighted selection",
                    method, backend_label
                );
            }
        }
    }
}

Method routing only applies to healthy backends. If the target backend is unhealthy, requests fall back to weighted selection across all healthy backends.

Use Cases

Archive Node Routing

Route historical queries to specialized archive nodes:

[[backends]]
label = "archive-node"
url = "https://archive.solana.com"
weight = 1

[method_routes]
getTransaction = "archive-node"
getConfirmedBlock = "archive-node"

High-Performance Endpoints

Pin latency-sensitive methods to low-latency backends:

[[backends]]
label = "local-validator"
url = "http://127.0.0.1:8899"
weight = 1

[method_routes]
sendTransaction = "local-validator"
simulateTransaction = "local-validator"

Cost Optimization

Route expensive queries to cheaper providers:

[[backends]]
label = "budget-rpc"
url = "https://cheap-rpc.example.com"
weight = 1

[method_routes]
getSignaturesForAddress = "budget-rpc"
getProgramAccounts = "budget-rpc"

WebSocket Load Balancing

Separate Selection Logic

WebSocket connections use a dedicated selection function (src/state.rs:108-146):

src/state.rs

pub fn select_ws_backend(&self) -> Option<(String, String)> {
    let state = self.state.load();

    // Filter to backends with ws_url configured and healthy (lock-free)
    let ws_backends: Vec<&RuntimeBackend> = state
        .backends
        .iter()
        .filter(|b| b.config.ws_url.is_some() && b.healthy.load(Ordering::Relaxed))
        .collect();

    if ws_backends.is_empty() {
        return None;
    }

    // Calculate total weight of WebSocket-capable backends
    let total_weight: u32 = ws_backends.iter().map(|b| b.config.weight).sum();

    // Weighted random selection
    let mut rng = rand::thread_rng();
    let mut random_weight = rng.gen_range(0..total_weight);

    for backend in &ws_backends {
        if random_weight < backend.config.weight {
            return Some((
                backend.config.label.clone(),
                backend.config.ws_url.as_ref().unwrap().clone(),
            ));
        }
        random_weight -= backend.config.weight;
    }

    // Fallback
    ws_backends.first().map(|b| (
        b.config.label.clone(),
        b.config.ws_url.as_ref().unwrap().clone(),
    ))
}

Key Differences

ws_url Required: Only backends with ws_url configured are considered
No Method Routing: WebSocket connections don’t use method-specific routing
Same Weights: Uses the same weight values as HTTP routing

Configuration

config.toml

[[backends]]
label = "mainnet-primary"
url = "https://api.mainnet-beta.solana.com"
ws_url = "wss://api.mainnet-beta.solana.com"  # WebSocket enabled
weight = 10

[[backends]]
label = "http-only-backend"
url = "https://http-only.example.com"
# No ws_url - excluded from WebSocket routing
weight = 5

Backends without ws_url continue to serve HTTP requests but are automatically excluded from WebSocket routing.

Health-Aware Routing

Lock-Free Health Checks

Backend health status is stored in AtomicBool for lock-free access:

src/state.rs

#[derive(Debug, Clone)]
pub struct RuntimeBackend {
    pub config: Backend,
    pub healthy: Arc<AtomicBool>,
}

Selection reads health status without blocking:

src/state.rs

let healthy_backends: Vec<&RuntimeBackend> = state
    .backends
    .iter()
    .filter(|b| b.healthy.load(Ordering::Relaxed))
    .collect();

Automatic Exclusion

Unhealthy backends are automatically excluded from selection:

Background health checker marks backends unhealthy
Next request filters out unhealthy backends
Traffic redistributes to remaining healthy backends
When backend recovers, traffic gradually returns

Health transitions are atomic and immediate. There’s no delay between a backend being marked unhealthy and being excluded from routing.

No Healthy Backends

If all backends are unhealthy, requests fail with 503 Service Unavailable:

src/handlers.rs

let (backend_label, backend_url) = match state.select_backend(rpc_method) {
    Some(selection) => selection,
    None => {
        tracing::error!("No healthy backends available for request");
        return (
            StatusCode::SERVICE_UNAVAILABLE,
            "No healthy backends available",
        )
            .into_response();
    }
};

See Health Checks for details on health monitoring.

Configuration Reference

Backend Definition

config.toml

[[backends]]
label = "backend-name"        # Unique identifier
url = "https://rpc.example.com" # HTTP endpoint
ws_url = "wss://rpc.example.com" # WebSocket endpoint (optional)
weight = 10                     # Relative traffic weight (must be > 0)

Method Routing

config.toml

[method_routes]
method_name = "backend-label"  # Must match an existing backend label

Validation Rules

Configuration is validated at startup (src/config.rs):

Backend Validation
Method Route Validation

At least one backend required
Labels must be unique and non-empty
Weights must be > 0
URLs must be valid HTTP(S) endpoints

Invalid configuration causes the router to exit with an error at startup. Always test configuration changes with --config flag before deploying.

Monitoring Load Distribution

Metrics Labels

All request metrics include the backend label:

# Total requests per backend
sum by (backend) (rpc_requests_total)

# Request rate per backend
rate(rpc_requests_total{backend="mainnet-primary"}[5m])

# Traffic distribution (percentage)
sum by (backend) (rate(rpc_requests_total[5m])) 
  / ignoring(backend) group_left
sum(rate(rpc_requests_total[5m]))

Grafana Dashboard Queries

sum by (backend) (rate(rpc_requests_total[5m]))

See Metrics for the complete list of available metrics.

Hot Reload

Backend configuration can be reloaded without downtime:

# Update config.toml
vim config.toml

# Send SIGHUP to reload
kill -HUP $(pidof sol-rpc-router)

The reload process (src/main.rs:133-191):

Reloads config.toml
Validates new configuration
Preserves health status for matching backend labels
Atomically swaps routing state
New requests use updated configuration

In-flight requests complete with the old configuration. Only new requests use the reloaded backends and weights.

Performance Considerations

Selection Overhead

Backend selection runs on every request:

Lock-free atomic health checks
O(n) iteration over backends
RNG generation per request

With typical backend counts (2-10), overhead is negligible (<1μs).

Weight Precision

Weights are u32 values supporting:

Fine-grained ratios (e.g., 1000:1)
Large weight values without overflow
Total weight limit: 4,294,967,295

Random Distribution

The algorithm uses rand::thread_rng() for randomness:

Thread-local RNG (no contention)
Cryptographically insecure (fast)
Uniform distribution over time

For more deterministic distribution, consider round-robin alternatives.

Get Started

Configuration

Features

Operations

Overview

Selection Algorithm

Weighted Random Selection

How It Works

Weight Distribution

Example Configuration

Traffic Distribution

Dynamic Rebalancing

Method-Specific Routing

Configuration

Routing Behavior

Use Cases

WebSocket Load Balancing

Separate Selection Logic

Key Differences

Configuration

Health-Aware Routing

Lock-Free Health Checks

Automatic Exclusion

No Healthy Backends

Configuration Reference

Backend Definition

Method Routing

Validation Rules

Monitoring Load Distribution

Metrics Labels

Grafana Dashboard Queries

Hot Reload

Performance Considerations

Build docs developers (and LLMs) love

Get Started

Configuration

Features

Operations

​Overview

​Selection Algorithm

​Weighted Random Selection

​How It Works

​Weight Distribution

​Example Configuration

​Traffic Distribution

​Dynamic Rebalancing

​Method-Specific Routing

​Configuration

​Routing Behavior

​Use Cases

​WebSocket Load Balancing

​Separate Selection Logic

​Key Differences

​Configuration

​Health-Aware Routing

​Lock-Free Health Checks

​Automatic Exclusion

​No Healthy Backends

​Configuration Reference

​Backend Definition

​Method Routing

​Validation Rules

​Monitoring Load Distribution

​Metrics Labels

​Grafana Dashboard Queries

​Hot Reload

​Performance Considerations

Build docs developers (and LLMs) love

Overview

Selection Algorithm

Weighted Random Selection

How It Works

Weight Distribution

Example Configuration

Traffic Distribution

Dynamic Rebalancing

Method-Specific Routing

Configuration

Routing Behavior

Use Cases

WebSocket Load Balancing

Separate Selection Logic

Key Differences

Configuration

Health-Aware Routing

Lock-Free Health Checks

Automatic Exclusion

No Healthy Backends

Configuration Reference

Backend Definition

Method Routing

Validation Rules

Monitoring Load Distribution

Metrics Labels

Grafana Dashboard Queries

Hot Reload

Performance Considerations