Load Balancing and Service Selection

Kratos provides a powerful selector system for load balancing and service instance selection. It works with service discovery to intelligently route traffic based on various strategies.

Core Interfaces

Selector

selector/selector.go:13-19

type Selector interface {
    Rebalancer
    
    // Select nodes
    // if err == nil, selected and done must not be empty
    Select(ctx context.Context, opts ...SelectOption) (selected Node, done DoneFunc, err error)
}

Rebalancer

selector/selector.go:22-25

type Rebalancer interface {
    // Apply is apply all nodes when any changes happen
    Apply(nodes []Node)
}

Node

selector/selector.go:33-53

type Node interface {
    // Scheme is service node scheme (http/grpc)
    Scheme() string
    
    // Address is the unique address under the same service
    Address() string
    
    // ServiceName is service name
    ServiceName() string
    
    // InitialWeight is the initial value of scheduling weight
    // if not set return nil
    InitialWeight() *int64
    
    // Version is service node version
    Version() string
    
    // Metadata is the kv pair metadata associated with the service instance
    Metadata() map[string]string
}

Built-in Selectors

Weighted Round Robin (WRR)

Default load balancing strategy:

import (
    "github.com/go-kratos/kratos/v2/selector/wrr"
    "github.com/go-kratos/kratos/v2/selector"
)

// Set as global selector
selector.SetGlobalSelector(wrr.NewBuilder())

// Or use directly
sel := wrr.NewBuilder().Build()

Features:

Weighted distribution based on node weights
Smooth weighted round-robin algorithm
No starvation of low-weight nodes

Random

Random selection:

import (
    "github.com/go-kratos/kratos/v2/selector/random"
    "github.com/go-kratos/kratos/v2/selector"
)

selector.SetGlobalSelector(random.NewBuilder())

Features:

Simple random selection
Weighted random based on node weights
Good for simple use cases

P2C (Power of Two Choices)

Latency-aware load balancing:

import (
    "github.com/go-kratos/kratos/v2/selector/p2c"
    "github.com/go-kratos/kratos/v2/selector"
)

selector.SetGlobalSelector(p2c.NewBuilder())

Features:

Selects two random nodes and picks the better one
Considers latency and success rate
Better load distribution under varying latencies
Uses EWMA (Exponentially Weighted Moving Average)

Using Selectors

Global Selector

Set once at application startup:

import (
    "github.com/go-kratos/kratos/v2/selector"
    "github.com/go-kratos/kratos/v2/selector/wrr"
)

func init() {
    // Set global selector (used by all clients)
    selector.SetGlobalSelector(wrr.NewBuilder())
}

Client Integration

Selectors are automatically used with service discovery:

import (
    "github.com/go-kratos/kratos/v2/transport/http"
    "github.com/go-kratos/kratos/contrib/registry/consul/v2"
)

// Selector automatically applied
client, err := http.NewClient(
    ctx,
    http.WithEndpoint("discovery:///user-service"),
    http.WithDiscovery(dis),
    // Uses global selector for load balancing
)

Node Filtering

Filter by Version

import "github.com/go-kratos/kratos/v2/selector/filter"

client, err := http.NewClient(
    ctx,
    http.WithEndpoint("discovery:///api-service"),
    http.WithDiscovery(dis),
    http.WithNodeFilter(
        filter.Version("v2.0.0"),
    ),
)

Custom Node Filter

selector/filter.go

import "github.com/go-kratos/kratos/v2/selector"

// Filter by region
func RegionFilter(region string) selector.NodeFilter {
    return func(ctx context.Context, nodes []selector.Node) []selector.Node {
        filtered := make([]selector.Node, 0, len(nodes))
        for _, node := range nodes {
            if node.Metadata()["region"] == region {
                filtered = append(filtered, node)
            }
        }
        return filtered
    }
}

// Usage
client, err := http.NewClient(
    ctx,
    http.WithEndpoint("discovery:///api-service"),
    http.WithDiscovery(dis),
    http.WithNodeFilter(
        RegionFilter("us-west"),
    ),
)

Multiple Filters

Chain multiple filters:

import "github.com/go-kratos/kratos/v2/selector/filter"

client, err := http.NewClient(
    ctx,
    http.WithEndpoint("discovery:///api-service"),
    http.WithDiscovery(dis),
    http.WithNodeFilter(
        filter.Version("v2.0.0"),
        RegionFilter("us-west"),
        EnvironmentFilter("production"),
    ),
)

Weighted Nodes

Setting Weights

Set weights via service metadata:

import "github.com/go-kratos/kratos/v2"

app := kratos.New(
    kratos.Name("api-service"),
    kratos.Metadata(map[string]string{
        "weight": "100",  // Higher weight = more traffic
    }),
    kratos.Server(httpSrv),
    kratos.Registrar(r),
)

Weight Distribution

With WRR selector:

Node A: weight 100 → receives ~66% of traffic
Node B: weight 50 → receives ~33% of traffic

Peer Context

Access selected node information:

import "github.com/go-kratos/kratos/v2/selector"

func MyMiddleware() middleware.Middleware {
    return func(handler middleware.Handler) middleware.Handler {
        return func(ctx context.Context, req interface{}) (interface{}, error) {
            // Get selected peer
            if p, ok := selector.FromPeerContext(ctx); ok {
                log.Infof("Selected node: %s", p.Node.Address())
            }
            return handler(ctx, req)
        }
    }
}

Subset Selection

Limit the number of nodes to reduce connection overhead:

client, err := http.NewClient(
    ctx,
    http.WithEndpoint("discovery:///api-service"),
    http.WithDiscovery(dis),
    http.WithSubset(10),  // Max 10 nodes
)

Benefits:

Reduces connection overhead for large clusters
Maintains good load distribution
Default: 25 nodes

Done Callback

Provide feedback for adaptive load balancing:

selector/selector.go:56-66

type DoneInfo struct {
    // Response Error
    Err error
    
    // Response Metadata
    ReplyMD ReplyMD
    
    // BytesSent indicates if any bytes have been sent to the server
    BytesSent bool
    
    // BytesReceived indicates if any byte has been received from the server
    BytesReceived bool
}

type DoneFunc func(ctx context.Context, di DoneInfo)

The DoneFunc is automatically called after each request to provide feedback for future selections.

Balancer Interface

Low-level balancer interface:

selector/balancer.go:9-11

type Balancer interface {
    Pick(ctx context.Context, nodes []WeightedNode) (selected WeightedNode, done DoneFunc, err error)
}

Weighted Node

selector/balancer.go:19-33

type WeightedNode interface {
    Node
    
    // Raw returns the original node
    Raw() Node
    
    // Weight is the runtime calculated weight
    Weight() float64
    
    // Pick the node
    Pick() DoneFunc
    
    // PickElapsed is time elapsed since the latest pick
    PickElapsed() time.Duration
}

Custom Selector

Implement a custom selector:

import "github.com/go-kratos/kratos/v2/selector"

type MySelector struct {
    nodes []selector.Node
}

func (s *MySelector) Select(ctx context.Context, opts ...selector.SelectOption) (selector.Node, selector.DoneFunc, error) {
    if len(s.nodes) == 0 {
        return nil, nil, selector.ErrNoAvailable
    }
    
    // Custom selection logic
    node := s.nodes[0]
    
    done := func(ctx context.Context, di selector.DoneInfo) {
        // Handle completion feedback
    }
    
    return node, done, nil
}

func (s *MySelector) Apply(nodes []selector.Node) {
    s.nodes = nodes
}

type MyBuilder struct{}

func (b *MyBuilder) Build() selector.Selector {
    return &MySelector{}
}

// Usage
selector.SetGlobalSelector(&MyBuilder{})

EWMA Node

Exponentially Weighted Moving Average for latency tracking:

import "github.com/go-kratos/kratos/v2/selector/node/ewma"

// Used internally by P2C selector
node := ewma.New(originalNode)

// Provides dynamic weight based on latency
weight := node.Weight()

Best Practices

Choose Right Selector

Use WRR for general purpose load balancing
Use P2C for latency-sensitive applications
Use Random for simple scenarios

Set Appropriate Weights

Weight nodes based on capacity (CPU, memory, network) not just equally.

Use Node Filters

Filter by version, region, or environment to control traffic routing.

Enable Subset Selection

Use subset for large clusters (>25 nodes) to reduce connection overhead.

Monitor Selection

Track which nodes are selected to ensure even distribution.

Handle No Available Nodes

Always handle ErrNoAvailable when all nodes are filtered out.

Selection Strategies Comparison

Strategy	Best For	Complexity	Latency Aware
WRR	General purpose, weighted distribution	Low	No
Random	Simple scenarios, quick decisions	Very Low	No
P2C	Latency-sensitive, uneven load	Medium	Yes

Complete Example

package main

import (
    "context"
    "github.com/go-kratos/kratos/v2/selector"
    "github.com/go-kratos/kratos/v2/selector/p2c"
    "github.com/go-kratos/kratos/v2/selector/filter"
    "github.com/go-kratos/kratos/v2/transport/http"
    "github.com/go-kratos/kratos/contrib/registry/consul/v2"
)

func main() {
    // Set global selector
    selector.SetGlobalSelector(p2c.NewBuilder())
    
    // Create discovery
    consulClient, _ := api.NewClient(api.DefaultConfig())
    dis := consul.New(consulClient)
    
    // Custom region filter
    regionFilter := func(ctx context.Context, nodes []selector.Node) []selector.Node {
        region := "us-west"
        filtered := make([]selector.Node, 0)
        for _, node := range nodes {
            if node.Metadata()["region"] == region {
                filtered = append(filtered, node)
            }
        }
        return filtered
    }
    
    // Create client with filters
    client, err := http.NewClient(
        context.Background(),
        http.WithEndpoint("discovery:///api-service"),
        http.WithDiscovery(dis),
        http.WithNodeFilter(
            filter.Version("v2.0.0"),
            regionFilter,
        ),
        http.WithSubset(10),
    )
    if err != nil {
        log.Fatal(err)
    }
    defer client.Close()
    
    // Make requests - automatically load balanced
    for i := 0; i < 100; i++ {
        var reply Response
        err := client.Invoke(
            context.Background(),
            http.MethodGet,
            "/api/resource",
            nil,
            &reply,
        )
        if err != nil {
            log.Error(err)
        }
    }
}

Registry

Service discovery

HTTP Client

HTTP client integration

gRPC Client

gRPC client integration

Metadata

Node metadata

Get Started

Core Concepts

Components

Middleware

Guides

CLI Reference

Load Balancing and Service Selection

Core Interfaces

Selector

Rebalancer

Node

Built-in Selectors

Weighted Round Robin (WRR)

Random

P2C (Power of Two Choices)

Using Selectors

Global Selector

Client Integration

Node Filtering

Filter by Version

Custom Node Filter

Multiple Filters

Weighted Nodes

Setting Weights

Weight Distribution

Peer Context

Subset Selection

Done Callback

Balancer Interface

Weighted Node

Custom Selector

EWMA Node

Best Practices

Selection Strategies Comparison

Complete Example

Registry

HTTP Client

gRPC Client

Metadata

Build docs developers (and LLMs) love

Get Started

Core Concepts

Components

Middleware

Guides

CLI Reference

​Core Interfaces

​Selector

​Rebalancer

​Node

​Built-in Selectors

​Weighted Round Robin (WRR)

​Random

​P2C (Power of Two Choices)

​Using Selectors

​Global Selector

​Client Integration

​Node Filtering

​Filter by Version

​Custom Node Filter

​Multiple Filters

​Weighted Nodes

​Setting Weights

​Weight Distribution

​Peer Context

​Subset Selection

​Done Callback

​Balancer Interface

​Weighted Node

​Custom Selector

​EWMA Node

​Best Practices

​Selection Strategies Comparison

​Complete Example

​Related Components

Registry

HTTP Client

gRPC Client

Metadata

Build docs developers (and LLMs) love

Core Interfaces

Selector

Rebalancer

Node

Built-in Selectors

Weighted Round Robin (WRR)

Random

P2C (Power of Two Choices)

Using Selectors

Global Selector

Client Integration

Node Filtering

Filter by Version

Custom Node Filter

Multiple Filters

Weighted Nodes

Setting Weights

Weight Distribution

Peer Context

Subset Selection

Done Callback

Balancer Interface

Weighted Node

Custom Selector

EWMA Node

Best Practices

Selection Strategies Comparison

Complete Example

Related Components