Envoy Proxy Topology

KubeLB uses Envoy proxy as its data plane for load balancing. The deployment topology determines how Envoy proxy instances are deployed and shared across tenants and services.

Topology Overview

KubeLB supports three deployment topologies for Envoy proxy:

Shared

One Envoy proxy per tenant cluster (Default)

Global

One Envoy proxy for all tenant clusters (Deprecated)

Dedicated

One Envoy proxy per service (Deprecated)

Current Status: The shared topology is the only supported topology. The dedicated and global topologies are deprecated and will be removed in a future release. Both deprecated topologies now default to shared topology.

Shared Topology (Default)

In shared topology, a single Envoy proxy deployment is created for each tenant cluster. All services and routes from that tenant are configured to use this shared Envoy proxy.

Architecture

Benefits

Resource Efficiency

Significantly reduces resource consumption compared to dedicated topology. A single Envoy instance handles all traffic for a tenant.

Tenant Isolation

Each tenant has its own Envoy proxy, providing fault isolation. Issues in one tenant’s traffic don’t affect others.

Simplified Management

Fewer Envoy instances to monitor and maintain compared to dedicated topology.

Scalability

Easy to scale per tenant by adjusting replica count or using DaemonSet mode.

Configuration

apiVersion: kubelb.k8c.io/v1alpha1
kind: Config
metadata:
  name: kubelb
  namespace: kubelb
spec:
  envoyProxy:
    # Shared topology (default)
    topology: shared
    
    # Number of Envoy replicas per tenant
    replicas: 3
    
    # Use DaemonSet instead of Deployment
    useDaemonset: false
    
    # Ensure pods are spread across nodes
    singlePodPerNode: true
    
    # Resource requirements
    resources:
      requests:
        cpu: "500m"
        memory: "512Mi"
      limits:
        cpu: "2000m"
        memory: "2Gi"

Deployment Modes

Deployment (Default)
DaemonSet

Envoy runs as a Kubernetes Deployment with a configurable number of replicas.

envoyProxy:
  topology: shared
  replicas: 3
  useDaemonset: false

Use Cases:

Clusters with specific capacity requirements
Fine-grained control over replica count
Clusters where not all nodes need Envoy

Envoy runs as a DaemonSet with one pod per node.

envoyProxy:
  topology: shared
  useDaemonset: true
  # replicas field is ignored

Use Cases:

High availability requirements
Maximum throughput across all nodes
Local traffic routing preferences

Pod Distribution

Control how Envoy pods are distributed across nodes:

envoyProxy:
  # Ensure only one pod per node (for Deployment mode)
  singlePodPerNode: true
  
  # Node selector for Envoy pods
  nodeSelector:
    node-role.kubernetes.io/worker: ""
  
  # Tolerations for tainted nodes
  tolerations:
    - key: "dedicated"
      operator: "Equal"
      value: "kubelb"
      effect: "NoSchedule"
  
  # Custom affinity rules
  affinity:
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        - labelSelector:
            matchExpressions:
              - key: app
                operator: In
                values:
                  - envoy
          topologyKey: kubernetes.io/hostname

When singlePodPerNode: true, KubeLB adds pod anti-affinity rules to prevent multiple Envoy pods on the same node.

Global Topology (Deprecated)

Global topology is deprecated and will be removed in a future release. It now defaults to shared topology.

In global topology, a single Envoy proxy deployment is shared across all tenant clusters.

Architecture

Why It’s Deprecated

No Fault Isolation: Issues in one tenant affect all tenants
Scaling Challenges: Hard to scale for specific tenant needs
Resource Contention: All tenants compete for the same Envoy resources
Configuration Complexity: Large xDS snapshots with all tenant configurations

Dedicated Topology (Deprecated)

Dedicated topology was deprecated in v1.1.0 and now defaults to shared topology.

In dedicated topology, a separate Envoy proxy deployment was created for each LoadBalancer service.

Architecture

Why It Was Deprecated

Resource Intensive: Hundreds of Envoy instances for many services
Operational Overhead: Too many deployments to monitor and maintain
Cost: High resource consumption in the management cluster
Complexity: Difficult to manage at scale

Envoy xDS Configuration

Regardless of topology, KubeLB uses the Envoy xDS (Discovery Service) protocol to dynamically configure Envoy proxies.

xDS Server

The KubeLB Manager hosts an xDS control plane server:

// From internal/envoy/server.go
type Server struct {
    config        *v1alpha1.Config
    Cache         cachev3.SnapshotCache
    listenAddress string
    enableAdmin   bool
}

The xDS server:

Listens on port 18000 (default)
Implements Envoy’s gRPC-based xDS APIs
Maintains a snapshot cache of configurations
Pushes updates to connected Envoy proxies

Configuration Resources

KubeLB configures three main xDS resource types:

Listeners
Clusters
Endpoints

Define ports that Envoy listens on:

TCP Listener: For Layer 4 TCP services
UDP Listener: For Layer 4 UDP services
HTTP Listener: For Layer 7 HTTP/HTTPS traffic

Each LoadBalancer service port gets a dedicated listener.

Bootstrap Configuration

Each Envoy proxy starts with a bootstrap configuration that:

// From internal/envoy/bootstrap.go
func (s *Server) GenerateBootstrap() string {
    cfg := &envoyBootstrap.Bootstrap{
        DynamicResources: &envoyBootstrap.Bootstrap_DynamicResources{
            LdsConfig: /* Listener Discovery Service */,
            CdsConfig: /* Cluster Discovery Service */,
        },
        StaticResources: &envoyBootstrap.Bootstrap_StaticResources{
            Listeners: [
                /* Readiness probe listener */,
                /* Health check listener */,
                /* Stats/metrics listener */,
            ],
            Clusters: [
                /* xDS cluster for control plane */,
                /* Admin cluster for local admin API */,
            ],
        },
        Admin: /* Admin interface configuration */,
    }
}

Static resources:

xDS Cluster: Connects to KubeLB Manager’s xDS server
Admin Listener: Local admin interface (127.0.0.1:9001)
Stats Listener: Prometheus metrics endpoint (port 19001)
Readiness Probe: Health endpoint (port 19003)

Dynamic Updates

When services change:

CCM detects change in tenant cluster
CCM updates LoadBalancer/Route CRD in management cluster
KubeLB Manager controller reconciles the change
Manager generates new xDS snapshot
xDS server pushes update to connected Envoy proxies
Envoy applies the configuration without restart

High Availability

For production deployments, ensure high availability:

Multiple Replicas

envoyProxy:
  topology: shared
  replicas: 3  # Minimum recommended
  singlePodPerNode: true

Pod Disruption Budget

KubeLB automatically creates a PodDisruptionBudget for Envoy deployments:

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: envoy-tenant-a
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app: envoy
      tenant: tenant-a

Graceful Shutdown

Configure graceful shutdown to drain connections before termination:

envoyProxy:
  gracefulShutdown:
    # Enable graceful shutdown
    disabled: false
    
    # Maximum time to drain connections
    drainTimeout: "60s"
    
    # Minimum time before checking connection count
    minDrainDuration: "5s"
    
    # Total grace period for pod termination
    terminationGracePeriodSeconds: 300
    
    # Shutdown manager sidecar image
    shutdownManagerImage: "docker.io/envoyproxy/gateway:v1.3.0"

With graceful shutdown enabled:

Pod receives TERM signal
Shutdown manager starts draining Envoy
Envoy stops accepting new connections
Existing connections are allowed to complete
After drainTimeout or when connections reach zero, Envoy exits

Overload Manager

Protect Envoy from resource exhaustion:

envoyProxy:
  overloadManager:
    # Enable overload protection
    enabled: true
    
    # Maximum heap size (bytes)
    maxHeapSizeBytes: 2147483648  # 2GB
    
    # Maximum concurrent connections
    maxActiveDownstreamConnections: 50000

Overload actions:

95% heap: Start shrinking heap by freeing memory
98% heap: Stop accepting new requests
Max connections: Reject new connections

Monitoring and Observability

Metrics

Envoy exposes Prometheus metrics on port 19001:

curl http://envoy-pod:19001/stats/prometheus

Key metrics to monitor:

envoy_cluster_upstream_cx_active: Active connections to upstreams
envoy_cluster_membership_healthy: Healthy endpoints per cluster
envoy_listener_downstream_cx_total: Total downstream connections
envoy_server_memory_allocated: Memory usage

Health Checks

Envoy provides multiple health endpoints:

Readiness: http://envoy:19003/ready - Ready to accept traffic
Liveness: http://envoy:19004/healthz - Envoy process is alive
Admin: http://127.0.0.1:9001/ - Admin interface (localhost only)

Access Logs

Envoy logs all requests to stdout:

[2024-01-15T10:30:45.123Z] "GET / HTTP/1.1" 200 - 0 1234 5 3 "192.168.1.100" "curl/7.68.0" "req-id-123" "example.com" "10.0.1.5:30080"

Format includes:

Timestamp
Request method and path
Response code
Bytes sent/received
Duration
Client IP
Upstream host

Troubleshooting

Envoy Pod Not Starting

Check:

Bootstrap configuration is valid
xDS server is reachable from the pod
Resource requests/limits are not too restrictive
Node selectors and tolerations are correctly configured

kubectl logs -n kubelb envoy-tenant-a-xxx -c envoy

No xDS Configuration Updates

Verify:

Envoy is connected to xDS server (check logs)
LoadBalancer/Route CRDs exist in management cluster
KubeLB Manager controller is running
No errors in manager logs

# Check xDS connections
kubectl logs -n kubelb kubelb-manager-xxx | grep "gRPC connection"

High Memory Usage

Configure overload manager:

envoyProxy:
  overloadManager:
    enabled: true
    maxHeapSizeBytes: 2147483648

Or increase resource limits:

envoyProxy:
  resources:
    limits:
      memory: "4Gi"

Best Practices

Use Shared Topology

Always use shared topology for the best balance of resource efficiency and isolation.

Deploy Multiple Replicas

Run at least 3 Envoy replicas per tenant for high availability.

Enable Graceful Shutdown

Configure graceful shutdown to prevent connection drops during rolling updates.

Monitor Resource Usage

Set up monitoring and alerting for Envoy memory and CPU usage.

Use DaemonSet for Large Clusters

Consider DaemonSet mode for tenant clusters with many nodes.

Next Steps

Configuration Reference

Complete Config CRD reference

Load Balancing

Understand Layer 4 and Layer 7 load balancing

Monitoring

Set up monitoring and metrics

Performance Tuning

Optimize KubeLB performance

Overview

Installation

Core Concepts

Guides

Operations

Security

​Topology Overview

Shared

Global

Dedicated

​Shared Topology (Default)

​Architecture

​Benefits

​Configuration

​Deployment Modes

​Pod Distribution

​Global Topology (Deprecated)

​Architecture

​Why It’s Deprecated

​Dedicated Topology (Deprecated)

​Architecture

​Why It Was Deprecated

​Envoy xDS Configuration

​xDS Server

​Configuration Resources

​Bootstrap Configuration

​Dynamic Updates

​High Availability

​Multiple Replicas

​Pod Disruption Budget

​Graceful Shutdown

​Overload Manager

​Monitoring and Observability

​Metrics

​Health Checks

​Access Logs

​Troubleshooting

​Best Practices

​Next Steps

Configuration Reference

Load Balancing

Monitoring

Performance Tuning

Build docs developers (and LLMs) love

Topology Overview

Shared Topology (Default)

Architecture

Benefits

Configuration

Deployment Modes

Pod Distribution

Global Topology (Deprecated)

Architecture

Why It’s Deprecated

Dedicated Topology (Deprecated)

Architecture

Why It Was Deprecated

Envoy xDS Configuration

xDS Server

Configuration Resources

Bootstrap Configuration

Dynamic Updates

High Availability

Multiple Replicas

Pod Disruption Budget

Graceful Shutdown

Overload Manager

Monitoring and Observability

Metrics

Health Checks

Access Logs

Troubleshooting

Best Practices

Next Steps