High-Level Architecture
KubeLB uses a hub-and-spoke model with three main components:Component Roles
-
KubeLB Manager (
cmd/kubelb/)- Runs in the management cluster
- Hosts the Envoy xDS control plane
- Receives LoadBalancer and Route configurations from tenant CCMs
- Manages Envoy proxy deployments for traffic routing
-
KubeLB CCM (
cmd/ccm/)- Runs in tenant clusters
- Watches Services, Ingresses, Gateway API resources
- Propagates configurations as LoadBalancer and Route CRDs to the management cluster
- Reports node status and endpoints
Directory Structure
API Resources
KubeLB defines several Custom Resource Definitions (CRDs):Core Resources (api/ce/kubelb.k8c.io/v1alpha1/)
LoadBalancer
LoadBalancer
Represents a Layer 4 load balancer configuration for a Service.Controller:
internal/controllers/kubelb/loadbalancer_controller.goKey Fields:spec.endpoints: Backend endpoints (node IPs and ports)spec.ports: Port mappings for the servicespec.type: LoadBalancer type configuration
Route
Route
Represents Layer 7 routing configuration for Ingress or Gateway API resources.Controller:
internal/controllers/kubelb/route_controller.goKey Fields:spec.endpoints: Backend endpointsspec.routes: HTTP/GRPC route rulesspec.tls: TLS configuration
Tenant
Tenant
Cluster-scoped resource defining a tenant cluster and its configuration.Controller:
internal/controllers/kubelb/tenant_controller.goKey Fields:spec.ingress: Ingress controller settingsspec.loadBalancer: LoadBalancer settings
Config
Config
Cluster-scoped resource for global KubeLB settings.Key Fields:
spec.envoyProxy: Envoy proxy configurationspec.propagateAllAnnotations: Annotation propagation rules
SyncSecret
SyncSecret
Propagates secrets from tenant clusters to the management cluster.Controllers:
- CCM:
internal/controllers/ccm/sync_secret_controller.go - Manager:
internal/controllers/kubelb/sync_secret_controller.go
Traffic Flow
Layer 4 (LoadBalancer Service)
Layer 7 (Ingress/HTTPRoute)
Controller Patterns
KubeLB uses controller-runtime with standard Kubernetes controller patterns.Reconciler Pattern
All controllers implement theReconcile method:
Finalizers
Controllers use finalizers to ensure proper cleanup:Metrics
Controllers export Prometheus metrics for observability:- Manager:
kubelb_manager_* - CCM:
kubelb_ccm_*
Structured Logging
Use logr with verbosity levels:V(0): Default - important eventsV(1): Debug - detailed flowV(2): Trace - verbose debugging
Import Order
KubeLB enforces strict import ordering via.gimps.yaml:
- Standard library (
fmt,context, etc.) - External packages (
github.com/pkg/errors) - Kubermatic packages (
k8c.io/kubelb,github.com/kubermatic/**) - Envoy packages (
github.com/envoyproxy/**) - Kubernetes packages (
k8s.io/**,sigs.k8s.io/**)
Error Handling
Use github.com/pkg/errors for error wrapping:Copyright Headers
All new files must include the Apache 2.0 license header with the current year:Next Steps
Testing
Learn how to write and run tests
Code Generation
Understand CRD and RBAC generation
