Skip to main content
The KubeLB CCM (Cloud Controller Manager) is deployed in tenant clusters and is responsible for:
  • Watching Kubernetes Services, Ingresses, and Gateway API resources
  • Propagating load balancer configurations to the management cluster
  • Managing node endpoints for load balancing
  • Synchronizing secrets and certificates when needed

Prerequisites

Before installing the CCM, ensure you have:
  • A Kubernetes cluster meeting the tenant cluster requirements
  • KubeLB Manager installed in the management cluster
  • kubectl configured to access the tenant cluster
  • Helm 3.x installed
  • A kubeconfig file with access to the management cluster
  • A unique tenant name (RFC 1123 lowercase format)

Installation Steps

1

Create the namespace

Create a dedicated namespace for KubeLB CCM:
kubectl create namespace kubelb
2

Create the management cluster secret

Create a Secret containing the kubeconfig for accessing the management cluster:
kubectl --namespace kubelb create secret generic kubelb-cluster \
  --from-file=kubelb=/path/to/management-cluster-kubeconfig.yaml
The kubeconfig file must be named kubelb within the Secret. The Secret name can be customized using kubelb.clusterSecretName in values.yaml.
3

Pull the Helm chart

Download the KubeLB CCM Helm chart from the OCI registry:
helm pull oci://quay.io/kubermatic/helm-charts/kubelb-ccm --version=v1.3.1 --untardir "." --untar
4

Apply CRDs

Install the Custom Resource Definitions (CRDs):
kubectl apply -f kubelb-ccm/crds/
5

Create values file

Create a values.yaml file with your configuration. Here’s a minimal example:
values.yaml
kubelb:
  # REQUIRED: Unique tenant identifier
  tenantName: tenant-cluster-1
  
  # Secret containing management cluster kubeconfig
  clusterSecretName: kubelb-cluster
  
  # Node address type for routing (ExternalIP or InternalIP)
  nodeAddressType: ExternalIP
  
  # Use LoadBalancer class filtering (false = watch all LoadBalancer services)
  useLoadBalancerClass: false

# Resource limits
resources:
  limits:
    cpu: 500m
    memory: 512Mi
  requests:
    cpu: 100m
    memory: 128Mi
The tenantName must be unique across all tenant clusters connected to the same management cluster.
6

Install the Helm chart

Install KubeLB CCM using Helm:
helm install kubelb-ccm kubelb-ccm/ \
  --namespace kubelb \
  -f values.yaml

Configuration Options

Basic Configuration

kubelb:
  tenantName: my-tenant
  clusterSecretName: kubelb-cluster
  nodeAddressType: ExternalIP

Gateway API Support

Enable Gateway API controllers for HTTPRoute, GRPCRoute, and Gateway resources:
kubelb:
  enableGatewayAPI: true
  useGatewayClass: true
  
  # Optionally install and manage Gateway API CRDs
  installGatewayAPICRDs: true
  gatewayAPICRDsChannel: standard  # or 'experimental'
If installGatewayAPICRDs is false, Gateway API CRDs must be installed manually before enabling Gateway API support.

Layer 7 Resource Controllers

Control which Layer 7 controllers are enabled:
kubelb:
  # Ingress controller
  disableIngressController: false
  useIngressClass: true
  
  # Gateway API controllers
  disableGatewayController: false
  disableHTTPRouteController: false
  disableGRPCRouteController: false

Ingress to Gateway Conversion

Automatically convert Ingress resources to Gateway API HTTPRoutes:
kubelb:
  ingressConversion:
    enabled: true
    gatewayName: kubelb
    gatewayNamespace: kubelb
    gatewayClass: kubelb
    
    # Optional: Filter by ingress class
    ingressClass: "nginx"
    
    # Optional: Domain replacement
    domainReplace: ".old.example.com"
    domainSuffix: ".new.example.com"
    
    # Annotations for the created Gateway
    gatewayAnnotations: "cert-manager.io/cluster-issuer=letsencrypt,external-dns.alpha.kubernetes.io/target=lb.example.com"
    
    # Feature flags
    propagateExternalDnsAnnotations: true
    copyTLSSecrets: true
    disableEnvoyGatewayFeatures: false
Set standaloneMode: true to run only the Ingress converter, disabling all other controllers.

Secret Synchronization

Enable automatic secret synchronization for TLS certificates:
kubelb:
  enableSecretSynchronizer: true
Secrets labeled with kubelb.k8c.io/managed-by: kubelb will be automatically synchronized to the management cluster.

Resource Filtering

Control which resources the CCM watches:
kubelb:
  useLoadBalancerClass: false
  useIngressClass: false
  useGatewayClass: false

Monitoring

Enable Prometheus metrics:
serviceMonitor:
  enabled: true

metrics:
  port: 9445
Enable Grafana dashboards:
grafana:
  dashboards:
    enabled: true
    annotations:
      grafana_folder: "KubeLB"

Security

Chart Signature Verification

Verify the Helm chart signature before installation:
cosign verify quay.io/kubermatic/helm-charts/kubelb-ccm:v1.3.1 \
  --certificate-identity-regexp="^https://github.com/kubermatic/kubelb/.github/workflows/release.yml@refs/tags/v.*" \
  --certificate-oidc-issuer=https://token.actions.githubusercontent.com

Image Signature Verification

Verify the CCM container image:
cosign verify quay.io/kubermatic/kubelb-ccm:v1.3.1 \
  --certificate-identity-regexp="^https://github.com/kubermatic/kubelb/.github/workflows/release.yml@refs/tags/v.*" \
  --certificate-oidc-issuer=https://token.actions.githubusercontent.com

Security Context

The CCM runs with restrictive security settings:
podSecurityContext:
  runAsNonRoot: true
  seccompProfile:
    type: RuntimeDefault

securityContext:
  allowPrivilegeEscalation: false
  capabilities:
    drop:
      - ALL
  runAsUser: 65532

Kubeconfig Secret Security

Ensure the kubeconfig Secret has minimal RBAC permissions in the management cluster. Only grant access to create and manage LoadBalancer and Route CRDs.

Advanced Configuration

High Availability

Run multiple replicas with leader election:
replicaCount: 2

kubelb:
  enableLeaderElection: true

Custom Volumes

Mount additional volumes for certificates or configuration:
extraVolumes:
  - name: ca-certs
    configMap:
      name: custom-ca-certs

extraVolumeMounts:
  - name: ca-certs
    mountPath: /etc/ssl/certs/custom
    readOnly: true

Priority Class

Set pod priority for critical workloads:
priorityClassName: "system-cluster-critical"

Logging Configuration

Adjust log verbosity:
kubelb:
  logLevel: debug  # debug, info, error, panic, or numeric value

Multiple Tenant Clusters

To connect multiple tenant clusters to the same management cluster:
1

Use unique tenant names

Ensure each tenant cluster has a unique tenantName value:
# Cluster 1
kubelb:
  tenantName: production-us-east
# Cluster 2
kubelb:
  tenantName: production-eu-west
2

Use the same management cluster

All tenant clusters should use the same management cluster kubeconfig in their secrets.
3

Install CCM in each cluster

Repeat the installation steps in each tenant cluster with the appropriate tenant name.

Upgrading

To upgrade KubeLB CCM to a new version:
1

Update CRDs

kubectl apply -f kubelb-ccm/crds/
2

Upgrade Helm release

helm upgrade kubelb-ccm oci://quay.io/kubermatic/helm-charts/kubelb-ccm \
  --version=v1.3.1 \
  --namespace kubelb \
  -f values.yaml

Troubleshooting

Check CCM Status

kubectl get pods -n kubelb -l app.kubernetes.io/name=kubelb-ccm

View CCM Logs

kubectl logs -n kubelb -l app.kubernetes.io/name=kubelb-ccm -f

Verify Management Cluster Connection

kubectl exec -n kubelb deploy/kubelb-ccm -- \
  cat /kubelb/kubelb | grep server

Check LoadBalancer Resources

Verify LoadBalancer CRDs are being created in the management cluster:
# Run on management cluster
kubectl get loadbalancers -A

Common Issues

CCM fails to connect to management cluster: Verify the kubeconfig Secret is correctly created and the management cluster API is accessible from the tenant cluster.
LoadBalancer services not getting external IPs: Check that useLoadBalancerClass is set correctly and services have the appropriate loadBalancerClass annotation if filtering is enabled.
Gateway API controllers failing to start: Ensure Gateway API CRDs are installed or set installGatewayAPICRDs: true.

Next Steps

After installing the CCM:
  1. Verify the installation to ensure the CCM is communicating with the Manager
  2. Create your first LoadBalancer service to test the setup

Build docs developers (and LLMs) love