After installing KubeLB Manager and CCM, verify that all components are running correctly and can communicate with each other.
Verify Management Cluster First, verify that the KubeLB Manager is running properly in the management cluster.
Check Manager Deployment
Check Manager pods
Verify the Manager pod is running: kubectl get pods -n kubelb -l app.kubernetes.io/name=kubelb-manager
Expected output: NAME READY STATUS RESTARTS AGE kubelb-manager-xxxxxxxxxx-xxxxx 2/2 Running 0 5m
The Manager pod has 2 containers: the manager itself and kube-rbac-proxy.
Check Envoy proxy deployment
Verify the Envoy proxy pods are running: kubectl get pods -n kubelb -l app=envoy-proxy
Expected output (for replicas: 2): NAME READY STATUS RESTARTS AGE envoy-proxy-xxxxxxxxxx-xxxxx 1/1 Running 0 5m envoy-proxy-xxxxxxxxxx-xxxxx 1/1 Running 0 5m
Check Manager service
Verify the Manager service is created: kubectl get svc -n kubelb kubelb-manager
Expected output: NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubelb-manager ClusterIP 10.96.123.456 <none> 8001/TCP 5m
Check CRDs
Verify that KubeLB CRDs are installed: kubectl get crds | grep kubelb.k8c.io
Expected output: configs.kubelb.k8c.io loadbalancers.kubelb.k8c.io routes.kubelb.k8c.io tenants.kubelb.k8c.io
Check Manager Logs Review the Manager logs for any errors:
kubectl logs -n kubelb -l app.kubernetes.io/name=kubelb-manager -c manager --tail=50
Look for successful initialization messages:
INFO Starting manager INFO Starting server INFO Envoy xDS control plane started
If you see errors related to Gateway API, ensure Gateway API CRDs are installed or disable Gateway API support by setting kubelb.enableGatewayAPI: false.
Verify Tenant Cluster Next, verify that the KubeLB CCM is running in each tenant cluster.
Check CCM Deployment
Check CCM pods
Verify the CCM pod is running: kubectl get pods -n kubelb -l app.kubernetes.io/name=kubelb-ccm
Expected output: NAME READY STATUS RESTARTS AGE kubelb-ccm-xxxxxxxxxx-xxxxx 2/2 Running 0 5m
Check management cluster secret
Verify the management cluster kubeconfig secret exists: kubectl get secret -n kubelb kubelb-cluster
Expected output: NAME TYPE DATA AGE kubelb-cluster Opaque 1 10m
Check CRDs
Verify that KubeLB CRDs are installed: kubectl get crds | grep kubelb.k8c.io
Expected output should include the same CRDs as in the management cluster. Check CCM Logs Review the CCM logs for any errors:
kubectl logs -n kubelb -l app.kubernetes.io/name=kubelb-ccm -c manager --tail=50
Look for successful initialization:
INFO Starting manager INFO Connected to management cluster INFO Tenant registered: tenant-cluster-1
Test Load Balancer Creation Create a test LoadBalancer service to verify end-to-end functionality.
Deploy Test Application
Create test deployment
In the tenant cluster, create a simple test application: apiVersion : apps/v1 kind : Deployment metadata : name : test-app namespace : default spec : replicas : 2 selector : matchLabels : app : test-app template : metadata : labels : app : test-app spec : containers : - name : nginx image : nginx:latest ports : - containerPort : 80 --- apiVersion : v1 kind : Service metadata : name : test-app-lb namespace : default spec : type : LoadBalancer selector : app : test-app ports : - port : 80 targetPort : 80 protocol : TCP
Apply the manifest: kubectl apply -f test-app.yaml
Check service status
Wait for the service to get an external IP (this may take a minute): kubectl get svc test-app-lb -w
Expected output: NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE test-app-lb LoadBalancer 10.96.123.456 203.0.113.10 80:31234/TCP 30s
Verify LoadBalancer resource in management cluster
Switch to the management cluster and check that the LoadBalancer CRD was created: kubectl get loadbalancers -A
Expected output: NAMESPACE NAME AGE tenant-cluster-1 default-test-app-lb 1m
Check Envoy proxy service
Verify that a LoadBalancer service was created for the Envoy proxy: kubectl get svc -n kubelb
Look for services related to your tenant: NAME TYPE EXTERNAL-IP tenant-cluster-1-default-test-app-lb LoadBalancer 203.0.113.10
Test connectivity
Test that you can reach the application through the load balancer: You should see the nginx welcome page. Cleanup Test Resources Remove the test application:
kubectl delete -f test-app.yaml
Verify the LoadBalancer resource is removed from the management cluster:
# Run on management cluster kubectl get loadbalancers -A
Verify Metrics (Optional) If you enabled monitoring, verify that metrics are being collected.
Check Metrics Endpoints Manager Metrics
CCM Metrics
# Port-forward to the Manager metrics endpoint kubectl port-forward -n kubelb svc/kubelb-manager 9443:9443 # In another terminal, check metrics curl -k https://localhost:9443/metrics
Check ServiceMonitor If using Prometheus Operator:
kubectl get servicemonitor -n kubelb
Troubleshooting Manager Not Starting Check pod status
Check events
Check logs
kubectl describe pod -n kubelb -l app.kubernetes.io/name=kubelb-manager
CCM Cannot Connect to Management Cluster
Verify secret contents
kubectl get secret -n kubelb kubelb-cluster -o jsonpath='{.data.kubelb}' | base64 -d
Ensure the kubeconfig is valid and points to the correct management cluster.
Test connectivity
kubectl exec -n kubelb deploy/kubelb-ccm -c manager -- \ sh -c 'kubectl --kubeconfig=/kubelb/kubelb cluster-info'
Check RBAC permissions
Verify the kubeconfig has permissions to create LoadBalancer and Route resources in the management cluster.
Service Not Getting External IP
Check LoadBalancer class filtering
If useLoadBalancerClass: true, ensure services have the correct annotation: metadata : annotations : loadBalancerClass : kubelb
Check CCM logs
kubectl logs -n kubelb -l app.kubernetes.io/name=kubelb-ccm -c manager | grep -i error
Verify LoadBalancer CRD creation
Check if the LoadBalancer CRD was created in the management cluster: kubectl get loadbalancers -A
Check management cluster LoadBalancer implementation
Ensure the management cluster has a working LoadBalancer implementation (cloud provider or MetalLB).
Envoy Proxy Issues Check Envoy logs
Check Envoy configuration
Check xDS connection
kubectl logs -n kubelb -l app=envoy-proxy --tail=100
Health Checks Use these commands for ongoing health monitoring:
Manager Health
CCM Health (run in tenant cluster)
Resources (run in management cluster)
kubectl get pods -n kubelb -l app.kubernetes.io/name=kubelb-manager kubectl get pods -n kubelb -l app=envoy-proxy kubectl get svc -n kubelb
Next Steps Once verification is complete:
Review the Architecture documentation to understand the traffic flow
Learn how to use LoadBalancer services, Ingress, and Gateway API resources with KubeLB
Configure monitoring and alerting for production deployments
Set up high availability by increasing replica counts
