Upgrading - KloudMate Agent

This guide covers upgrading the KloudMate Agent for each supported platform. Always review the release notes before upgrading.

Pre-Upgrade Checklist

Check Current Version

Verify your current agent version:Linux:

kmagent --version
dpkg -l | grep kmagent  # Debian/Ubuntu
rpm -q kmagent          # RHEL/CentOS

Docker:

docker inspect km-agent | grep Image

Kubernetes:

kubectl get pods -n km-agent -o jsonpath='{.items[0].spec.containers[0].image}'
helm list -n km-agent

Review Release Notes

Check for breaking changes at GitHub Releases

Backup Configuration

Linux:

sudo cp /etc/kmagent/config.yaml /etc/kmagent/config.yaml.backup

Kubernetes:

kubectl get configmap -n km-agent km-agent-configmap-daemonset -o yaml > configmap-backup.yaml
helm get values kloudmate-release -n km-agent > values-backup.yaml

Verify Connectivity

Ensure the system can reach GitHub and KloudMate endpoints:

curl -I https://api.github.com/repos/kloudmate/km-agent/releases/latest
curl -I https://otel.kloudmate.com:4318

Linux Upgrade

The Linux installation script automatically fetches and installs the latest version.

Automatic Upgrade (Recommended)

The installation script handles upgrades automatically. It will:

Fetch the latest release from GitHub
Download the appropriate package for your architecture
Install/upgrade the package
Restart the service with existing configuration

scripts/install_linux.sh

KM_API_KEY="<YOUR_API_KEY>" KM_COLLECTOR_ENDPOINT="https://otel.kloudmate.com:4318" bash -c "$(curl -L https://cdn.kloudmate.com/scripts/install_linux.sh)"

The script detects your OS and architecture:

scripts/install_linux.sh

ARCH=$(uname -m)
DEB_ARCH="amd64"
RPM_ARCH="x86_64"

if [ "$ARCH" = "aarch64" ] || [ "$ARCH" = "arm64" ]; then
  DEB_ARCH="arm64"
  RPM_ARCH="aarch64"
fi

Manual Upgrade

If you prefer to upgrade manually:

Debian/Ubuntu
RHEL/CentOS

Download Package:

wget https://github.com/kloudmate/km-agent/releases/download/v1.2.0/kmagent_1.2.0_amd64.deb

Install:

sudo dpkg -i kmagent_1.2.0_amd64.deb

Restart Service:

sudo systemctl daemon-reexec
sudo systemctl restart kmagent

Verify:

kmagent --version
sudo systemctl status kmagent

Download Package:

wget https://github.com/kloudmate/km-agent/releases/download/v1.2.0/kmagent-1.2.0.x86_64.rpm

Install:

sudo dnf install -y kmagent-1.2.0.x86_64.rpm
# or
sudo yum install -y kmagent-1.2.0.x86_64.rpm

Restart Service:

sudo systemctl daemon-reexec
sudo systemctl restart kmagent

Verify:

kmagent --version
sudo systemctl status kmagent

Post-Upgrade Verification (Linux)

Check Service Status

sudo systemctl status kmagent

Service should be active (running)

Verify Version

kmagent --version

Check Logs

sudo journalctl -u kmagent -f

Look for:

INFO agent start sequence initiated
INFO collector instance created, starting run loop

Test Health Endpoint

curl http://localhost:13133/

Should return 200 OK

Docker Upgrade

Docker upgrades involve pulling the latest image and recreating the container.

Automatic Upgrade (Recommended)

Use the installation script to upgrade:

scripts/install_docker.sh

KM_API_KEY="<YOUR_API_KEY>" KM_COLLECTOR_ENDPOINT="https://otel.kloudmate.com:4318" bash -c "$(curl -L https://cdn.kloudmate.com/scripts/install_docker.sh)"

The script automatically:

Pulls the latest image
Stops and removes the old container
Creates a new container with the updated image

scripts/install_docker.sh

echo "📥 Pulling Docker image: $IMAGE_NAME..."
docker pull $IMAGE_NAME

echo "🛑 Stopping and removing any existing 'km-agent' container..."
if [ "$(docker ps -aq -f name=km-agent)" ]; then
    docker stop km-agent
    docker rm km-agent
fi

Manual Upgrade

Pull Latest Image

docker pull ghcr.io/kloudmate/km-agent:latest

Stop and Remove Old Container

docker stop km-agent
docker rm km-agent

Create New Container

docker run -d \
  --privileged \
  --pid host \
  --restart always \
  --network host \
  --name km-agent \
  -e KM_COLLECTOR_ENDPOINT="https://otel.kloudmate.com:4318" \
  -e KM_API_KEY="<YOUR_API_KEY>" \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v /var/log:/var/log \
  -v /var/lib/docker/containers:/var/lib/docker/containers:ro \
  -v /:/hostfs:ro \
  ghcr.io/kloudmate/km-agent:latest

Post-Upgrade Verification (Docker)

Check Container Status

docker ps -f name=km-agent

Container should be Up

Verify Image Version

docker inspect km-agent | grep Image

Check Logs

docker logs km-agent --tail 50

Look for successful startup messages

Test Health Endpoint

docker exec km-agent curl http://localhost:13133/

Cleanup Old Images

After successful upgrade, remove old images:

# List all km-agent images
docker images | grep km-agent

# Remove old images
docker image prune -f

Kubernetes Upgrade

Kubernetes upgrades are managed through Helm.

Helm Upgrade

The recommended approach is to upgrade via Helm, which handles rolling updates automatically.

Update Helm Repository

helm repo update kloudmate

Check Available Versions

helm search repo kloudmate/km-kube-agent --versions

Upgrade Release

Upgrade to latest:

README.md

helm upgrade kloudmate-release kloudmate/km-kube-agent \
  --namespace km-agent \
  --reuse-values

Upgrade to specific version:

helm upgrade kloudmate-release kloudmate/km-kube-agent \
  --namespace km-agent \
  --version 1.2.0 \
  --reuse-values

Upgrade with new values:

helm upgrade kloudmate-release kloudmate/km-kube-agent \
  --namespace km-agent \
  --set API_KEY="<YOUR_API_KEY>" \
  --set COLLECTOR_ENDPOINT="https://otel.kloudmate.com:4318" \
  --set clusterName="production-cluster"

Monitor Rollout

DaemonSet:

kubectl rollout status daemonset/km-agent -n km-agent

Deployment:

kubectl rollout status deployment/km-agent-cluster -n km-agent

Update Container Images Only

If you only need to update container images:

kubectl set image daemonset/km-agent -n km-agent \
  km-agent=ghcr.io/kloudmate/km-kube-agent:v1.2.0

kubectl set image deployment/km-agent-cluster -n km-agent \
  km-agent=ghcr.io/kloudmate/km-kube-agent:v1.2.0

Post-Upgrade Verification (Kubernetes)

Check Pod Status

kubectl get pods -n km-agent

All pods should be Running and Ready

Verify Image Versions

kubectl get pods -n km-agent -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.spec.containers[0].image}{"\n"}{end}'

Check Rollout History

kubectl rollout history daemonset/km-agent -n km-agent
kubectl rollout history deployment/km-agent-cluster -n km-agent

Review Pod Logs

kubectl logs -n km-agent -l app.kubernetes.io/component=node-agent --tail=50

Test Health Endpoint

kubectl exec -n km-agent ds/km-agent -- curl http://localhost:13133/

Rollback (If Needed)

If the upgrade causes issues:

Helm Rollback
kubectl Rollback

View revision history:

helm history kloudmate-release -n km-agent

Rollback to previous:

helm rollback kloudmate-release -n km-agent

Rollback to specific revision:

helm rollback kloudmate-release 3 -n km-agent

DaemonSet:

kubectl rollout undo daemonset/km-agent -n km-agent

Deployment:

kubectl rollout undo deployment/km-agent-cluster -n km-agent

To specific revision:

kubectl rollout undo daemonset/km-agent --to-revision=2 -n km-agent

Windows Upgrade

Windows upgrades require downloading and running the latest installer from the releases page.

Download Latest Installer

Download the .exe installer from GitHub Releases

Stop Service

Stop-Service -Name "KloudMate Agent"

Run Installer

Execute the downloaded installer. It will:

Detect existing installation
Preserve configuration
Upgrade binaries
Restart service

Verify Installation

Get-Service -Name "KloudMate Agent"

Upgrade Strategies

Zero-Downtime Upgrades (Kubernetes)

For production environments, use rolling updates:

deployment/helm/km-kube-agent/values.yaml

strategy:
  type: RollingUpdate
  rollingUpdate:
    maxUnavailable: 1
    maxSurge: 1

Helm automatically performs rolling updates for DaemonSets and Deployments.

Blue-Green Deployment (Kubernetes)

For critical environments:

Deploy New Version to Test Namespace

helm install kloudmate-test kloudmate/km-kube-agent \
  --namespace km-agent-test \
  --create-namespace \
  --set API_KEY="<YOUR_API_KEY>"

Validate New Version

Run tests against the test environment

Switch Traffic

Update production namespace:

helm upgrade kloudmate-release kloudmate/km-kube-agent \
  --namespace km-agent \
  --reuse-values

Cleanup Test Environment

helm uninstall kloudmate-test -n km-agent-test
kubectl delete namespace km-agent-test

Configuration Migration

When upgrading between major versions, configuration may need migration.

Configuration Management in Kubernetes:Manually updating ConfigMaps is not recommended, as configurations may be overwritten by KloudMate API updates. Use the KloudMate Agent Config Editor web interface instead.

From the README:

README.md

⚠️ **Configuration Management**: Manually updating the ConfigMaps for the DaemonSet or Deployment agents is **not recommended**, as these configurations may be overwritten by updates sent from KloudMate APIs. The recommended approach is to use the **KloudMate Agent Config Editor**.

Troubleshooting Upgrades

Upgrade Fails

Package Manager Errors (Linux)

Dependency conflicts:

# Debian/Ubuntu
sudo apt-get install -f -y

# RHEL/CentOS
sudo dnf distro-sync

Helm Upgrade Failures

Check for failed resources:

kubectl get events -n km-agent --sort-by='.lastTimestamp'

Verify CRDs are up to date:

kubectl get crd instrumentations.opentelemetry.io -o yaml

Image Pull Failures

Verify image exists:

docker pull ghcr.io/kloudmate/km-kube-agent:latest

Check network connectivity to ghcr.io

Service Won’t Start After Upgrade

Check Logs

# Linux
sudo journalctl -u kmagent -n 100

# Docker
docker logs km-agent --tail 100

# Kubernetes
kubectl logs -n km-agent <pod-name> --tail 100

Verify Configuration

Ensure configuration files weren’t corrupted during upgrade

Check Permissions

# Linux
ls -la /usr/bin/kmagent
ls -la /etc/kmagent/config.yaml

Rollback If Necessary

Reinstall previous version or use Helm rollback

Best Practices

Test First

Always test upgrades in non-production environments first

Backup Configuration

Save configuration files before upgrading

Monitor After Upgrade

Watch metrics and logs for 24 hours post-upgrade

Plan Rollback

Have a rollback plan ready before upgrading

Schedule Maintenance

Upgrade during low-traffic periods

Document Changes

Record version numbers and any issues encountered

Next Steps

Monitoring

Set up monitoring for the upgraded agent

Troubleshooting

Resolve common issues after upgrade

Getting Started

Installation

Configuration

Features

Architecture

Operations

​Pre-Upgrade Checklist

​Linux Upgrade

​Automatic Upgrade (Recommended)

​Manual Upgrade

​Post-Upgrade Verification (Linux)

​Docker Upgrade

​Automatic Upgrade (Recommended)

​Manual Upgrade

​Post-Upgrade Verification (Docker)

​Cleanup Old Images

​Kubernetes Upgrade

​Helm Upgrade

​Update Container Images Only

​Post-Upgrade Verification (Kubernetes)

​Rollback (If Needed)

​Windows Upgrade

​Upgrade Strategies

​Zero-Downtime Upgrades (Kubernetes)

​Blue-Green Deployment (Kubernetes)

​Configuration Migration

​Troubleshooting Upgrades

​Upgrade Fails

​Service Won’t Start After Upgrade

​Best Practices

Test First

Backup Configuration

Monitor After Upgrade

Plan Rollback

Schedule Maintenance

Document Changes

​Next Steps

Monitoring

Troubleshooting

Build docs developers (and LLMs) love

Pre-Upgrade Checklist

Linux Upgrade

Automatic Upgrade (Recommended)

Manual Upgrade

Post-Upgrade Verification (Linux)

Docker Upgrade

Automatic Upgrade (Recommended)

Manual Upgrade

Post-Upgrade Verification (Docker)

Cleanup Old Images

Kubernetes Upgrade

Helm Upgrade

Update Container Images Only

Post-Upgrade Verification (Kubernetes)

Rollback (If Needed)

Windows Upgrade

Upgrade Strategies

Zero-Downtime Upgrades (Kubernetes)

Blue-Green Deployment (Kubernetes)

Configuration Migration

Troubleshooting Upgrades

Upgrade Fails

Service Won’t Start After Upgrade

Best Practices

Next Steps