Skip to main content

​
Update Strategy

Proxmox VE receives regular updates including security patches, bug fixes, and feature releases. This guide ensures updates are applied safely without disrupting production services. Update Schedule:
  • Security patches: Apply within 7 days of release
  • Minor updates: Monthly maintenance window
  • Major version upgrades (e.g., 8.x → 9.x): Planned maintenance with testing
Never update both Proxmox nodes simultaneously. Always update one node, verify stability, then update the second node. This ensures at least one node remains available if issues arise.

​
Pre-Update Checklist

1

Review Proxmox Changelog

Check official Proxmox release notes:https://pve.proxmox.com/wiki/RoadmapLook for:
  • Breaking changes
  • Known issues
  • Recommended procedures
2

Verify Backups are Current

PBS Dashboard (https://192.168.30.12:8007):
  • Last successful backup < 24 hours old
  • All critical VMs backed up:
    • docker-prod-01
    • auth-prod-01
    • immich-prod-01
    • pbs-prod-01
    • dns-prod-01 and dns-prod-02 (LXCs)
3

Check Cluster Health

Proxmox UI → Datacenter → ClusterVerify:
  • Both nodes show green status
  • Quorum is healthy (2 nodes + QDevice = 3 votes)
  • No split-brain warnings
4

Review Active VMs and Resource Usage

Datacenter → Summary
  • Note which VMs are running on each node
  • Check if any VMs can be temporarily migrated or shut down
  • Verify no resource-intensive tasks running (PBS backups, large file transfers)
5

Notify Users (If Applicable)

If updating during production hours:
  • Post notification in shared communication channel (Discord, etc.)
  • Estimate downtime window (typically 15-30 minutes per node)

​
Updating Proxmox (Standard Process)

​
Update pve-prod-02 First (Secondary Node)

Why secondary first?: pve-prod-02 runs fewer critical services (PBS + dns-prod-02). If update causes issues, primary services remain unaffected.
1

SSH to pve-prod-02

ssh [email protected]
2

Update Package Lists

apt update
Review output for any repository errors or warnings.
3

Check Available Updates

apt list --upgradable
Review the list. Look for:
  • pve-manager (Proxmox core)
  • proxmox-ve (meta-package)
  • Kernel updates (pve-kernel-*)
4

Perform Update

apt dist-upgrade -y
This updates all packages including kernel and Proxmox components.
Use dist-upgrade not upgrade. dist-upgrade properly handles dependency changes required by Proxmox updates.
5

Review Update Output

Watch for:
  • Kernel installation messages
  • Service restart prompts
  • Any error messages (rare but possible)
6

Reboot Node

If kernel was updated (check output above):
reboot
If only userspace packages updated:
systemctl restart pvedaemon pveproxy pvestatd
Recommendation: Reboot anyway to ensure clean state.
7

Verify Node After Reboot

Wait 2-3 minutes for boot, then:
  • Proxmox UI → Check pve-prod-02 shows green status
  • SSH back in: ssh [email protected]
  • Check kernel version: uname -r
  • Verify VMs auto-started: pct list and qm list
8

Test Critical Services

  • PBS: Access https://192.168.30.12:8007 — verify login works
  • dns-prod-02: ping 192.168.30.15 — verify LXC is up
  • Cluster: Proxmox UI → Datacenter → Cluster — verify both nodes visible

​
Update pve-prod-01 (Primary Node)

Wait at least 24 hours after updating pve-prod-02 before proceeding with pve-prod-01. This allows time to catch any delayed issues from the first update.
1

Review pve-prod-02 Stability

Confirm no issues from previous update:
  • Check Proxmox logs: pve-prod-02 → System → Syslog
  • Verify PBS backups completed successfully overnight
  • No unexpected reboots or service failures
2

Optional: Migrate Critical VMs

If you want zero downtime for critical services:Proxmox UI → [Select VM] → MigrateMigrate docker-prod-01 to pve-prod-02 temporarily.
Migration limitations: VMs using local storage (not NFS) cannot live-migrate. docker-prod-01 uses local disk for appdata — migration requires shutdown.For true zero-downtime, skip migration and accept brief service interruption during reboot.
3

Update pve-prod-01

Repeat same process as pve-prod-02:
ssh [email protected]
apt update
apt list --upgradable
apt dist-upgrade -y
reboot
4

Verify Node After Reboot

  • Check pve-prod-01 status in cluster
  • Verify all VMs/LXCs auto-started
  • Test service access via Traefik (https://homarr.giohosted.com)
5

Migrate VMs Back (If Migrated)

If you migrated docker-prod-01 earlier, migrate back to pve-prod-01.

​
Post-Update Verification

1

Check Proxmox Version

Proxmox UI → Datacenter → SummaryVerify both nodes show identical versions:
  • Proxmox VE: X.Y-Z
  • Kernel: X.Y.Z-pve
2

Verify Cluster Quorum

Datacenter → ClusterExpected votes: 3 (node1 + node2 + QDevice on Pi)If quorum is broken:
pvecm status  # Check cluster status
systemctl status corosync  # Verify Corosync running
3

Run PBS Backup Manually

Test that backups still work post-update:Datacenter → Backup → [Select Job] → Run NowVerify backup completes without errors.
4

Check VM Performance

Monitor for 24-48 hours:
  • Beszel metrics for CPU/RAM usage anomalies
  • Service response times in Uptime Kuma
  • Docker container logs for errors
5

Update Documentation

Record update in homelab-docs:
  • Date updated
  • Version before → after
  • Any issues encountered
  • Downtime duration

​
Major Version Upgrades (e.g., Proxmox 8.x → 9.x)

Major version upgrades require additional planning.
1

Review Official Upgrade Guide

Proxmox publishes detailed upgrade guides:https://pve.proxmox.com/wiki/Upgrade_from_X.Y_to_Z.0Read completely before proceeding.
2

Run pve8to9 Checklist Tool

Proxmox provides pre-upgrade checkers:
pve8to9 --full
Review output for blockers or warnings.
3

Snapshot Critical VMs

Before starting upgrade:Proxmox UI → [Select VM] → Snapshots → Take Snapshot
  • docker-prod-01-pre-upgrade
  • auth-prod-01-pre-upgrade
  • immich-prod-01-pre-upgrade
4

Update Sources List

sed -i 's/bullseye/bookworm/g' /etc/apt/sources.list
sed -i 's/bullseye/bookworm/g' /etc/apt/sources.list.d/*.list
Or manually edit and replace repository URLs per official guide.
5

Perform Upgrade

apt update
apt dist-upgrade
This may take 30-60 minutes. Do not interrupt.
6

Reboot and Verify

reboot
After reboot:
  • Check Proxmox web UI loads
  • Verify version in Datacenter → Summary
  • Test VM start/stop functionality
Major upgrades should be tested in a lab environment first if possible. Consider building a test Proxmox VM to simulate the upgrade before applying to production.

​
Troubleshooting Update Issues

​
Issue: Held Packages Prevent Update

Symptom: 
The following packages have been kept back:
  pve-kernel-6.x.x-x-pve
Fix: 
apt install pve-kernel-6.x.x-x-pve  # Explicitly install held kernel
apt dist-upgrade

​
Issue: Repository Errors (No Subscription)

Symptom: 
E: Failed to fetch https://enterprise.proxmox.com/...
Fix: Disable enterprise repo, enable no-subscription repo 
# Disable enterprise repo
echo "#deb https://enterprise.proxmox.com/debian/pve bullseye pve-enterprise" > /etc/apt/sources.list.d/pve-enterprise.list

# Enable no-subscription repo
echo "deb http://download.proxmox.com/debian/pve bullseye pve-no-subscription" > /etc/apt/sources.list.d/pve-no-subscription.list

apt update

​
Issue: Cluster Split-Brain After Reboot

Symptom: Nodes show as offline in each other’s UI, but both are actually running. Fix: 
# On each node, check Corosync status
systemctl status corosync

# Restart Corosync and cluster services
systemctl restart corosync
systemctl restart pve-cluster

# Check quorum
pvecm status
If QDevice is unreachable: 
# On pi-prod-01, restart QDevice
systemctl restart corosync-qdevice

​
Issue: VMs Don’t Auto-Start After Reboot

Fix: Check VM config for auto-start: 
qm config <VMID> | grep onboot
Enable auto-start: 
qm set <VMID> --onboot 1

​
Issue: Web UI Unreachable After Update

Check: 
# Verify pveproxy is running
systemctl status pveproxy

# Restart if needed
systemctl restart pveproxy pvedaemon

# Check firewall
iptables -L -n | grep 8006

​
Emergency Rollback

If an update causes critical issues:
1

Boot Previous Kernel

At GRUB boot menu:
  • Select “Advanced options for Proxmox VE”
  • Choose previous kernel version
2

Hold Problematic Packages

apt-mark hold pve-manager proxmox-ve pve-kernel-*
This prevents automatic updates until issue is resolved.
3

Restore from PBS Snapshot (Worst Case)

If node is completely broken:
  • Boot from Proxmox installation USB
  • Install Proxmox fresh on new storage
  • Restore VMs from PBS on pve-prod-02

​
Update Automation (Optional)

For automated security updates only (not major upgrades): 
# Install unattended-upgrades
apt install unattended-upgrades

# Configure to only install security updates
dpkg-reconfigure -plow unattended-upgrades
Edit /etc/apt/apt.conf.d/50unattended-upgrades: 
Unattended-Upgrade::Origins-Pattern {
  "origin=Debian,codename=${distro_codename},label=Debian-Security";
};

Unattended-Upgrade::Automatic-Reboot "false";  // Never auto-reboot
Not recommended for homelab: Auto-updates can break configurations. Manual updates with testing are safer for production services.

​
Update Maintenance Schedule

Monthly (First Sunday, 02:00-06:00):
  • Update pve-prod-02
  • Wait 7 days, then update pve-prod-01
Immediately (As Released):
  • Critical security patches (review CVE severity)
Planned Maintenance (Twice Yearly):
  • Major version upgrades (e.g., Proxmox 8.x → 9.x)
  • Debian base OS upgrades (e.g., Bullseye → Bookworm)

Build docs developers (and LLMs) love

Get started for freeTalk to us