Skip to main content

​
Introduction

GuestOS is the operating system that runs inside a QEMU virtual machine on top of HostOS. It contains the replica and orchestrator binaries and is where the actual Internet Computer protocol execution occurs. The GuestOS virtual machine provides:
  • Consistent runtime environment across various hardware platforms
  • Secure isolation from the host system
  • Unified upgrade mechanism for the OS and all IC services
All IC protocol operations—including consensus, message routing, and canister execution—run exclusively in GuestOS. HostOS does not perform any trusted protocol operations.

​
Architecture

​
Virtual Machine Execution

GuestOS runs as a QEMU/KVM virtual machine with specific characteristics:
  • CPU Passthrough: Direct access to physical CPU features and capabilities
  • Memory: Configurable memory allocation for replica operations
  • Storage: Virtual disk with encrypted partitions for data and state
  • Networking: Virtualized network interface managed by HostOS

​
Core Components

GuestOS includes several key components:
The core Internet Computer protocol implementation that handles:
  • Consensus participation
  • Message routing
  • Canister execution
  • State management
Manages the replica lifecycle and coordinates:
  • Replica upgrades
  • Configuration updates
  • Health monitoring
  • Registry synchronization
Supervises and starts IC services:
  • Monitors replica process
  • Handles service restarts
  • Manages service dependencies
Exports system metrics for monitoring:
  • Hardware metrics
  • System performance
  • Resource utilization
  • Custom IC metrics

​
Disk Layout

GuestOS uses a sophisticated partition structure to enable secure A/B upgrades: 
/dev/vda1  - esp       (EFI system partition)
/dev/vda2  - grub      (Grub bootloader modules and config)
/dev/vda3  - config    (System config store)
/dev/vda4  - A_boot    (Boot partition for system A)
/dev/vda5  - A_root    (Root partition for system A)
/dev/vda6  - A_var     (Mutable data partition for system A)
/dev/vda7  - B_boot    (Boot partition for system B)
/dev/vda8  - B_root    (Root partition for system B)
/dev/vda9  - B_var     (Mutable data partition for system B)
/dev/vda10 - store     (Encrypted LVM for payload data)

​
Key Partitions

Contains the primary GRUB bootloader at EFI/Boot/bootx64.efi. This bootloader is signed and verified by EFI firmware for tamper resistance.

​
Boot Sequence

The GuestOS boot process involves several IC-specific initialization steps:

​
Early Boot (Initramfs)

  1. Machine ID Loading: Check /config for existing machine-id
  2. Root Filesystem Switch: Mount the root partition (read-only)
  3. Boot Partition Mount: Mount appropriate boot partition (A or B)

​
System Initialization

1

Mount Configuration Media

Service: mount-config.serviceLocates and mounts the configuration device (USB, device with serial “config”, or filesystem labeled “CONFIG”) to /mnt/config.
2

Set Up Encryption

Service: setup-data-encryption.serviceOn first boot, creates the 10th partition for encrypted storage, generates encryption key, and sets up encrypted LVM.
3

Set Up SSH Keys

Service: generate-host-ssh-keys.serviceGenerates SSH host keys on first boot and stores them in the config partition. Keys are bind-mounted to /etc/ssh on subsequent boots.
4

Initialize Logical Volumes

Service: setup-lvs.serviceCreates volume group store and logical volumes:
  • store/shared-crypto
  • store/shared-backup
  • store/shared-data
5

Bootstrap IC Node

Service: bootstrap-ic-node.serviceOn first boot only, processes ic-bootstrap.tar from /mnt/config to populate:
  • SSH authorized keys
  • /var/lib/ic/data initial state
  • /var/lib/ic/crypto cryptographic material
6

Configure Networking

Service: generate-network-config.serviceParses network configuration from config partition and generates systemd-networkd configuration files.
7

Start IC Services

Services: node_exporter.service, nodemanager.serviceStarts monitoring and the node manager, which then launches the IC replica.

​
Security Features

GuestOS implements multiple layers of security:

​
Encryption

  • Var Partitions: Encrypted to protect intermediate data and logs
  • Store Partition: Encrypted LVM for all IC payload data
  • Block Device Protection: Both confidentiality and integrity protection

​
Integrity Protection

  • dm-verity: Root filesystem integrity verification
  • Signed Bootloader: EFI firmware verifies GRUB signature
  • Signed Boot Files: GRUB verifies kernel and initrd signatures
  • Read-only Root: Immutable root filesystem prevents tampering

​
SELinux

GuestOS includes a custom SELinux policy that enforces mandatory access control. The policy restricts:
  • Process capabilities
  • File access permissions
  • Network operations
  • Inter-process communication
For detailed information about the SELinux implementation, refer to the SELinux documentation in ic-os/guestos/docs/SELinux.adoc.

​
Communication with HostOS

GuestOS communicates with HostOS through VM sockets (VSOCK), a controlled interface that maintains isolation:
To maintain security isolation, GuestOS is restricted to strictly defined VSOCK commands. All commands are initiated from GuestOS, not HostOS.
Available VSOCK commands allow GuestOS to:
  • Request system upgrades
  • Query HostOS status
  • Coordinate partition switching
  • Request controlled host operations
For complete details, see the vsock README.

​
System Users

GuestOS includes several specialized user accounts:
UsernameHome DirectoryDescription
backup/var/lib/backupBackup subnet state operations
readonly/var/lib/readonlyRead-only administrative access
admin/var/lib/adminAdministrative access for Node Providers
node_exporter/home/node_exporterNode exporter service account
For development images (dev target), the root account is enabled with username and password both set to root. Production images (prod target) do not have console access.

​
Running GuestOS Locally

For development and testing, you can run GuestOS directly with QEMU:
Requires QEMU with qemu-system and qemu-kvm installed.
qemu-system-x86_64 \
  -machine type=q35,accel=kvm \
  -enable-kvm \
  -nographic \
  -m 4G \
  -bios /usr/share/ovmf/OVMF.fd \
  -device vhost-vsock-pci,guest-cid=$$CID \
  -drive file=disk.img,format=raw,if=virtio \
  -netdev user,id=user.0,hostfwd=tcp::2222-:22 \
  -device virtio-net,netdev=user.0
Press Ctrl-A followed by x to exit the QEMU console.

​
Configuration

GuestOS configuration is managed through several mechanisms:

​
Bootstrap Configuration

Initial configuration provided via ic-bootstrap.tar on the configuration media:
  • Network settings
  • SSH keys
  • Initial IC registry configuration
  • Node identity and cryptographic material

​
Runtime Configuration

Generated at boot time from templates:
  • ic.json.template → ic.json: Main IC configuration
  • Network configuration from config partition
  • NFTables rules from IC registry

​
Config Partition

Small partition (100 MB) preserved across upgrades containing:
  • Machine ID (stable system identifier)
  • SSH host keys (stable across upgrades)
  • Node exporter TLS certificates
  • Bootstrap configuration

​
Performance Benchmarking

GuestOS performance is continuously monitored through automated benchmarks:
  • Runs daily on bare metal hardware
  • Executed as part of Bazel Test Bare Metal CI job
  • Results tracked over time to detect regressions
  • Tests real-world workloads on actual hardware
For manual benchmarking instructions, see ic-os/dev-tools/bare_metal_deployment/README.md.

​
Development

The GuestOS system logic and Ubuntu configuration is contained in the ic-os/components/ directory.

Adding Files

Add files to components/ directory and enumerate in components/guestos.bzl

Removing Files

Remove from components/ and remove enumeration from all relevant .bzl files

System Services

Add systemd service files to start and control your services

Late Injection

Some binaries injected after rootfs construction (see defs.bzl)
For detailed development instructions, see the components README.

​
Next Steps

Build GuestOS

Learn how to build GuestOS images

HostOS

Understand the host operating system

Build docs developers (and LLMs) love

Get started for freeTalk to us