Skip to main content
The Build Scheduler is the orchestration engine that manages the parallel building of packages in Azure Linux. It reads a dependency graph from the grapher tool and intelligently schedules package builds across multiple workers, respecting dependencies and maximizing parallelism.

​
Usage

scheduler \
  --input <graph.dot> \
  --output <built-graph.dot> \
  --output-build-state-csv-file <build-state.csv> \
  --work-dir <work-dir> \
  --worker-tar <worker_chroot.tar.gz> \
  --repo-file <local.repo> \
  --rpm-dir <rpms-dir> \
  --toolchain-rpms-dir <toolchain-dir> \
  --srpm-dir <srpms-dir> \
  --cache-dir <cache-dir> \
  --build-logs-dir <logs-dir> \
  --dist-tag <dist-tag> \
  --distro-release-version <version> \
  --distro-build-number <build-number> \
  --build-agent <agent-type> \
  [options]

​
Core Parameters

​
input
string
required
Path to the DOT graph file to build (output from grapher)
​
output
string
required
Path to save the built DOT graph file
​
output-build-state-csv-file
string
required
Path to save the CSV file with build state information
​
work-dir
string
required
The directory to create the build folder
​
worker-tar
string
required
Full path to worker_chroot.tar.gz
​
repo-file
string
required
Full path to local.repo
​
rpm-dir
string
required
The directory to use as the local repo and to submit RPM packages to
​
toolchain-rpms-dir
string
required
Directory that contains already built toolchain RPMs. Should contain top level directories for architecture.
​
srpm-dir
string
required
The output directory for source RPM packages
​
cache-dir
string
required
The cache directory containing downloaded dependency RPMS from Azure Linux Base
​
build-logs-dir
string
required
Directory to store package build logs
​
dist-tag
string
required
The distribution tag SRPMs will be built with (e.g., .azl3)
​
distro-release-version
string
required
The distro release version that the SRPM will be built with
​
distro-build-number
string
required
The distro build number that the SRPM will be built with
​
build-agent
string
required
Type of build agent to build packages with. Options: chroot, test

​
Build Control Parameters

​
workers
int
default:"0"
Number of concurrent build agents to spawn. If set to 0, automatically sets to the logical CPU count.
​
build-attempts
int
default:"1"
Sets the number of times to try building a package before giving up
​
check-attempts
int
default:"1"
Sets the minimum number of times to test a package if the tests fail
​
max-cascading-rebuilds
int
Sets the maximum number of cascading dependency rebuilds caused by a package being rebuilt (leave unset for unbounded)
​
extra-layers
int
default:"0"
Sets the number of additional layers in the graph beyond the goal packages to build
​
timeout
duration
default:"99h"
Max duration for any individual package build/test (e.g., 2h, 30m)
​
max-cpu
string
Max number of CPUs used for package building

​
Package Selection Parameters

​
packages
string
Space separated list of top-level packages that should be built. Omit to build all packages.
​
rebuild-packages
string
Space separated list of base package names that should be rebuilt
​
ignored-packages
string
Space separated list of specs ignoring rebuilds if their dependencies have been updated
​
image-config-file
string
Optional image config file to extract a package list from
​
base-dir
string
Base directory for relative file paths from the config. Defaults to config’s directory.

​
Test Parameters

​
tests
string
Space separated list of tests that should be run. Omit to run package tests.
​
rerun-tests
string
Space separated list of package tests that should be re-run
​
ignored-tests
string
Space separated list of package tests that should not be run

​
Build Options

​
stop-on-failure
boolean
Stop the entire build on first package failure
​
no-cleanup
boolean
Don’t delete the chroot folders after builds are done
​
no-cache
boolean
Disables using prebuilt cached packages
​
optimize-with-cached-implicit
boolean
Optimize the build process by allowing cached implicit packages to optimize the initial build graph
​
allow-toolchain-rebuilds
boolean
Allow toolchain packages to rebuild without causing an error
​
toolchain-manifest
string
Path to a list of RPMs which are created by the toolchain. RPMs from this list are considered ‘prebuilt’ and will not be rebuilt.

​
Ccache Parameters

​
use-ccache
boolean
Automatically install and use ccache during package builds
​
ccache-dir
string
The directory used to store ccache outputs
​
ccache-config
string
The ccache configuration file path

​
License Checking

​
license-check-mode
string
default:"default"
Do additional validation of licenses after the build. Options: none, warn, error, default
​
license-check-name-file
string
File containing license names to check for
​
license-check-exception-file
string
File containing license exceptions
​
license-results-file
string
File to save the license check results to
​
license-summary-file
string
File to save the license check summary to

​
Advanced Parameters

​
rpmmacros-file
string
Optional file path to an rpmmacros file for rpmbuild to use
​
build-agent-program
string
Path to the build agent that will be invoked to build packages
​
timestamp-file
string
File that stores timestamps for this program
​
log-file
string
Path to the log file
​
log-level
string
Log level (panic, fatal, error, warn, info, debug, trace)

​
How It Works

  1. Load Graph: Reads the dependency graph from the grapher tool
  2. Initialize Build State: Determines which packages need to be built
  3. Start Workers: Spawns the specified number of build worker processes
  4. Schedule Builds: Identifies leaf nodes (packages with all dependencies met) and assigns them to workers
  5. Monitor Progress: Tracks build results and updates the graph
  6. Handle Dependencies: As packages complete, unblocks dependent packages
  7. Cascade Updates: Optionally rebuilds packages whose dependencies changed
  8. Finalize: Saves the final graph and build state CSV

​
Build Agents

​
Chroot Agent

The standard build agent that uses chroot environments (via pkgworker):
  • Isolated build environments
  • Full RPM build support
  • Suitable for production builds

​
Test Agent

A simplified agent for testing the scheduler:
  • Mocks package builds
  • Useful for debugging scheduler logic
  • Not for production use

​
Example Usage

​
Standard Full Build

scheduler \
  --input build/pkg_artifacts/graph.dot \
  --output build/pkg_artifacts/built_graph.dot \
  --output-build-state-csv-file build/pkg_artifacts/build_state.csv \
  --work-dir build/worker_chroots \
  --worker-tar toolkit/worker_chroot.tar.gz \
  --repo-file build/local.repo \
  --rpm-dir build/rpms \
  --toolchain-rpms-dir out/toolchain_rpms \
  --srpm-dir build/srpms \
  --cache-dir build/rpm_cache \
  --build-logs-dir build/logs \
  --dist-tag .azl3 \
  --distro-release-version 3.0 \
  --distro-build-number 20240101 \
  --build-agent chroot \
  --workers 8 \
  --log-level info

​
Build Specific Packages

scheduler \
  --input build/pkg_artifacts/graph.dot \
  --output build/pkg_artifacts/built_graph.dot \
  --packages "nginx python3 openssl" \
  [other required options...]

​
Build with Tests

scheduler \
  --input build/pkg_artifacts/graph.dot \
  --output build/pkg_artifacts/built_graph.dot \
  --tests "python3 openssl" \
  --check-attempts 3 \
  [other required options...]

​
Rebuild Specific Packages

scheduler \
  --input build/pkg_artifacts/graph.dot \
  --output build/pkg_artifacts/built_graph.dot \
  --rebuild-packages "kernel systemd" \
  --max-cascading-rebuilds 50 \
  [other required options...]

​
Development Build with Ccache

scheduler \
  --input build/pkg_artifacts/graph.dot \
  --output build/pkg_artifacts/built_graph.dot \
  --use-ccache \
  --ccache-dir /var/cache/ccache \
  --no-cleanup \
  --stop-on-failure \
  --build-attempts 3 \
  [other required options...]

​
Build from Image Config

scheduler \
  --input build/pkg_artifacts/graph.dot \
  --output build/pkg_artifacts/built_graph.dot \
  --image-config-file imageconfigs/core.json \
  --base-dir imageconfigs \
  [other required options...]

​
Build State CSV

The scheduler generates a CSV file with the following information for each package:
  • Package name and version
  • Build status (success, failed, cached, skipped)
  • Build time
  • Dependencies
  • Build attempt number
This file is useful for:
  • Tracking build progress
  • Analyzing build performance
  • Identifying problematic packages
  • CI/CD reporting

​
Worker Pool Management

The scheduler manages a pool of concurrent workers:
  • Automatic Scaling: Set --workers 0 to use all available CPU cores
  • Resource Control: Use --max-cpu to limit CPU usage per package
  • Timeout Protection: --timeout prevents runaway builds
  • Retry Logic: --build-attempts handles transient failures

​
Build Optimization

​
Caching

  • Use --no-cache to force rebuild all packages
  • Default behavior reuses already-built packages when possible
  • --optimize-with-cached-implicit enables aggressive graph pruning

​
Cascading Rebuilds

When a package is rebuilt, its dependents may also need rebuilding:
  • --max-cascading-rebuilds limits the cascade depth
  • Prevents excessive rebuilds in large dependency chains
  • Set to 0 to disable cascading rebuilds entirely

​
Toolchain Handling

  • --toolchain-manifest lists packages considered “prebuilt”
  • --allow-toolchain-rebuilds permits rebuilding toolchain packages
  • Toolchain packages are typically built separately and reused

​
Notes

  • The scheduler automatically handles build parallelism based on the dependency graph
  • Failed builds can be retried with --build-attempts
  • Use --stop-on-failure in CI/CD to fail fast
  • The --no-cleanup flag is useful for debugging build failures
  • License checking can be enforced to ensure compliance
  • Build logs for each package are saved in --build-logs-dir
  • The scheduler gracefully handles SIGINT/SIGTERM for clean shutdown

​
See Also

Build docs developers (and LLMs) love

Get started for freeTalk to us