Builder System

Overview

The builder system is responsible for transforming your Dockerfile into a container image and optionally publishing it to a registry. Buildr uses a plugin-based architecture that allows different builder implementations.

Builder Selection

Builders are selected using a factory pattern based on the builder parameter in package options:

builder/__init__.py:4-7

def select(spec):
    if spec == 'docker':
        return DockerBuilder()
    raise Exception('Unknown spec {}'.format(spec))

Currently, only the docker builder is implemented, but the architecture supports adding alternative builders (e.g., buildah, kaniko, buildkit).

DockerBuilder

The DockerBuilder class uses the Docker Engine API to build and push images.

Implementation

docker_builder.py:9-62

class DockerBuilder:
    def __init__(self):
        self.docker_client = None

    def build(self, img, path='.'):
        if self.docker_client is None:
            self.docker_client = APIClient(version='auto')

        bld = self.docker_client.build(
            path=path,
            tag=img,
            encoding='utf-8'
        )

        for line in bld:
            self._process_stream(line)

    def publish(self, img):
        if self.docker_client is None:
            self.docker_client = APIClient(version='auto')

        # TODO: do we need to set tag?
        for line in self.docker_client.push(img, stream=True):
            self._process_stream(line)

    def _process_stream(self, line):
        ln = line.decode('utf-8').strip()

        # try to decode json
        try:
            ljson = json.loads(ln)

            if ljson.get('error'):
                msg = str(ljson.get('error', ljson))
                logger.error('Build failed: ' + msg)
                raise Exception('Image build failed: ' + msg)
            else:
                if ljson.get('stream'):
                    msg = 'Build output: {}'.format(ljson['stream'].strip())
                elif ljson.get('status'):
                    msg = 'Push output: {} {}'.format(
                        ljson['status'],
                        ljson.get('progress')
                    )
                elif ljson.get('aux'):
                    msg = 'Push finished: {}'.format(ljson.get('aux'))
                else:
                    msg = str(ljson)

                logger.debug(msg)

        except json.JSONDecodeError:
            logger.warning('JSON decode error: {}'.format(ln))

Lazy Client Initialization

docker_builder.py:10-11

def __init__(self):
    self.docker_client = None

The Docker client is initialized lazily (only when needed) rather than in __init__. This approach:

Avoids unnecessary Docker connections if builder isn’t used
Defers connection errors until actual build time
Allows the builder to be instantiated without Docker installed (useful for testing)

docker_builder.py:14-15

if self.docker_client is None:
    self.docker_client = APIClient(version='auto')

APIClient(version='auto') automatically negotiates the API version with the Docker daemon, ensuring compatibility.

Building Images

The build() method creates a Docker image from a Dockerfile:

docker_builder.py:13-24

def build(self, img, path='.'):
    if self.docker_client is None:
        self.docker_client = APIClient(version='auto')

    bld = self.docker_client.build(
        path=path,
        tag=img,
        encoding='utf-8'
    )

    for line in bld:
        self._process_stream(line)

Parameters

img

string

required

The full image tag (e.g., docker.io/myrepo/myapp:latest)

path

string

default:"."

Build context path containing the Dockerfile

Build Process

Initialize Client

Ensures Docker client is connected

Invoke Docker Build

Calls docker_client.build() with:

path: Build context directory
tag: Image name and tag
encoding: UTF-8 for text output

Stream Processing

Iterates through build output stream, processing each line for errors and logging

Example Build Output

# When you call:
builder.build('myrepo/myapp:latest', '.')

# Docker API returns streaming JSON:
{"stream": "Step 1/4 : FROM python:3-alpine"}
{"stream": " ---> abc123def456"}
{"stream": "Step 2/4 : COPY ./ /app/"}
{"stream": " ---> Using cache"}
{"stream": "Step 3/4 : RUN pip install..."}
{"stream": "Successfully built def456abc789"}
{"stream": "Successfully tagged myrepo/myapp:latest"}

Publishing Images

The publish() method pushes the built image to a container registry:

docker_builder.py:26-32

def publish(self, img):
    if self.docker_client is None:
        self.docker_client = APIClient(version='auto')

    # TODO: do we need to set tag?
    for line in self.docker_client.push(img, stream=True):
        self._process_stream(line)

Usage
Authentication

@Containerize(
    package={
        'name': 'myapp',
        'repository': 'docker.io/myrepo',
        'publish': True  # Enable publishing
    }
)
def main():
    print('This will be pushed to the registry!')

Publishing requires Docker authentication. Before running, ensure you’re logged in:

docker login docker.io
# or for other registries:
docker login gcr.io
docker login registry.example.com

The builder uses your existing Docker credentials from ~/.docker/config.json.

Push Stream Format

{"status": "The push refers to repository [docker.io/myrepo/myapp]"}
{"status": "Preparing", "progressDetail": {}, "id": "abc123"}
{"status": "Pushing", "progressDetail": {"current": 1024, "total": 4096}, "progress": "[==>  ] 1.024kB/4.096kB", "id": "abc123"}
{"status": "Pushed", "progressDetail": {}, "id": "abc123"}
{"aux": {"Tag": "latest", "Digest": "sha256:def456...", "Size": 12345}}

Stream Processing

The _process_stream() method handles Docker API output:

docker_builder.py:34-61

def _process_stream(self, line):
    ln = line.decode('utf-8').strip()

    # try to decode json
    try:
        ljson = json.loads(ln)

        if ljson.get('error'):
            msg = str(ljson.get('error', ljson))
            logger.error('Build failed: ' + msg)
            raise Exception('Image build failed: ' + msg)
        else:
            if ljson.get('stream'):
                msg = 'Build output: {}'.format(ljson['stream'].strip())
            elif ljson.get('status'):
                msg = 'Push output: {} {}'.format(
                    ljson['status'],
                    ljson.get('progress')
                )
            elif ljson.get('aux'):
                msg = 'Push finished: {}'.format(ljson.get('aux'))
            else:
                msg = str(ljson)

            logger.debug(msg)

    except json.JSONDecodeError:
        logger.warning('JSON decode error: {}'.format(ln))

Processing Logic

Decode Bytes to String

Docker API returns bytes, decode to UTF-8 and strip whitespace

Parse JSON

Each line is a JSON object containing build/push status

Error Handling

If the JSON contains an error key, log and raise an exception to halt the build

Log Categorization

stream: Build step output (logged at debug level)
status: Push progress (logged with progress bar)
aux: Final push metadata (logged when complete)

Invalid JSON Handling

If JSON parsing fails, log a warning and continue (non-fatal)

Error Detection Example

When Docker encounters an error during build:

{
  "error": "Error: No such file or directory: requirements.txt",
  "errorDetail": {
    "message": "Error: No such file or directory: requirements.txt"
  }
}

The builder will:

Extract the error message
Log: Build failed: Error: No such file or directory: requirements.txt
Raise: Exception('Image build failed: ...')
Stop the containerization process

Using the Builder

The builder is automatically selected and invoked by the Containerize decorator:

containerize.py:58-72

# In Containerize.__init__:
self.builder = builder.select(self.package.builder)

# In Containerize.__call__:
write_dockerfile(self.package, exec_file)
self.builder.build(self.image)

if self.package.publish:
    self.builder.publish(self.image)

Complete Flow

from metaparticle_pkg import Containerize

@Containerize(
    package={
        'name': 'demo',
        'repository': 'myrepo',
        'builder': 'docker',  # Selects DockerBuilder
        'publish': True
    }
)
def main():
    print('Hello!')

if __name__ == '__main__':
    main()

Builder Selection

builder.select('docker') returns a DockerBuilder instance

Dockerfile Generation

write_dockerfile() creates ./Dockerfile

Image Build

builder.build('myrepo/demo:latest') builds the image from current directory

Image Push

builder.publish('myrepo/demo:latest') pushes to the registry (if publish: True)

Logging

The builder uses Python’s logging module:

docker_builder.py:6

logger = logging.getLogger(__name__)

Log levels:

Level	When	Example
`ERROR`	Build/push fails	`Build failed: No such file: Dockerfile`
`DEBUG`	Build/push progress	`Build output: Step 1/4 : FROM python:3`
`WARNING`	Invalid JSON	`JSON decode error: invalid json`

To see detailed build output, configure logging in your application:

import logging
logging.basicConfig(level=logging.DEBUG)

Docker API Client

Buildr uses the low-level docker.APIClient instead of the high-level docker.Client:

docker_builder.py:4

from docker import APIClient

Why APIClient?

Streaming Support

APIClient.build() returns a stream generator, allowing real-time build output processing

Fine-Grained Control

Direct access to Docker Engine API provides more control over build parameters

Error Handling

Stream-based approach allows catching and handling errors as they occur

Version Negotiation

version='auto' automatically adapts to the Docker daemon’s API version

Best Practices

Use .dockerignore

Exclude unnecessary files to speed up builds:

.dockerignore

__pycache__
*.pyc
.git
.venv
*.egg-info
.pytest_cache

This reduces the build context size sent to Docker daemon.

Layer Caching

Order Dockerfile commands from least to most frequently changing:

FROM python:3-alpine

# Dependencies change less often - cache this layer
COPY requirements.txt /app/
RUN pip install -r /app/requirements.txt

# Code changes frequently - separate layer
COPY . /app/

CMD python /app/main.py

Authentication

For private registries, authenticate before running:

docker login your-registry.example.com

Or use environment variables:

export DOCKER_CONFIG=/path/to/config

Error Handling

Wrap containerization in try/except for production:

try:
    containerized_app()
except Exception as e:
    logger.error(f'Containerization failed: {e}')
    # Fallback to local execution
    app()

Getting Started

Core Concepts

Guides

Examples

Builder System

Overview

Builder Selection

DockerBuilder

Implementation

Lazy Client Initialization

Building Images

Parameters

Build Process

Example Build Output

Publishing Images

Push Stream Format

Stream Processing

Processing Logic

Using the Builder

Complete Flow

Logging

Docker API Client

Why APIClient?

Streaming Support

Fine-Grained Control

Error Handling

Version Negotiation

Best Practices

Next Steps

Runners

Architecture

Build docs developers (and LLMs) love

Getting Started

Core Concepts

Guides

Examples

​Overview

​Builder Selection

​DockerBuilder

​Implementation

​Lazy Client Initialization

​Building Images

​Parameters

​Build Process

​Example Build Output

​Publishing Images

​Push Stream Format

​Stream Processing

​Processing Logic

​Using the Builder

​Complete Flow

​Logging

​Docker API Client

​Why APIClient?

Streaming Support

Fine-Grained Control

Error Handling

Version Negotiation

​Best Practices

​Next Steps

Runners

Architecture

Build docs developers (and LLMs) love

Overview

Builder Selection

DockerBuilder

Implementation

Lazy Client Initialization

Building Images

Parameters

Build Process

Example Build Output

Publishing Images

Push Stream Format

Stream Processing

Processing Logic

Using the Builder

Complete Flow

Logging

Docker API Client

Why APIClient?

Best Practices

Next Steps