Skip to main content

​
Backend Architecture Overview

Backends are Phase 2 of the AXON pipeline: they transform model-agnostic IR into LLM-specific prompt structures ready for execution. 
Model-Agnostic IR → [Backend Compiler] → LLM-Specific Prompts → [Runtime Executor]

​
Design Philosophy

​
Separation of Concerns

Phase 1 (Compiler) decides WHAT to do:
  • Which reasoning steps to execute
  • What anchors to enforce
  • What types to expect
Phase 2 (Backends) decides HOW to say it:
  • System prompt structure for Claude vs GPT-4
  • Tool declaration format (Anthropic vs OpenAI)
  • Chain-of-thought framing
  • Structured output schemas
Phase 3 (Runtime) executes and validates results.

​
Model Agnosticism at the Core

The IR layer has zero knowledge of any LLM provider: 
@dataclass(frozen=True)
class IRReason(IRNode):
    """Chain-of-thought directive — model-agnostic."""
    about: str = ""
    depth: int = 1
    show_work: bool = False
    ask: str = ""
Backends translate this into:
  • Claude: Extended thinking blocks with explicit work-showing
  • GPT-4: System message with chain-of-thought instructions
  • Gemini: markdown-formatted reasoning prompts

​
Backend Interface

​
BaseBackend Protocol

All backends implement the BaseBackend abstract class: 
class BaseBackend(ABC):
    """Abstract base class for all AXON model backends."""

    @property
    @abstractmethod
    def name(self) -> str:
        """The canonical name of this backend (e.g., 'anthropic')."""
        ...

    @abstractmethod
    def compile_step(
        self, step: IRNode, context: CompilationContext
    ) -> CompiledStep:
        """Compile a single IR step into a backend-specific prompt."""
        ...

    @abstractmethod
    def compile_system_prompt(
        self,
        persona: IRPersona | None,
        context: IRContext | None,
        anchors: list[IRAnchor],
    ) -> str:
        """Build the system prompt from persona, context, and anchors."""
        ...

    @abstractmethod
    def compile_tool_spec(self, tool: IRToolSpec) -> dict[str, Any]:
        """Compile a tool specification into the backend's native format."""
        ...

​
Compilation Output

CompiledProgram — Complete backend output: 
@dataclass
class CompiledProgram:
    backend_name: str = ""
    execution_units: list[CompiledExecutionUnit] = field(default_factory=list)
    metadata: dict[str, Any] = field(default_factory=dict)
CompiledExecutionUnit — One run statement: 
@dataclass
class CompiledExecutionUnit:
    flow_name: str = ""
    persona_name: str = ""
    context_name: str = ""
    system_prompt: str = ""
    steps: list[CompiledStep] = field(default_factory=list)
    tool_declarations: list[dict[str, Any]] = field(default_factory=list)
    anchor_instructions: list[str] = field(default_factory=list)
    active_anchors: list[dict[str, Any]] = field(default_factory=list)
CompiledStep — One cognitive step: 
@dataclass
class CompiledStep:
    step_name: str = ""
    system_prompt: str = ""
    user_prompt: str = ""
    tool_declarations: list[dict[str, Any]] = field(default_factory=list)
    output_schema: dict[str, Any] | None = None
    metadata: dict[str, Any] = field(default_factory=dict)

​
Compilation Flow

​
Entry Point

def compile_program(self, ir: IRProgram) -> CompiledProgram:
    # Build tool lookup
    tools = {tool.name: tool for tool in ir.tools}
    
    execution_units: list[CompiledExecutionUnit] = []
    
    for run in ir.runs:
        # Build compilation context
        ctx = CompilationContext(
            persona=run.resolved_persona,
            context=run.resolved_context,
            anchors=list(run.resolved_anchors),
            tools=tools,
            flow=run.resolved_flow,
        )
        
        # Phase 1: Compile system prompt (persona + anchors)
        system_prompt = self.compile_system_prompt(
            persona=run.resolved_persona,
            context=run.resolved_context,
            anchors=list(run.resolved_anchors),
        )
        
        # Phase 2: Compile each step
        compiled_steps: list[CompiledStep] = []
        for step in run.resolved_flow.steps:
            compiled = self.compile_step(step, ctx)
            compiled_steps.append(compiled)
        
        unit = CompiledExecutionUnit(
            flow_name=run.flow_name,
            system_prompt=system_prompt,
            steps=compiled_steps,
            # ...
        )
        execution_units.append(unit)
    
    return CompiledProgram(
        backend_name=self.name,
        execution_units=execution_units,
    )

​
Compilation Context

@dataclass
class CompilationContext:
    """Carries state through the step compilation process."""
    persona: IRPersona | None = None
    context: IRContext | None = None
    anchors: list[IRAnchor] = field(default_factory=list)
    tools: dict[str, IRToolSpec] = field(default_factory=dict)
    flow: IRFlow | None = None
    prior_step_names: list[str] = field(default_factory=list)
    effort: str = ""

​
Available Backends

​
Production-Ready

Anthropic (Claude)

Claude-optimized prompts with extended thinking

Google Gemini

Gemini API with system_instruction formatting

​
Stubs (Phase 2 Expansion)

OpenAI (GPT)

Chat Completions API format (planned)

Ollama (Local)

Local model adaptations (planned)

​
Key Differences Between Backends

​
System Prompt Structure

Anthropic: 
system_prompt = """
You are LegalExpert.
Your areas of expertise: contract law, IP.
Communication tone: precise.

[HARD CONSTRAINTS — THESE RULES ARE ABSOLUTE]
CONSTRAINT 1: NoHallucination
  → You MUST: cite all sources
  → You MUST NOT: hallucinate, speculate
"""
Gemini: 
system_instruction = """
Your identity is LegalExpert.
Expertise areas: contract law, IP.
Tone of communication: precise.

## Mandatory Constraints
The following rules are absolute.

### Constraint 1: NoHallucination
- **MUST**: cite all sources
- **MUST NOT**: hallucinate, speculate
"""

​
Tool Declaration Format

Anthropic (Messages API): 
{
    "name": "WebSearch",
    "description": "External tool: WebSearch. Provider: brave.",
    "input_schema": {
        "type": "object",
        "properties": {
            "query": {"type": "string", "description": "..."}
        },
        "required": ["query"]
    }
}
Gemini (FunctionDeclaration): 
{
    "name": "WebSearch",
    "description": "Tool: WebSearch. Provider: brave.",
    "parameters": {
        "type": "OBJECT",
        "properties": {
            "query": {"type": "STRING", "description": "..."}
        },
        "required": ["query"]
    }
}
Note: Gemini uses UPPERCASE type names.

​
Structured Output

Anthropic: Uses JSON schema in content blocks Gemini: Uses response_schema parameter OpenAI: Uses response_format with JSON mode

​
Anchor Enforcement

Backends inject anchor constraints into the system prompt as hard constraints: 
def compile_anchor_instruction(self, anchor: IRAnchor) -> str:
    """Compile a single anchor into a natural-language enforcement instruction."""
    parts: list[str] = [f"[CONSTRAINT: {anchor.name}]"]
    
    if anchor.require:
        parts.append(f"  REQUIRE: {anchor.require}")
    if anchor.reject:
        parts.append(f"  REJECT: {', '.join(anchor.reject)}")
    if anchor.confidence_floor is not None:
        parts.append(f"  CONFIDENCE FLOOR: {anchor.confidence_floor}")
    if anchor.unknown_response:
        parts.append(f"  WHEN UNCERTAIN: \"{anchor.unknown_response}\"")
    
    return "\n".join(parts)
The runtime then validates responses against these anchors (see Runtime Validators).

​
Adding a New Backend

​
1. Implement BaseBackend

from axon.backends.base_backend import BaseBackend, CompiledStep, CompilationContext

class MyBackend(BaseBackend):
    @property
    def name(self) -> str:
        return "my_backend"
    
    def compile_step(self, step: IRNode, context: CompilationContext) -> CompiledStep:
        # Transform IR step into your model's prompt format
        ...
    
    def compile_system_prompt(
        self, persona, context, anchors
    ) -> str:
        # Build system-level instructions
        ...
    
    def compile_tool_spec(self, tool: IRToolSpec) -> dict:
        # Convert IRToolSpec to your model's tool format
        ...

​
2. Register Backend

# In axon/backends/__init__.py
from .my_backend import MyBackend

BACKENDS = {
    "anthropic": AnthropicBackend,
    "gemini": GeminiBackend,
    "my_backend": MyBackend,
}

​
3. Test with Mock Client

from axon.runtime.executor import Executor
from tests.mocks import MockModelClient

backend = MyBackend()
compiled = backend.compile_program(ir_program)

client = MockModelClient()
executor = Executor(client=client)
result = await executor.execute(compiled)

​
Next Steps

Anthropic Backend

Deep dive into Claude-specific compilation

Runtime Executor

See how compiled programs are executed

Build docs developers (and LLMs) love

Get started for freeTalk to us