​

The JIT Compiler

​

Architecture Overview

• Tier 1: Adaptive interpreter with specialized bytecode
• Tier 2: JIT compiler for hot code paths

​

When JIT Compilation Occurs

1. JUMP_BACKWARD has an inline cache counter
2. Counter decrements on each execution
3. When counter reaches zero (threshold exceeded):
   • Call _PyOptimizer_Optimize() in Python/optimizer.c
   • Pass current frame and instruction pointer
   • Create optimized executor for the trace

​

Backoff Counter

​

Executors

​

Executor Storage

struct PyCodeObject {
    // ...
    _PyExecutorArray *co_executors;  // Array of executors
    // ...
};

​

ENTER_EXECUTOR Instruction

1. JUMP_BACKWARD replaced with ENTER_EXECUTOR
2. oparg contains index into co_executors array
3. Subsequent iterations use the executor directly

​

Executor Exits

• Return to tier 1 interpreter
• Transfer to another executor

​

The Micro-op Optimizer

​

Trace Translation

1. Identifies trace - Sequence of bytecode starting from hot jump
2. Expands to micro-ops - Each bytecode → sequence of micro-ops
3. Optimizes - Apply optimization passes
4. Creates executor - Instance of _PyUOpExecutor_Type

​

Micro-ops (uops)

# Bytecode instruction
LOAD_ATTR  5

# Expands to micro-ops (example):
GUARD_TYPE_VERSION
LOAD_ATTR_SLOT

​

Optimization Pass

• Dead code elimination
• Redundant guard removal
• Constant propagation
• Type specialization

​

JIT Interpreter

​

Enabling

./configure --enable-experimental-jit=interpreter

​

Execution

1. Jump to tier2_dispatch: label in Python/ceval.c
2. Loop executes micro-ops via switch statement
3. Switch cases in Python/executor_cases.c.h
4. Generated by Tools/cases_generator/tier2_generator.py

​

Exit Instructions

• _EXIT_TRACE - Planned exit, return to tier 1
• _DEOPT - Deoptimization due to guard failure

​

Full JIT (Copy-and-Patch)

​

Enabling

./configure --enable-experimental-jit

​

Architecture

1. Pre-compiled stencils for each micro-op
2. Runtime patching fills in specific values
3. Efficient compilation without complex codegen

​

Stencil Generation

1. Read Python/executor_cases.c.h
2. For each micro-op, create .c file with template from Tools/jit/template.c
3. Compile with LLVM to produce object files
4. Extract machine code into jit_stencils.h

​

JIT Compilation

1. Allocate executable memory
2. For each micro-op:
   • Copy stencil code
   • Patch runtime values (constants, object pointers, etc.)
3. Set executor->jit_code to point to compiled function

​

JIT Function Signature

typedef _Py_CODEUNIT *(*jit_func)(
    _PyInterpreterFrame *frame,
    PyObject **stack_pointer,
    PyThreadState *tstate
);

​

Execution

1. Check if executor->jit_code is set
2. Call JIT function instead of tier 2 interpreter
3. Function returns next instruction pointer
4. Continue execution from returned location

​

Executor Invalidation

​

Executor List

struct _is {
    // ...
    _PyExecutorObject *executor_list_head;
    // ...
};

​

Invalidation Triggers

• Type modified (method added/removed)
• Global/builtin modified
• Module dict modified
• Code object modified

​

Invalidation Process

1. Iterate executor_list_head
2. Mark affected executors as invalid
3. Next ENTER_EXECUTOR will recompile or deoptimize

​

Example: JIT in Action

def hot_loop(n):
    total = 0
    for i in range(n):  # JUMP_BACKWARD here
        total += i
    return total

# First iterations: tier 1 interpreter
# Counter in JUMP_BACKWARD decrements

hot_loop(10)  # May still use tier 1

# After threshold iterations:
# - JUMP_BACKWARD triggers JIT compilation
# - Trace compiled to executor  
# - JUMP_BACKWARD replaced with ENTER_EXECUTOR

hot_loop(1000)  # Now uses JIT executor

​

Trace Example

GUARD_TYPE_VERSION  # total is int
GUARD_TYPE_VERSION  # i is int  
BINARY_OP_ADD_INT   # total += i
STORE_FAST 0        # store total
LOAD_FAST 1         # load i
LOAD_CONST 1        # load 1
BINARY_OP_ADD_INT   # i += 1
STORE_FAST 1        # store i
LOAD_FAST 1         # load i
LOAD_FAST 2         # load n
COMPARE_OP_INT <    # i < n
POP_JUMP_IF_TRUE -X # back to start or exit

​

Performance Benefits

​

Reduced Dispatch Overhead

• Tier 1: Decode + dispatch for every instruction
• JIT: Direct machine code execution

​

Better Register Allocation

• Tier 1: Stack-based with memory ops
• JIT: Can keep values in registers across instructions

​

Inlining Opportunities

• Micro-ops can inline small operations
• Eliminates call overhead

​

Typical Speedup

• 2-4x faster than tier 1 interpreter
• Still slower than compiled languages (C, Rust)
• Best for tight loops with predictable types

​

Configuration

​

Build Options

# JIT interpreter only (debugging)
./configure --enable-experimental-jit=interpreter

# Full JIT (copy-and-patch)
./configure --enable-experimental-jit

# No JIT (default)
./configure

​

Runtime Control

​

Debugging JIT

​

JIT Stats

./configure --enable-experimental-jit=interpreter --enable-pystats
make

import sys
sys._stats_on()
# Run code
sys._stats_off() 
sys._stats_dump()

​

Disabling JIT

./configure
make

​

Implementation Status

​

Supported Platforms

• x86-64 (Linux, macOS, Windows)
• ARM64 (Linux, macOS)

​

Limitations

• Not all bytecode instructions have micro-op translations
• Some operations force deoptimization to tier 1
• Exception handling may deoptimize

​

Further Reading

​

Videos

​

Papers

​

Related Topics

• Bytecode Interpreter - Tier 1 execution and specialization
• Code Objects - Where executors are stored
• Compiler Design - Bytecode generation