Skip to main content

Overview

GEPA provides a comprehensive callback system for observing and instrumenting optimization runs. Callbacks are synchronous, observational (cannot modify state), and receive full GEPAState access for maximum flexibility. All callbacks implement the GEPACallback protocol and receive typed event objects containing relevant parameters.

GEPACallback Protocol

The GEPACallback protocol defines optional callback methods for different optimization events. Implement only the methods you need.
from gepa.core.callbacks import GEPACallback

class MyCallback:
    def on_optimization_start(self, event: OptimizationStartEvent) -> None:
        print(f"Starting optimization with {event['trainset_size']} training examples")

    def on_iteration_end(self, event: IterationEndEvent) -> None:
        status = 'accepted' if event['proposal_accepted'] else 'rejected'
        print(f"Iteration {event['iteration']}: {status}")

Using Callbacks

Pass callbacks to the optimize() function:
from gepa import optimize

result = optimize(
    seed_candidate={"instructions": "..."},
    trainset=data,
    callbacks=[MyCallback()],
    # ... other parameters
)

Callback Events

Each callback method receives a single event TypedDict containing all relevant parameters. This design allows easy extension without breaking changes.

Optimization Lifecycle

on_optimization_start

Called when optimization begins.
event
OptimizationStartEvent

on_optimization_end

Called when optimization completes.
event
OptimizationEndEvent

Iteration Events

on_iteration_start

Called at the start of each iteration.
event
IterationStartEvent

on_iteration_end

Called at the end of each iteration.
event
IterationEndEvent

Evaluation Events

on_evaluation_start

Called before evaluating a candidate.
event
EvaluationStartEvent

on_evaluation_end

Called after evaluating a candidate.
event
EvaluationEndEvent

on_evaluation_skipped

Called when an evaluation is skipped.
event
EvaluationSkippedEvent

on_valset_evaluated

Called after evaluating on the validation set.
event
ValsetEvaluatedEvent

Candidate Selection Events

on_candidate_selected

Called when a candidate is selected for mutation.
event
CandidateSelectedEvent

on_minibatch_sampled

Called when a training minibatch is sampled.
event
MinibatchSampledEvent

Proposal Events

on_reflective_dataset_built

Called after building the reflective dataset.
event
ReflectiveDatasetBuiltEvent

on_proposal_start

Called before proposing new instructions.
event
ProposalStartEvent

on_proposal_end

Called after proposing new instructions.
event
ProposalEndEvent

Acceptance Events

on_candidate_accepted

Called when a new candidate is accepted.
event
CandidateAcceptedEvent

on_candidate_rejected

Called when a candidate is rejected.
event
CandidateRejectedEvent

Merge Events

on_merge_attempted

Called when a merge is attempted.
event
MergeAttemptedEvent

on_merge_accepted

Called when a merge is accepted.
event
MergeAcceptedEvent

on_merge_rejected

Called when a merge is rejected.
event
MergeRejectedEvent

State Events

on_pareto_front_updated

Called when the Pareto front is updated.
event
ParetoFrontUpdatedEvent

on_state_saved

Called after state is saved to disk.
event
StateSavedEvent

on_budget_updated

Called when the evaluation budget is updated.
event
BudgetUpdatedEvent

Error Events

on_error

Called when an error occurs during optimization.
event
ErrorEvent

CompositeCallback

The CompositeCallback class allows you to register multiple callbacks that all receive events.
from gepa.core.callbacks import CompositeCallback

composite = CompositeCallback([callback1, callback2, callback3])

result = optimize(
    seed_candidate={"instructions": "..."},
    trainset=data,
    callbacks=[composite],
)
You can also add callbacks dynamically:
composite = CompositeCallback()
composite.add(my_callback)
composite.add(another_callback)

Example: Progress Tracking Callback

from gepa.core.callbacks import (
    OptimizationStartEvent,
    IterationEndEvent,
    OptimizationEndEvent,
)

class ProgressCallback:
    def __init__(self):
        self.best_score = float('-inf')
        self.iterations = 0
    
    def on_optimization_start(self, event: OptimizationStartEvent) -> None:
        print(f"Starting optimization with:")
        print(f"  Training examples: {event['trainset_size']}")
        print(f"  Validation examples: {event['valset_size']}")
    
    def on_iteration_end(self, event: IterationEndEvent) -> None:
        self.iterations += 1
        if event['proposal_accepted']:
            print(f"✓ Iteration {event['iteration']}: New candidate accepted")
        else:
            print(f"✗ Iteration {event['iteration']}: Candidate rejected")
    
    def on_optimization_end(self, event: OptimizationEndEvent) -> None:
        print(f"\nOptimization complete after {event['total_iterations']} iterations")
        print(f"Best candidate: {event['best_candidate_idx']}")
        print(f"Total evaluations: {event['total_metric_calls']}")

Source Reference

The callback system is implemented in src/gepa/core/callbacks.py.

Build docs developers (and LLMs) love