Prompt Optimization

import gepa

trainset, valset, _ = gepa.examples.aime.init_dataset()

seed_prompt = {
    "system_prompt": "You are a helpful assistant. Answer the question. "
                     "Put your final answer in the format '### <answer>'"
}

result = gepa.optimize(
    seed_candidate=seed_prompt,
    trainset=trainset,
    valset=valset,
    task_lm="openai/gpt-4.1-mini",
    max_metric_calls=150,
    reflection_lm="openai/gpt-5",
)

print("Optimized prompt:", result.best_candidate['system_prompt'])

import dspy

optimizer = dspy.GEPA(
    metric=your_metric,
    max_metric_calls=150,
    reflection_lm="openai/gpt-5",
)
optimized_program = optimizer.compile(
    student=MyProgram(), 
    trainset=trainset, 
    valset=valset
)

from gepa.optimize_anything import (
    optimize_anything, GEPAConfig, EngineConfig, ReflectionConfig,
)

result = optimize_anything(
    seed_candidate={"system_prompt": "You are a helpful math assistant."},
    evaluator=evaluate_math_problem,
    dataset=trainset,
    valset=valset,
    objective="Optimize system prompt for AIME math competition problems.",
    config=GEPAConfig(
        engine=EngineConfig(max_metric_calls=350),
        reflection=ReflectionConfig(reflection_lm="openai/gpt-5"),
    ),
)

def evaluate(candidate: dict, example: dict) -> tuple[float, dict]:
    result = execute_task(candidate, example)
    return result.score, {
        "reasoning": result.chain_of_thought,
        "correct_answer": example.answer,
        "model_answer": result.answer,
        "error_type": classify_error(result, example)
    }

You will be given one math problem as plain text. Your job is to solve it 
correctly and return:

- reasoning: a concise, logically ordered solution that uses identities/structure 
  to avoid brute force, ends with a quick verification.
- answer: the final requested number/expression only (no extra words).

Domain-specific strategies:

1) Base-conversion/digit rearrangement:
- Translate positional notation correctly: in base b, (a b c)_b = a·b^2 + b·b + c
- Enforce digit ranges strictly (e.g., in base 9, digits ∈ {0,…,8})
- Use modular constraints to prune:
  • Mod 9 often collapses coefficients
  • Mod 8: 99 ≡ 3, 71 ≡ 7 ⇒ 3a ≡ 7b (mod 8)

2) Palindromes across bases:
- Bound the base length by magnitude (e.g., n < 1000 ⇒ octal has 3–4 digits)
- Characterize palindromes:
  • 3-digit octal: (A B A)_8 = 65A + 8B
  • 4-digit octal: (A B B A)_8 = 513A + 72B (with A ≥ 1)

[... continues with 6 more problem categories ...]

Your task is to generate a new search query optimized for the **second hop** 
of a multi-hop retrieval system.

Key Observations:
- First-hop documents often cover one entity or aspect in the question
- Remaining relevant documents often involve connected or higher-level concepts 
  mentioned in summary_1 but not explicitly asked in the original question
- The query should target these *missing*, but logically linked, documents

Practical Strategy:
- Read the summary carefully to spot references to bigger contexts or other 
  entities not covered in the first hop
- Ask yourself: "What entity or aspect does this summary hint at that could 
  answer the original question but was not found yet?"
- Formulate a precise, focused factual query targeting that entity

[...]

from pydantic_ai import Agent
from gepa import optimize

# Agent with optimized system prompt
agent = Agent(
    model='openai:gpt-4',
    system_prompt=optimized_prompt
)

import mlflow.genai

optimized_prompts = mlflow.genai.optimize_prompts(
    prompt_template="Your initial prompt",
    training_data=train_data,
    optimizer="gepa",
    max_iterations=150,
)

from opik.optimizers import GEPAOptimizer

optimizer = GEPAOptimizer(
    metric=your_metric,
    max_calls=150
)
optimized_agent = optimizer.optimize(agent, dataset)