Overview
The Think Thoroughly Agent performs comprehensive market analysis by breaking down questions into hypothetical and conditional scenarios, researching each scenario independently, and synthesizing findings into a final prediction. It uses CrewAI for orchestrating multiple research and prediction tasks.
Base Class: ThinkThoroughlyBase
Abstract base class for Think Thoroughly agent variants.
class ThinkThoroughlyBase(ABC):
identifier: AgentIdentifier
model: KnownModelName
model_for_generate_prediction_for_one_outcome: KnownModelName
def __init__(self, enable_langfuse: bool, memory: bool = True) -> None
Configuration Properties
Unique identifier for the agent variant
Model used for scenario generation and final decision
model_for_generate_prediction_for_one_outcome
Model used for individual scenario predictions
Enable Langfuse observability
Enable long-term memory storage
Core Components
Handles Omen market data queries
Vector database for finding correlated markets
_long_term_memory
LongTermMemoryTableHandler | None
Stores predictions and scenarios for future reference
Methods
answer_binary_market
def answer_binary_market(
self,
question: str,
n_iterations: int = 1,
created_time: DatetimeUTC | None = None,
) -> ProbabilisticAnswer | None
The market question to predict on
Number of iterative refinement passes. Higher values allow predictions to be adjusted based on previous scenario results.
When the market was created (used for time-based analysis)
return
ProbabilisticAnswer | None
Final probabilistic prediction synthesized from all scenarios
get_hypohetical_scenarios
def get_hypohetical_scenarios(self, question: str) -> Scenarios
List of 5 hypothetical scenarios, plus the original question
get_required_conditions
def get_required_conditions(self, question: str) -> Scenarios
List of 3 conditional scenarios
generate_prediction_for_one_outcome
@staticmethod
def generate_prediction_for_one_outcome(
unique_id: UUID,
model: KnownModelName,
scenario: str,
original_question: str,
previous_scenarios_and_answers: list[tuple[str, AnswerWithScenario]] | None = None,
) -> AnswerWithScenario | None
generate_final_decision
def generate_final_decision(
self,
question: str,
scenarios_with_probabilities: list[tuple[str, AnswerWithScenario]],
created_time: DatetimeUTC | None,
research_report: str | None = None,
) -> ProbabilisticAnswer
def get_correlated_markets(self, question: str) -> list[CorrelatedMarketInput]
Implementation Variants
ThinkThoroughlyWithItsOwnResearch
Uses CrewAI to perform independent web research for each scenario.
class ThinkThoroughlyWithItsOwnResearch(ThinkThoroughlyBase):
identifier = THINK_THOROUGHLY
model = "openai:gpt-4-turbo-2024-04-09"
model_for_generate_prediction_for_one_outcome = "openai:gpt-4-turbo-2024-04-09"
Research Process
For each scenario:
- Research Task: Uses Tavily search to gather web evidence
- Prediction Task: Analyzes research to generate probability
- Sequential Processing: Research output feeds into prediction
task_research_one_outcome = Task(
description=RESEARCH_OUTCOME_PROMPT,
agent=researcher,
expected_output=RESEARCH_OUTCOME_OUTPUT,
)
task_create_probability_for_one_outcome = Task(
description=PROBABILITY_FOR_ONE_OUTCOME_PROMPT,
expected_output=PROBABILITY_CLASS_OUTPUT,
agent=predictor,
output_pydantic=ProbabilisticAnswer,
context=[task_research_one_outcome],
)
ThinkThoroughlyWithPredictionProphetResearch
Uses PredictionProphet library for research instead of CrewAI.
class ThinkThoroughlyWithPredictionProphetResearch(ThinkThoroughlyBase):
identifier = THINK_THOROUGHLY_PROPHET
model = "openai:gpt-4-turbo-2024-04-09"
model_for_generate_prediction_for_one_outcome = "openai:gpt-4o-2024-08-06"
Key Differences
- Uses
prophet_research() for web research
- Uses
prophet_make_prediction() for predictions
- No iterative refinement: Does not support
previous_scenarios_and_answers
- Sets
initial_subqueries_limit=0 (agent generates its own subqueries)
Deployable Agents
DeployableThinkThoroughlyAgent
Production agent using internal research.
class DeployableThinkThoroughlyAgent(DeployableThinkThoroughlyAgentBase):
agent_class = ThinkThoroughlyWithItsOwnResearch
bet_on_n_markets_per_run = 1
Betting Strategy
def get_betting_strategy(self, market: AgentMarket) -> BettingStrategy:
return FullBinaryKellyBettingStrategy(
max_position_amount=get_maximum_possible_bet_amount(
min_=USD(1),
max_=USD(5),
trading_balance=market.get_trade_balance(APIKeys()),
),
max_price_impact=None, # No price impact limit
)
Usage
from prediction_market_agent.agents.think_thoroughly_agent.deploy import (
DeployableThinkThoroughlyAgent
)
from prediction_market_agent_tooling.markets.markets import MarketType
agent = DeployableThinkThoroughlyAgent(
enable_langfuse=True,
place_trades=True,
)
agent.deploy(market_type=MarketType.OMEN)
DeployableThinkThoroughlyProphetResearchAgent
Production agent using PredictionProphet research.
class DeployableThinkThoroughlyProphetResearchAgent(
DeployableThinkThoroughlyAgentBase
):
agent_class = ThinkThoroughlyWithPredictionProphetResearch
bet_on_n_markets_per_run = 1
Prediction Workflow
The Think Thoroughly agent follows a systematic 5-step process:
Step 1: Generate Scenarios
hypothetical_scenarios = self.get_hypohetical_scenarios(question)
conditional_scenarios = self.get_required_conditions(question)
- Hypothetical scenarios: 5 alternative phrasings/interpretations
- Conditional scenarios: 3 necessary conditions
- Total: ~8 scenarios to analyze
Step 2: Parallel Research
sub_predictions = par_generator(
items=[
(enable_langfuse, unique_id, model, scenario, question, scenarios_with_probs, generate_prediction_for_one_outcome)
for scenario in all_scenarios
],
func=process_scenario,
)
Step 3: Collect Predictions
scenarios_with_probs = []
for scenario, prediction in sub_predictions:
if prediction is None:
continue
scenarios_with_probs.append((scenario, prediction))
self.save_answer_to_long_term_memory(prediction)
correlated_markets = self.get_correlated_markets(question)
Step 5: Synthesize Final Decision
final_answer = self.generate_final_decision(
question,
scenarios_with_probs,
created_time=created_time,
research_report=research_report, # Only for Prophet variant
)
CrewAI Integration
Agent Roles
Research Analyst
Agent(
role="Research Analyst",
goal="Research and report on some future event, giving high quality and nuanced analysis",
backstory=f"Current date is {current_date}. You are a senior research analyst who is adept at researching and reporting on future events.",
tools=[tavily_search_tool],
)
Professional Gambler
Agent(
role="Professional Gambler",
goal="Predict, based on some research you are presented with, whether or not a given event will occur",
backstory=f"Current date is {current_date}. You are a professional gambler who is adept at predicting and betting on the outcomes of future events.",
)
Task Orchestration
crew = Crew(
agents=[researcher, predictor],
tasks=[task_research, task_predict],
process=Process.sequential,
)
output = crew.kickoff(inputs={"sentence": scenario})
Data Models
Scenarios
class Scenarios(BaseModel):
scenarios: list[str]
AnswerWithScenario
class AnswerWithScenario(BaseModel):
scenario: str
original_question: str
p_yes: float
confidence: float
reasoning: str
class CorrelatedMarketInput(BaseModel):
question_title: str
current_p_yes: float
Advanced Usage
Custom Number of Iterations
agent = DeployableThinkThoroughlyAgent()
agent.agent = agent.agent_class(enable_langfuse=True)
# Run 3 iterations for iterative refinement
answer = agent.agent.answer_binary_market(
question="Will AI achieve AGI by 2030?",
n_iterations=3,
)
Direct Scenario Analysis
from prediction_market_agent.agents.think_thoroughly_agent.think_thoroughly_agent import (
ThinkThoroughlyWithItsOwnResearch
)
agent = ThinkThoroughlyWithItsOwnResearch(enable_langfuse=False)
# Get scenario breakdowns
hypothetical = agent.get_hypohetical_scenarios(
"Will SpaceX land humans on Mars by 2030?"
)
conditional = agent.get_required_conditions(
"Will SpaceX land humans on Mars by 2030?"
)
print(f"Hypothetical scenarios: {hypothetical.scenarios}")
print(f"Required conditions: {conditional.scenarios}")
Local Testing
if __name__ == "__main__":
agent = DeployableThinkThoroughlyAgent(
place_trades=False,
store_predictions=False,
store_trades=False,
)
agent.deploy_local(
market_type=MarketType.OMEN,
sleep_time=540, # 9 minutes
run_time=180, # 3 minutes per run
)
Parallel Processing
Uses par_generator with multiprocessing for parallel scenario research:
- Faster than sequential processing
- Each scenario researched independently
- Handles failures gracefully (continues with successful predictions)
Memory Management
Saves all intermediate predictions to long-term memory:
self.save_answer_to_long_term_memory(answer_with_scenario)
- Learning from past predictions
- Debugging scenario-level reasoning
- Building up historical knowledge
Time Analysis
Considers temporal aspects:
event_date = get_event_date_from_question(question)
n_remaining_days = (event_date - utcnow()).days if event_date else "Unknown"
n_market_open_days = (utcnow() - created_time).days if created_time else "Unknown"
Required API Keys
For GPT-4 models used in research and prediction
For web search functionality
For vector similarity search of correlated markets
Source Location
prediction_market_agent/agents/think_thoroughly_agent/deploy.py
prediction_market_agent/agents/think_thoroughly_agent/think_thoroughly_agent.py