Skip to main content

Overview

The AI Game Design Agent Team is a collaborative game design system powered by AG2 (formerly AutoGen)‘s AI Agent framework. This application generates comprehensive game concepts through the coordination of multiple specialized AI agents, each focusing on different aspects of game design based on user inputs such as game type, target audience, art style, and technical requirements. Built using AG2’s swarm feature with the initiate_swarm_chat() method.

Tutorial Available

Follow our complete step-by-step tutorial to build this from scratch

Architecture

Swarm Orchestration Pattern

The Game Design Team uses AG2’s swarm pattern for coordinated agent collaboration:

Agent Roles

Specialization: Narrative design and world-buildingResponsibilities:
  • Create compelling narratives aligned with game type
  • Design memorable characters with clear motivations
  • Develop game world, history, culture, and locations
  • Plan story progression and major plot points
  • Integrate narrative with mood/atmosphere
  • Support core gameplay mechanics through story
Outputs:
  • Character development and arcs
  • Plot structure
  • Dialogue writing approach
  • Lore and world-building
Specialization: Game mechanics and systems designResponsibilities:
  • Design core gameplay loops
  • Create progression systems (skills, abilities)
  • Define player interactions and controls
  • Balance gameplay for target audience
  • Design multiplayer interactions if applicable
  • Specify game modes and difficulty settings
Constraints Considered:
  • Budget limitations
  • Development time
  • Platform capabilities
Specialization: Art direction and audio designResponsibilities:
  • Define visual style guide
  • Design character and environment aesthetics
  • Plan visual effects and animations
  • Create audio direction (music, SFX, ambient)
  • Consider platform technical constraints
  • Align visuals with mood/atmosphere
Deliverables:
  • UI/UX design direction
  • Character art style
  • Environment art approach
  • Sound design framework
Specialization: Technical architecture and implementationResponsibilities:
  • Recommend game engine and tools
  • Define technical requirements per platform
  • Plan development pipeline and workflow
  • Identify technical challenges and solutions
  • Estimate resource requirements
  • Plan scalability and optimization
  • Design multiplayer infrastructure if needed
Considerations:
  • Budget constraints
  • Team size
  • Timeline
  • Platform requirements
Role: Orchestration and integrationResponsibilities:
  • Coordinate between specialized agents
  • Ensure cohesive integration of different aspects
  • Maintain consistency across designs
  • Manage agent handoffs
  • Aggregate final game concept

Implementation

from autogen import (
    SwarmAgent,
    SwarmResult,
    initiate_swarm_chat,
    OpenAIWrapper,
    AFTER_WORK,
    UPDATE_SYSTEM_MESSAGE
)

# LLM Configuration
llm_config = {
    "config_list": [{
        "model": "gpt-4o-mini",
        "api_key": api_key
    }]
}

# Context variables for agent communication
context_variables = {
    "story": None,
    "gameplay": None,
    "visuals": None,
    "tech": None,
}

# Story Agent
story_agent = SwarmAgent(
    "story_agent",
    llm_config=llm_config,
    functions=update_story_overview,
    update_agent_state_before_reply=[state_update]
)

# Gameplay Agent
gameplay_agent = SwarmAgent(
    "gameplay_agent",
    llm_config=llm_config,
    functions=update_gameplay_overview,
    update_agent_state_before_reply=[state_update]
)

# Visuals Agent
visuals_agent = SwarmAgent(
    "visuals_agent",
    llm_config=llm_config,
    functions=update_visuals_overview,
    update_agent_state_before_reply=[state_update]
)

# Tech Agent
tech_agent = SwarmAgent(
    "tech_agent",
    llm_config=llm_config,
    functions=update_tech_overview,
    update_agent_state_before_reply=[state_update]
)

# Register handoffs (circular flow)
story_agent.register_hand_off(AFTER_WORK(gameplay_agent))
gameplay_agent.register_hand_off(AFTER_WORK(visuals_agent))
visuals_agent.register_hand_off(AFTER_WORK(tech_agent))
tech_agent.register_hand_off(AFTER_WORK(story_agent))

Swarm Coordination Flow

Two-Phase Agent Execution

Each agent goes through two phases:
1

Summary Phase

Agent provides 2-3 sentence summary of their design ideas
# Agent forced to call update function
agent.llm_config['tool_choice'] = {
    "type": "function",
    "function": {"name": f"update_{agent_type}_overview"}
}
2

Context Sharing

Summary stored in context_variables and shared with other agents
context_variables["story"] = story_summary
# All subsequent agents can see this summary
3

Handoff

Agent hands off to next agent in sequence
return SwarmResult(
    agent="gameplay_agent",
    context_variables=context_variables
)
4

Detailed Phase

After all agents provide summaries, each generates detailed design
# Tools removed, agent generates full response
agent.llm_config["tools"] = None
system_prompt += f"Write the {agent_type} part of the report."

Context Accumulation

# Each agent sees summaries from previous agents
system_prompt += f"\n\nContext for you to refer to:"

for k, v in agent._context_variables.items():
    if v is not None:
        system_prompt += f"\n{k.capitalize()} Summary:\n{v}"

# Example when Visuals Agent executes:
# - Has Story summary
# - Has Gameplay summary
# - Can design visuals that complement both

Key Features

Specialized Expertise

Four specialized agents each bring domain expertise:
  • Story: Narrative design
  • Gameplay: Mechanics and systems
  • Visuals: Art and audio
  • Tech: Architecture and tools

Comprehensive Output

Complete game design documentation:
  • Narrative and world-building
  • Gameplay mechanics
  • Visual and audio direction
  • Technical specifications
  • Development roadmap

Customizable Input

Extensive input parameters:
  • Game type and audience
  • Art style and platforms
  • Budget and timeline
  • Core mechanics
  • Mood and atmosphere

Interactive Results

User-friendly presentation:
  • Quick summaries in sidebar
  • Detailed expandable sections
  • Organized by design aspect
  • Easy navigation

Input Parameters

# Essential game definition
background_vibe = "Epic fantasy with dragons"
game_type = "RPG"  # RPG, Action, Adventure, etc.
game_goal = "Save the kingdom from eternal winter"
target_audience = "Young Adults (18-25)"
player_perspective = "Third Person"
multiplayer = "Online Multiplayer"

Installation

1

Clone Repository

git clone https://github.com/Shubhamsaboo/awesome-llm-apps.git
cd advanced_ai_agents/multi_agent_apps/agent_teams/ai_game_design_agent_team
2

Install Dependencies

pip install -r requirements.txt
Required packages:
  • streamlit==1.41.1
  • autogen
3

Set OpenAI API Key

You’ll input your OpenAI API key in the Streamlit sidebarGet your key from platform.openai.com
4

Run Application

streamlit run game_design_agent_team.py

Usage Example

Input:
  • Background: “Epic fantasy with dragons”
  • Type: RPG
  • Audience: Young Adults (18-25)
  • Art Style: Stylized
  • Platforms: PC, PlayStation, Xbox
  • Budget: $100,000
  • Time: 18 months
Story Design Output:World: The kingdom of Aethermoor, where dragons and humans once lived in harmonyProtagonist: Young dragon rider discovering ancient conspiracyNarrative Arc:
  • Act 1: Discovery and training
  • Act 2: Unraveling the curse
  • Act 3: Final confrontation with dark force
Key Characters:
  • Mentor figure (tragic past)
  • Rival rider (becomes ally)
  • Ancient dragon (wisdom keeper)
Gameplay Design Output:Core Loop:
  1. Explore regions
  2. Complete quests and challenges
  3. Strengthen dragon bond
  4. Unlock new abilities
Progression:
  • Character leveling (1-50)
  • Dragon evolution stages
  • Skill trees for combat and magic
  • Equipment crafting system
Combat:
  • Ground combat (swordplay and magic)
  • Aerial combat (dragon riding)
  • Combo system
  • Boss encounters
Visual Design Output:Art Direction:
  • Stylized realism with painterly textures
  • Rich color palette (warm kingdoms, cool mountains)
  • Distinctive dragon designs per region
UI/UX:
  • Minimalist HUD
  • Dragon bond indicator
  • Radial menu for abilities
Audio:
  • Orchestral score with regional themes
  • Dynamic combat music
  • Dragon vocalizations
Technical Design Output:Engine: Unreal Engine 5Key Technologies:
  • Nanite for environment detail
  • Niagara for dragon effects
  • MetaSounds for audio
Platform Requirements:
  • PC: Medium-High settings, 16GB RAM
  • Consoles: Optimized 60fps mode
Development Pipeline:
  • Months 1-3: Prototyping and core mechanics
  • Months 4-9: Content creation
  • Months 10-15: Polish and balancing
  • Months 16-18: Testing and optimization

Advanced Features

Dynamic System Message Updates

def update_system_message_func(agent: SwarmAgent, messages) -> str:
    """
    Dynamically update agent instructions based on phase.
    """
    system_prompt = system_messages[agent.name]
    current_gen = agent.name.split("_")[0]
    
    # Phase 1: Generate summary
    if agent._context_variables.get(current_gen) is None:
        system_prompt += f"Call the update function to provide "
                        f"a 2-3 sentence summary of your {current_gen} ideas."
        # Force function calling
        agent.llm_config['tool_choice'] = {
            "type": "function",
            "function": {"name": f"update_{current_gen}_overview"}
        }
    # Phase 2: Generate detailed design
    else:
        # Remove tools
        agent.llm_config["tools"] = None
        system_prompt += f"Write the {current_gen} part of the report."
        # Clear message history for cost savings
        k = list(agent._oai_messages.keys())[-1]
        agent._oai_messages[k] = agent._oai_messages[k][:1]
    
    # Add context from other agents
    system_prompt += "\n\nContext from other agents:"
    for k, v in agent._context_variables.items():
        if v is not None:
            system_prompt += f"\n{k.capitalize()}: {v}"
    
    agent.client = OpenAIWrapper(**agent.llm_config)
    return system_prompt

Circular Handoff Pattern

# Agents hand off in circular fashion
story_agent.register_hand_off(AFTER_WORK(gameplay_agent))
gameplay_agent.register_hand_off(AFTER_WORK(visuals_agent))
visuals_agent.register_hand_off(AFTER_WORK(tech_agent))
tech_agent.register_hand_off(AFTER_WORK(story_agent))

# Flow:
# Story → Gameplay → Visuals → Tech → Story (detailed) → etc.

Best Practices

Specific is Better:
  • Provide detailed background vibe
  • List specific inspirations
  • Be clear about unique features
  • Specify technical constraints
Good Example:
Background: "Cyberpunk noir detective story in 
            rain-soaked neon city"
Inspiration: "Blade Runner, Disco Elysium, Deus Ex"
Unique: "Psychological interrogation system"
Consider Constraints:
  • Match scope to budget/time
  • Align complexity with team size
  • Choose appropriate platforms
Example:
  • $10K budget → Pixel art, single platform, focused scope
  • $100K budget → Stylized 3D, multiple platforms, moderate scope
  • $1M+ budget → High-fidelity, all platforms, AAA features
Review All Sections:
  • Check consistency across agents
  • Verify technical feasibility
  • Assess scope vs. constraints
  • Look for creative synergies
Iterate:
  • Run multiple times with variations
  • Adjust parameters based on output
  • Combine best ideas from runs
Optimize Usage:
  • Use gpt-4o-mini (default, cheaper)
  • Limit max_rounds to necessary amount
  • Clear message history between phases
  • Cache results for similar queries
Typical Costs:
  • One complete run: ~$0.05-0.15
  • High detail setting: ~$0.20-0.30

Technical Insights

Why Swarm Pattern?

Coordination:
  • Automatic agent handoffs
  • Context sharing built-in
  • Circular workflows supported
Flexibility:
  • Dynamic system messages
  • Phase-based execution
  • Function-based transitions
Efficiency:
  • Parallel processing potential
  • Message history management
  • Cost optimization

Mental Wellbeing Agent

Swarm pattern for mental health support

Legal Agent Team

Document analysis with team coordination

Finance Agent Team

Financial analysis with specialized agents

Build docs developers (and LLMs) love