Skip to main content
The game generator module parses AI responses containing structured game commands and executes them to build scenes. It translates natural language descriptions into actual editor actions.

​
Overview

When the AI assistant receives a user prompt like “Create a platformer with 3 platforms”, it returns structured JSON commands. The game generator parses these commands and executes them by calling the appropriate Zustand store actions.

​
Command types

​
GameCommand

interface GameCommand {
  type: 'add_object' | 'add_sprite' | 'set_transform' | 'set_material' | 'enable_web3' | 'message';
  [key: string]: any;
}
​
type
string
required
The command type to execute
Command types:
  • add_object - Add a 3D object to the scene
  • add_sprite - Add a 2D sprite to the scene
  • set_transform - Update position/rotation/scale of an object
  • set_material - Update material properties of an object
  • enable_web3 - Enable Web3 features
  • message - Text message to display to user

​
Functions

​
parseAIResponse

Extract JSON command array from AI response text. Looks for code blocks with json language tag. 
function parseAIResponse(text: string): {
  commands: GameCommand[];
  plainText: string;
}
​
text
string
required
The raw AI response text containing JSON commands
​
commands
GameCommand[]
Array of parsed commands (excluding message-only commands)
​
plainText
string
The response text with JSON blocks removed
Example: 
const aiResponse = `I'll create 3 platforms for you.

\`\`\`json
[
  {
    "type": "add_object",
    "objectType": "box",
    "name": "Platform 1",
    "position": { "x": 0, "y": 0, "z": 0 },
    "scale": { "x": 4, "y": 0.2, "z": 1 }
  }
]
\`\`\``;

const { commands, plainText } = parseAIResponse(aiResponse);
// commands: [{ type: 'add_object', objectType: 'box', ... }]
// plainText: "I'll create 3 platforms for you."

​
execute3DCommands

Execute 3D game commands by calling editor store actions. 
function execute3DCommands(commands: GameCommand[]): string[]
​
commands
GameCommand[]
required
Array of commands to execute
Returns: Array of log messages describing what was executed Supported command types:
  • add_object - Creates 3D objects (box, sphere, cylinder, plane, cone, torus, lights, camera)
  • set_transform - Updates position, rotation, or scale
  • set_material - Updates material properties (color, roughness, metalness)
  • enable_web3 - Toggles Web3 mode
Example: 
const commands = [
  {
    type: 'add_object',
    objectType: 'box',
    name: 'Player',
    position: { x: 0, y: 1, z: 0 },
    material: { color: '#ff0000' }
  }
];

const log = execute3DCommands(commands);
// log: ['✅ Added box: "Player"']

​
execute2DCommands

Execute 2D game commands by calling 2D editor store actions. 
function execute2DCommands(commands: GameCommand[]): string[]
​
commands
GameCommand[]
required
Array of 2D commands to execute
Returns: Array of log messages Supported command type:
  • add_sprite - Creates 2D sprites with position, size, color, and optional emoji/text
Example: 
const commands = [
  {
    type: 'add_sprite',
    name: 'Player',
    x: 400,
    y: 300,
    width: 32,
    height: 32,
    fillColor: '#00ff00',
    emoji: '😊'
  }
];

const log = execute2DCommands(commands);
// log: ['✅ Added sprite: "Player"']

​
executeCommands

Auto-detect command types and execute both 2D and 3D commands. 
function executeCommands(
  commands: GameCommand[],
  editorMode: '2d' | '3d' = '3d'
): string[]
​
commands
GameCommand[]
required
Array of mixed 2D/3D commands
​
editorMode
'2d' | '3d'
The current editor mode (defaults to ‘3d’)
Returns: Combined log messages from all executed commands

​
Command schemas

​
add_object command

{
  "type": "add_object",
  "objectType": "box" | "sphere" | "cylinder" | "plane" | "cone" | "torus" | "pointLight" | "directionalLight" | "ambientLight" | "camera",
  "name": "Object Name",
  "position": { "x": 0, "y": 0, "z": 0 },
  "rotation": { "x": 0, "y": 0, "z": 0 },
  "scale": { "x": 1, "y": 1, "z": 1 },
  "material": {
    "color": "#6366f1",
    "roughness": 0.4,
    "metalness": 0.1
  }
}

​
add_sprite command

{
  "type": "add_sprite",
  "name": "Sprite Name",
  "spriteType": "sprite" | "shape" | "text" | "tilemap",
  "x": 400,
  "y": 300,
  "width": 64,
  "height": 64,
  "fillColor": "#6366f1",
  "strokeColor": "#000000",
  "emoji": "🎮",
  "shapeType": "rect" | "circle" | "line" | "polygon",
  "text": "Hello",
  "fontSize": 24,
  "fontFamily": "Arial"
}

​
set_transform command

{
  "type": "set_transform",
  "name": "Object Name",
  "position": { "x": 0, "y": 0, "z": 0 },
  "rotation": { "x": 0, "y": 0, "z": 0 },
  "scale": { "x": 1, "y": 1, "z": 1 }
}

​
set_material command

{
  "type": "set_material",
  "name": "Object Name",
  "material": {
    "color": "#ff0000",
    "roughness": 0.8,
    "metalness": 0.0
  }
}

​
Complete example

import { useAIStore } from '@/store/aiStore';
import { parseAIResponse, executeCommands } from '@/lib/gameGenerator';

async function handleAIPrompt(userInput: string) {
  const aiStore = useAIStore();
  
  // Add user message
  aiStore.addMessage('user', userInput);
  aiStore.setLoading(true);
  
  try {
    // Call AI API
    const response = await fetch('/api/ai', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({
        message: userInput,
        apiKey: aiStore.apiKey,
      }),
    });
    
    const data = await response.json();
    
    // Parse AI response
    const { commands, plainText } = parseAIResponse(data.message);
    
    // Execute commands
    const log = executeCommands(commands);
    
    // Build response with execution log
    const fullResponse = plainText + '\n\n' + log.join('\n');
    aiStore.addMessage('assistant', fullResponse);
    
  } catch (error) {
    aiStore.setError('Failed to process AI request');
  } finally {
    aiStore.setLoading(false);
  }
}

​
Error handling

The executor includes error handling for invalid commands: 
const log = execute3DCommands([
  { type: 'add_object', objectType: 'invalid_type' }
]);
// log: ['⚠️ Unknown object type: invalid_type']
Common error messages:
  • ⚠️ Unknown object type: {type} - Invalid object type
  • ⚠️ Object not found: "{name}" - Referenced object doesn’t exist
  • ❌ Error executing {type}: {message} - General execution error

​
History integration

The executor automatically pushes to the undo/redo history after executing a batch of commands: 
// After all commands execute
if (commands.length > 0) {
  useEditorStore.getState().pushHistory();
}

​
See also

Build docs developers (and LLMs) love

Get started for freeTalk to us