Minecraft Integration

Overview

The Minecraft integration is an intelligent bot built on a four-layered cognitive architecture inspired by cognitive science. AIRI can understand natural language commands, explore the world autonomously, gather resources, craft items, and interact with players.

Features

Cognitive Architecture: Perception → Reflex → Conscious → Action layers
Natural Language Understanding: Chat with the bot using plain English
Autonomous Gameplay: Gather resources, craft items, build structures
World Awareness: Navigate terrain, avoid obstacles, track entities
Combat System: Fight hostile mobs and defend itself
Inventory Management: Smart item collection and crafting
Debug Dashboard: Web-based real-time monitoring and MCP integration
Mineflayer Viewer: Watch the bot’s POV in your browser

Prerequisites

Node.js 18 or higher
Minecraft Java Edition server (1.20 recommended)
OpenAI API key (or compatible LLM provider)
AIRI server runtime (optional, for integration)

Setup

1. Prepare Minecraft Server

You need a Minecraft server where the bot can connect. Options:

Local server: Download from minecraft.net
Aternos: Free hosting at aternos.org
Existing server: Use any Java Edition server

The bot supports Minecraft Java Edition versions 1.12 - 1.20. Version 1.20 is recommended.

2. Configure Environment

Navigate to the Minecraft service:

cd services/minecraft
cp .env .env.local

Edit .env.local:

# Minecraft Server
BOT_USERNAME='AIRI'
BOT_HOSTNAME='localhost'
BOT_PORT=25565
BOT_VERSION='1.20'
BOT_AUTH='offline'  # Use 'microsoft' for online mode

# OpenAI Configuration
OPENAI_API_KEY='sk-...'
OPENAI_MODEL='gpt-4o'
OPENAI_REASONING_MODEL='o1-mini'
OPENAI_API_BASEURL='https://api.openai.com/v1'

# AIRI Server (optional)
AIRI_WS_BASEURL='ws://localhost:6121/ws'
AIRI_CLIENT_NAME='minecraft-bot'

3. Install Dependencies

From project root:

pnpm install

4. Start the Bot

pnpm run -F @proj-airi/minecraft-bot start

You should see:

[Bot] Connecting to localhost:25565...
[Bot] Successfully logged in as AIRI
[MineflayerViewer] Started on http://localhost:3007
[DebugService] Debug server started on http://localhost:3008

Usage

In-Game Commands

Talk to the bot in Minecraft chat:

Gather 10 oak logs
Craft a wooden pickaxe
Find some iron ore
Build a small house
Follow me

Web Viewer

Open http://localhost:3007 to see the bot’s first-person view in real-time.

Debug Dashboard

Open http://localhost:3008 for:

Real-time bot status
Cognitive layer monitoring
Tool execution interface
MCP server integration

Cognitive Architecture

The bot operates on four distinct layers:

Layer A: Perception

Location: src/cognitive/perception/ Collects and processes raw Mineflayer events into normalized signals:

Chat messages
Health/hunger changes
Entity spawns
Block interactions
Combat events

Layer B: Reflex

Location: src/cognitive/reflex/ Handles immediate, instinctive reactions:

Auto-eat when hungry
Dodge incoming attacks
React to sudden damage
Idle behaviors (looking around)

Layer C: Conscious

Location: src/cognitive/conscious/ LLM-powered reasoning and planning:

Interprets user commands
Plans multi-step tasks
Manages goal hierarchies
Handles conversation

Layer D: Action

Location: src/cognitive/action/ Executes concrete tasks in the world:

Movement and pathfinding
Block breaking/placing
Item crafting
Entity interaction
Inventory management

Code Examples

Bot Initialization

From services/minecraft/src/main.ts:22:

import { initBot } from './composables/bot'
import { CognitiveEngine } from './cognitive'
import { Client } from '@proj-airi/server-sdk'

async function main() {
  // Initialize Mineflayer bot with plugins
  const { bot } = await initBot({
    botConfig: config.bot,
    plugins: [
      MineflayerPathfinder,
      MineflayerCollectBlock,
      MineflayerTool,
      MineflayerPVP,
    ],
    reconnect: {
      enabled: true,
      maxRetries: 5,
    },
  })

  // Connect to AIRI server
  const airiClient = new Client({
    name: config.airi.clientName,
    url: config.airi.wsBaseUrl,
  })

  // Load cognitive engine
  await bot.loadPlugin(CognitiveEngine({ airiClient }))
}

Configuration Loading

From services/minecraft/src/composables/config.ts:62:

export function initEnv(): void {
  config.bot = {
    username: getEnvVar('BOT_USERNAME', 'airi-bot'),
    host: getEnvVar('BOT_HOSTNAME', 'localhost'),
    port: getEnvNumber('BOT_PORT', 25565),
    auth: getEnvVar('BOT_AUTH', 'offline'),
    version: getEnvVar('BOT_VERSION', '1.20'),
  }

  config.openai = {
    apiKey: getEnvVar('OPENAI_API_KEY', ''),
    baseUrl: getEnvVar('OPENAI_API_BASEURL', ''),
    model: getEnvVar('OPENAI_MODEL', ''),
    reasoningModel: getEnvVar('OPENAI_REASONING_MODEL', ''),
  }
}

Skills System

The bot has modular skills in src/skills/: Movement (src/skills/movement.ts):

// Go to coordinates
await bot.skills.movement.goto(x, y, z)

// Follow entity
await bot.skills.movement.follow(player)

// Come to player
await bot.skills.movement.comeToPlayer(username)

Gathering (src/skills/blocks.ts):

// Collect specific block type
await bot.skills.blocks.collectBlock('oak_log', 10)

// Mine blocks in area
await bot.skills.blocks.mineBlocks('iron_ore', {
  count: 5,
  searchRadius: 32
})

Crafting (src/skills/crafting.ts):

// Craft items
await bot.skills.crafting.craft('wooden_pickaxe', 1)

// Ensure item exists (craft if needed)
await bot.skills.crafting.ensure('crafting_table')

Combat (src/skills/combat.ts):

// Attack entity
await bot.skills.combat.attack(entity)

// Defend against attacker
await bot.skills.combat.defend()

Plugin System

Extend bot capabilities with plugins:

interface BotPlugin {
  name: string
  install: (bot: Bot) => Promise<void> | void
  uninstall?: (bot: Bot) => Promise<void> | void
}

// Example plugin
const myPlugin: BotPlugin = {
  name: 'my-plugin',
  install(bot) {
    bot.on('chat', (username, message) => {
      if (message.includes('hello')) {
        bot.chat('Hello!')
      }
    })
  }
}

await bot.loadPlugin(myPlugin)

Advanced Features

Query DSL

Query the world from conscious layer:

// Find nearest entities
const zombies = query.entities()
  .type('zombie')
  .distance('< 10')
  .alive()
  .list()

// Check inventory
const hasDiamonds = query.inventory()
  .item('diamond')
  .count('> 3')
  .exists()

// Find blocks
const ironOres = query.blocks()
  .type('iron_ore')
  .radius(32)
  .list()

MCP Integration

The debug server exposes Model Context Protocol tools:

// Available MCP tools:
- minecraft_status     // Get bot status
- minecraft_execute    // Execute arbitrary code
- minecraft_move       // Move to coordinates
- minecraft_chat       // Send chat message
- minecraft_collect    // Collect blocks
- minecraft_craft      // Craft items

Troubleshooting

Bot can’t connect to server

Verify server is running and accessible
Check BOT_HOSTNAME and BOT_PORT
For online servers, use BOT_AUTH='microsoft'
Ensure Minecraft version matches: BOT_VERSION='1.20'

Bot is stuck or unresponsive

Check debug dashboard for task status
Send stop command in chat
Restart the bot service
Check logs for errors

LLM errors

Verify OPENAI_API_KEY is valid
Check API rate limits
Ensure OPENAI_MODEL is accessible
Try a different model endpoint

Pathfinding fails

Bot may be in difficult terrain
Increase search radius
Clear obstacles near bot
Use /tp to relocate bot

Performance Tips

Use o1-mini for complex reasoning tasks
Set shorter tick intervals for faster reactions
Reduce perception event frequency for lower CPU usage
Enable viewer only when debugging

Next Steps

Factorio Integration

Automate Factorio gameplay

API Reference

Full Minecraft API documentation

Get Started

Core Features

Platforms

Integrations

Configuration

Development

​Overview

​Features

​Prerequisites

​Setup

​1. Prepare Minecraft Server

​2. Configure Environment

​3. Install Dependencies

​4. Start the Bot

​Usage

​In-Game Commands

​Web Viewer

​Debug Dashboard

​Cognitive Architecture

​Layer A: Perception

​Layer B: Reflex

​Layer C: Conscious

​Layer D: Action

​Code Examples

​Bot Initialization

​Configuration Loading

​Skills System

​Plugin System

​Advanced Features

​Query DSL

​MCP Integration

​Troubleshooting

​Bot can’t connect to server

​Bot is stuck or unresponsive

​LLM errors

​Pathfinding fails

​Performance Tips

​Next Steps

Factorio Integration

API Reference

Build docs developers (and LLMs) love

Overview

Features

Prerequisites

Setup

1. Prepare Minecraft Server

2. Configure Environment

3. Install Dependencies

4. Start the Bot

Usage

In-Game Commands

Web Viewer

Debug Dashboard

Cognitive Architecture

Layer A: Perception

Layer B: Reflex

Layer C: Conscious

Layer D: Action

Code Examples

Bot Initialization

Configuration Loading

Skills System

Plugin System

Advanced Features

Query DSL

MCP Integration

Troubleshooting

Bot can’t connect to server

Bot is stuck or unresponsive

LLM errors

Pathfinding fails

Performance Tips

Next Steps