XML Parsing

Overview

Lich uses a streaming XML parser to process game data in real-time. The XMLParser class implements Ruby’s REXML::StreamListener interface to handle the continuous XML stream from the game server.

XMLParser Class

The XMLParser is the heart of Lich’s data extraction system:

# From lib/common/xmlparser.rb:36
class XMLParser
  attr_reader :mana, :max_mana, :health, :max_health, :spirit, :max_spirit,
              :stamina, :max_stamina, :stance_text, :stance_value,
              :mind_text, :mind_value, :prepared_spell, 
              :encumbrance_text, :encumbrance_value,
              :room_name, :room_title, :room_description,
              :room_exits, :room_exits_string
  
  include REXML::StreamListener
end

The XMLParser uses a stream-based approach rather than DOM parsing. This is efficient for continuous data and allows real-time processing as data arrives.

Stream Processing

The parser implements three key methods from REXML::StreamListener:

tag_start

Called when an opening tag is encountered:

def tag_start(name, attributes)
  @active_tags.push(name)
  @active_ids.push(attributes['id'].to_s)
  
  # Example: Room navigation
  if name == 'nav'
    Lich::Claim.lock if defined?(Lich::Claim)
    GameObj.clear_loot
    GameObj.clear_npcs
    GameObj.clear_pcs
    GameObj.clear_room_desc
    @room_id = attributes['rm'].to_i unless XMLData.game =~ /^DR/
  end
  
  # Example: Progress bars (stats)
  if name == 'progressBar'
    if attributes['id'] == 'mana'
      @mana, @max_mana = attributes['text'].scan(/-?\d+/).map(&:to_i)
    elsif attributes['id'] == 'health'
      @health, @max_health = attributes['text'].scan(/-?\d+/).map(&:to_i)
    end
  end
end

text

Called for text content between tags:

def text(text_string)
  # Room object parsing
  if @active_ids.include?('room objs')
    if @active_tags.include?('a')
      if @bold
        GameObj.new_npc(@obj_exist, @obj_noun, text_string)
      else
        GameObj.new_loot(@obj_exist, @obj_noun, text_string)
      end
    end
  end
  
  # Room player parsing
  elsif @active_ids.include?('room players')
    if @active_tags.include?('a')
      GameObj.new_pc(@obj_exist, @obj_noun, text_string)
    end
  end
end

tag_end

Called when a closing tag is encountered:

def tag_end(name)
  if name == 'compass'
    # Finalize room data
    @room_count += 1
    $room_count += 1
  end
  
  @last_tag = @active_tags.pop
  @last_id = @active_ids.pop
end

XML Tag Reference

Character Stats

<progressBar id="mana" value="50" text="120/240"/>
<progressBar id="health" value="100" text="150/150"/>
<progressBar id="spirit" value="100" text="10/10"/>
<progressBar id="stamina" value="80" text="100/125"/>

These are parsed into XMLData attributes:

echo "Mana: #{XMLData.mana}/#{XMLData.max_mana}"
echo "Health: #{XMLData.health}/#{XMLData.max_health}"
echo "Spirit: #{XMLData.spirit}/#{XMLData.max_spirit}"
echo "Stamina: #{XMLData.stamina}/#{XMLData.max_stamina}"

Room Information

<nav rm="12345"/>
<streamWindow id="main" subtitle="   - [Town Square, Southeast]"/>
<component id='room desc'>A bustling town square...</component>
<component id='room exits'>
  <d cmd='go door'>a wooden door</d>
  <d cmd='north'>north</d>
  <d cmd='south'>south</d>
</component>

Accessed via:

echo "Room: #{XMLData.room_name}"
echo "Description: #{XMLData.room_description}"
echo "Exits: #{XMLData.room_exits.join(', ')}"
echo "Room ID: #{XMLData.room_id}"

Objects and NPCs

<component id='room objs'>
  <a exist="12345" noun="troll">a cave troll</a>
  <a exist="12346" noun="sword">a steel broadsword</a>
</component>

Parsed into GameObj registries:

# NPCs (bold text in XML)
GameObj.npcs.each { |npc| echo npc.name }

# Loot (non-bold text)
GameObj.loot.each { |item| echo item.name }

The @bold flag tracks whether text is bold (NPC) or normal (loot). This is set by <pushBold> and <popBold> tags.

Player Characters

<component id='room players'>
  <a exist="54321" noun="Tamsin">Tamsin</a> who is sitting
</component>

Accessed via:

GameObj.pcs.each { |pc|
  echo "#{pc.name} - Status: #{pc.status}"
}

Real-Time Data Access

All parsed data is immediately available through the global XMLData object:

# Character vitals
while XMLData.health < XMLData.max_health * 0.5
  fput "rest"
  sleep 5
end

# Mana tracking
if XMLData.mana >= 50
  fput "prep 506"
  fput "cast"
end

# Room awareness
if XMLData.room_name =~ /Dangerous Area/
  echo "Warning! Dangerous location!"
end

Stream Types

Lich tracks different stream contexts:

if name == 'pushStream'
  @in_stream = true
  @current_stream = attributes['id'].to_s
  
  # Clear inventory when entering inv stream
  if attributes['id'].to_s == 'inv'
    GameObj.clear_inv if XMLData.game =~ /^GS/
  end
end

if name == 'popStream'
  @in_stream = false
  @current_stream = String.new
end

Common streams:

inv - Inventory display
combat - Combat messages
room - Room descriptions
familiar - Familiar view
percWindow - DragonRealms active spells

Game-Specific Parsing

GemStone IV

if XMLData.game =~ /^GS/
  # GS-specific: Room titles include UID
  # "   - [Town Square] - [12345]"
  @room_title = '[' + attributes['subtitle'][3..-1].gsub(/ - \d+$/, '') + ']'
end

DragonRealms

if XMLData.game =~ /^DR/
  # DR-specific: Active spell tracking
  if @dr_active_spell_tracking
    case text_string
    when /(?<spell>^[^\(]+)\((?<duration>\d+|Indefinite|OM|Fading)/i
      spell = Regexp.last_match[:spell].strip
      duration = Regexp.last_match[:duration].to_i
      @dr_active_spells_tmp[spell] = duration
    end
  end
end

Active Spells / Buffs

Lich 5 supports the newer PSM 3.0 dialog format:

def tag_start(name, attributes)
  if name == 'dialogData' && attributes['clear'] == 't'
    @dialogs[attributes["id"]]&.clear
    ActiveSpell.request_update
  end
  
  if name == 'progressBar' && PSM_3_DIALOG_IDS.include?(@active_ids[-2])
    parse_psm3_progressbar(@active_ids[-2], attributes)
  end
end

def parse_psm3_progressbar(kind, attributes)
  @dialogs[kind] ||= {}
  id = attributes["id"].to_i
  name = attributes["text"]
  value = attributes["time"]  # "HH:MM:SS" or "Indefinite"
  
  if value.downcase.eql?("indefinite")
    @dialogs[kind][name] = Time.now + (10 * 31_536_000)  # 10 years
  else
    hour, minute, second = value.split(':')
    @dialogs[kind][name] = Time.now + (hour.to_i * 3600) + 
                           (minute.to_i * 60) + second.to_i
  end
end

Access active spells:

XMLData.active_spells.each { |name, expires_at|
  remaining = expires_at - Time.now
  echo "#{name}: #{remaining.to_i} seconds"
}

Prompt Handling

The prompt tag provides timing information:

if name == 'prompt'
  @server_time = attributes['time'].to_i
  @server_time_offset = (Time.now.to_f - @server_time)
  
  # Update spell tracking on prompt
  if @dr_active_spell_tracking
    @dr_active_spell_tracking = false
    @dr_active_spells = @dr_active_spells_tmp
    @dr_active_spells_tmp = {}
  end
end

Injury Tracking

Injuries are tracked with special handling:

if name == 'image' && @active_ids.include?('injuries')
  if @injuries.keys.include?(attributes['id'])
    if attributes['name'] =~ /Injury/i
      @injuries[attributes['id']]['wound'] = attributes['name'].slice(/\d/).to_i
    elsif attributes['name'] =~ /Scar/i
      @injuries[attributes['id']]['wound'] = 0
      @injuries[attributes['id']]['scar'] = attributes['name'].slice(/\d/).to_i
    end
  end
end

Injuries are stored per body part:

@injuries = {
  'head' => { 'scar' => 0, 'wound' => 0 },
  'neck' => { 'scar' => 0, 'wound' => 0 },
  'chest' => { 'scar' => 0, 'wound' => 0 },
  'back' => { 'scar' => 0, 'wound' => 0 },
  'rightArm' => { 'scar' => 0, 'wound' => 0 },
  'leftArm' => { 'scar' => 0, 'wound' => 0 },
  # ... more body parts
}

Common Patterns

Waiting for Specific Text

# The parser processes XML continuously
# Use waitfor to block until text appears
waitfor "You feel more refreshed"
echo "Recovered!"

Monitoring Multiple Conditions

result = waitfor("arrives", "leaves", "attacks")

if result =~ /arrives/
  echo "Someone arrived"
elsif result =~ /leaves/
  echo "Someone left"
elsif result =~ /attacks/
  echo "Combat!"
end

State-Based Scripting

loop do
  if XMLData.health < 30
    fput "stop"
    fput "stance defensive"
  elsif XMLData.health > 80
    fput "stance offensive"
  end
  
  sleep 1
end

Performance Considerations

Stream-Based Design

Using REXML’s StreamListener avoids building a full DOM tree, keeping memory usage low even during extended sessions.

Efficient Object Tracking

GameObj registries are cleared and rebuilt on room changes rather than attempting to track individual object movements.

Minimal Regex

Tag parsing uses simple string operations where possible, reserving regex for complex pattern matching.

Debugging XML Issues

When XML parsing fails:

# Enable debugging
Lich.debug_messaging = true

# Raw XML is logged to:
# ~/.lich/lich.log

Common issues:

Nested quotes in attributes
Unescaped ampersands
Malformed closing tags

The XMLCleaner module in lib/games.rb handles many of these automatically.

Example: Custom Parser Hook

# Watch for specific XML patterns
beforedying {
  DownstreamHook.remove('my_parser')
}

parser = proc { |xml|
  if xml =~ /<resource id="coins" value="(\d+)"/
    coins = $1.to_i
    echo "Coins updated: #{coins}"
  end
  xml  # Return unmodified
}

DownstreamHook.add('my_parser', parser)

Downstream hooks receive raw XML before it’s parsed. Modifying the XML can break parsing for other scripts.

Get Started

Core Concepts

Script Development

Game Modules

Advanced

Overview

XMLParser Class

Stream Processing

tag_start

text

tag_end

XML Tag Reference

Character Stats

Room Information

Objects and NPCs

Player Characters

Real-Time Data Access

Stream Types

Game-Specific Parsing

GemStone IV

DragonRealms

Active Spells / Buffs

Prompt Handling

Injury Tracking

Common Patterns

Waiting for Specific Text

Monitoring Multiple Conditions

State-Based Scripting

Performance Considerations

Debugging XML Issues

Example: Custom Parser Hook

Next Steps

Game Objects

Scripting

Build docs developers (and LLMs) love

Get Started

Core Concepts

Script Development

Game Modules

Advanced

​Overview

​XMLParser Class

​Stream Processing

​tag_start

​text

​tag_end

​XML Tag Reference

​Character Stats

​Room Information

​Objects and NPCs

​Player Characters

​Real-Time Data Access

​Stream Types

​Game-Specific Parsing

​GemStone IV

​DragonRealms

​Active Spells / Buffs

​Prompt Handling

​Injury Tracking

​Common Patterns

​Waiting for Specific Text

​Monitoring Multiple Conditions

​State-Based Scripting

​Performance Considerations

​Debugging XML Issues

​Example: Custom Parser Hook

​Next Steps

Game Objects

Scripting

Build docs developers (and LLMs) love

Overview

XMLParser Class

Stream Processing

tag_start

text

tag_end

XML Tag Reference

Character Stats

Room Information

Objects and NPCs

Player Characters

Real-Time Data Access

Stream Types

Game-Specific Parsing

GemStone IV

DragonRealms

Active Spells / Buffs

Prompt Handling

Injury Tracking

Common Patterns

Waiting for Specific Text

Monitoring Multiple Conditions

State-Based Scripting

Performance Considerations

Debugging XML Issues

Example: Custom Parser Hook

Next Steps