Incremental Parsing

Incremental parsing is a performance optimization technique that reuses previously parsed content when appending new markdown to an existing document. Instead of reparsing the entire document, MarkdownView parses only the changed portion and merges it with the stable prefix.

Overview

When working with large or streaming markdown content, reparsing the entire document on every update can become expensive. MarkdownView’s incremental parser intelligently:

Identifies the stable prefix of blocks that haven’t changed
Parses only the tail portion that contains new or modified content
Merges the cached prefix with the newly parsed tail
Maintains accurate block ranges for future incremental updates

The incremental parser is automatically used when calling setMarkdown(string:) on MarkdownTextView. For manual control, use the parseIncremental method directly.

The `parseIncremental` Method

The core incremental parsing API is available in MarkdownParser:

public func parseIncremental(
    previousMarkdown: String,
    newMarkdown: String,
    previousBlocks: [MarkdownBlockNode],
    previousRanges: [RootBlockRange]? = nil
) -> IncrementalParseResult?

Location: Sources/MarkdownParser/MarkdownParser/MarkdownParser.swift:110

Parameters

previousMarkdown: The complete markdown string from the last parse
newMarkdown: The updated markdown string (must be longer than previous)
previousBlocks: The parsed block nodes from the previous parse result
previousRanges: Optional cached block ranges from a previous parseBlockRange() call

Return Value

Returns IncrementalParseResult on success, or nil if incremental parsing is not possible (e.g., content was deleted or modified in the middle).

public struct IncrementalParseResult {
    public let stablePrefixBlockCount: Int
    public let tailResult: ParseResult
    public let blockRanges: [RootBlockRange]
}

stablePrefixBlockCount: Number of blocks from the previous parse that remain unchanged
tailResult: Parse result containing only the new/modified blocks
blockRanges: Updated block ranges for the complete document

RootBlockRange

The RootBlockRange structure maps markdown source positions to output blocks:

public struct RootBlockRange {
    public let type: MarkdownNodeType
    public let startIndex: String.Index
    public let endIndex: String.Index
    public let outputBlockCount: Int
}

Location: Sources/MarkdownParser/MarkdownParser/MarkdownParser.swift:56

Properties

type: The markdown node type (e.g., paragraph, code block, list)
startIndex: Character position where this block starts in the source
endIndex: Character position where this block ends in the source
outputBlockCount: Number of rendered blocks produced after transformation

Some markdown blocks expand into multiple output blocks after processing. For example, a list might be transformed into multiple specialized block nodes. The outputBlockCount tracks this expansion.

Stable Prefix Optimization

The incremental parser uses a sophisticated algorithm to determine the stable prefix:

Prefix Validation: Verifies that newMarkdown starts with the exact previousMarkdown content
Tail Block Calculation: Determines how many blocks from the end might have changed
Boundary Detection: Uses block ranges to find the exact character position where stable content ends
Text Comparison: Ensures the stable portion of text hasn’t been modified

Location: Sources/MarkdownParser/MarkdownParser/MarkdownParser.swift:110-160

Tail Block Heuristics

The parser uses intelligent heuristics to determine the minimum tail size:

private func preferredTailRootBlockCount(
    newMarkdown: String,
    previousBlocks: [MarkdownBlockNode],
    ranges: [RootBlockRange]
) -> Int

Location: Sources/MarkdownParser/MarkdownParser/MarkdownParser.swift:205

Base: Reparse at least 1 block by default
Block Type: Reparse 2 blocks if the last block is a list, blockquote, code block, or table
Continuation Markers: Reparse 4 blocks if the suffix contains list markers, blockquotes, or open tables
Open Code Fence: Reparse 5 blocks if there’s an unclosed code fence (``` or ~~~)

Incremental parsing returns nil if:

The previous markdown was empty
The new markdown is shorter (content was deleted)
The new markdown doesn’t start with the previous content
No stable prefix could be identified

Usage Example

Direct API Usage

import MarkdownParser

let parser = MarkdownParser()

// Initial parse
let initialMarkdown = """
# Document Title

First paragraph with some content.
"""

let initialResult = parser.parse(initialMarkdown)
let initialRanges = parser.parseBlockRange(initialMarkdown)

// Append new content
let updatedMarkdown = initialMarkdown + """

Second paragraph appended to the document.

Some code here.
"""

// Incremental parse
if let incrementalResult = parser.parseIncremental(
    previousMarkdown: initialMarkdown,
    newMarkdown: updatedMarkdown,
    previousBlocks: initialResult.document,
    previousRanges: initialRanges
) {
    // Merge stable prefix with new tail
    let stableBlocks = Array(initialResult.document.prefix(incrementalResult.stablePrefixBlockCount))
    let completeDocument = stableBlocks + incrementalResult.tailResult.document
    
    print("Reused \(incrementalResult.stablePrefixBlockCount) stable blocks")
    print("Parsed \(incrementalResult.tailResult.document.count) new blocks")
} else {
    // Fall back to full parse
    let fullResult = parser.parse(updatedMarkdown)
}

Automatic Incremental Parsing

When using MarkdownTextView, incremental parsing is handled automatically:

let markdownView = MarkdownTextView()

// First update
markdownView.setMarkdown(string: "# Hello\n\nFirst paragraph.")

// Incremental update (automatically uses incremental parsing)
markdownView.setMarkdown(string: "# Hello\n\nFirst paragraph.\n\nSecond paragraph.")

Implementation: Sources/MarkdownView/MarkdownTextView/MarkdownTextView+Private.swift:66-165

Plain Text Append Fast Path

For simple text appends to the last paragraph, MarkdownView uses an even faster optimization that skips parsing entirely:

func makePlainTextAppendFastPath(for markdown: String) -> PreprocessedContent?

Location: Sources/MarkdownView/MarkdownTextView/MarkdownTextView+Private.swift:178 This fast path activates when:

The new content is an append (has the previous markdown as prefix)
The appended text contains only plain text (no markdown syntax)
The last block is a paragraph containing only text nodes
No special characters are present: \n\r`*_[]()!$<>|~\\

When streaming plain text responses from an LLM, the fast path delivers <0.1ms updates in steady state, as shown in the performance benchmarks.

Math Identifier Shifting

When merging incremental results, math expressions need special handling to avoid identifier conflicts:

func shiftMathIdentifiers(in result: ParseResult, by offset: Int) -> ParseResult

Location: Sources/MarkdownParser/MarkdownParser/MarkdownParser.swift:325 This function:

Finds the maximum math identifier in the stable prefix blocks
Shifts all math identifiers in the tail result by offset + 1
Updates the math context dictionary with shifted keys

This ensures each math expression has a unique identifier across the merged document.

Performance Characteristics

Operation	Time
Plain-text stream append (steady-state)	<0.1 ms
Parse 500 blocks (full parse)	~5 ms
Parse 50 blocks (incremental tail)	~1 ms
Merge stable prefix with tail	<0.1 ms

Performance measurements from README.md benchmarks. Actual performance depends on document complexity and hardware.

Best Practices

1. Cache Block Ranges

let ranges = parser.parseBlockRange(markdown)
// Cache ranges for future incremental parses
let incrementalResult = parser.parseIncremental(
    previousMarkdown: previous,
    newMarkdown: updated,
    previousBlocks: blocks,
    previousRanges: ranges  // Reuse cached ranges
)

2. Handle Nil Gracefully

if let incrementalResult = parser.parseIncremental(...) {
    // Use incremental result
} else {
    // Fall back to full parse
    let fullResult = parser.parse(newMarkdown)
}

3. Use MarkdownTextView for Automatic Management

The MarkdownTextView handles all incremental parsing logic automatically, including caching ranges and merging results.

Getting Started

Core Concepts

Guides

Advanced

Overview

The `parseIncremental` Method

Parameters

Return Value

RootBlockRange

Properties

Stable Prefix Optimization

Tail Block Heuristics

Usage Example

Direct API Usage

Automatic Incremental Parsing

Plain Text Append Fast Path

Math Identifier Shifting

Performance Characteristics

Best Practices

1. Cache Block Ranges

2. Handle Nil Gracefully

3. Use MarkdownTextView for Automatic Management

See Also

Build docs developers (and LLMs) love

Getting Started

Core Concepts

Guides

Advanced

​Overview

​The parseIncremental Method

​Parameters

​Return Value

​RootBlockRange

​Properties

​Stable Prefix Optimization

​Tail Block Heuristics

​Usage Example

​Direct API Usage

​Automatic Incremental Parsing

​Plain Text Append Fast Path

​Math Identifier Shifting

​Performance Characteristics

​Best Practices

​1. Cache Block Ranges

​2. Handle Nil Gracefully

​3. Use MarkdownTextView for Automatic Management

​See Also

Build docs developers (and LLMs) love

Overview

The `parseIncremental` Method

Parameters

Return Value

RootBlockRange

Properties

Stable Prefix Optimization

Tail Block Heuristics

Usage Example

Direct API Usage

Automatic Incremental Parsing

Plain Text Append Fast Path

Math Identifier Shifting

Performance Characteristics

Best Practices

1. Cache Block Ranges

2. Handle Nil Gracefully

3. Use MarkdownTextView for Automatic Management

See Also