Skip to main content

​
Overview

decodeStreamSync converts TOON-formatted lines into a stream of structured JSON events without building the complete value tree in memory. This enables memory-efficient parsing, custom transformations, and progressive processing of TOON data.
This is the synchronous variant that accepts and returns synchronous iterables. For async sources like file streams or network responses, use decodeStream instead.

​
Function Signature

function decodeStreamSync(
  lines: Iterable<string>,
  options?: DecodeStreamOptions
): Iterable<JsonStreamEvent>

​
Parameters

​
lines
Iterable<string>
required
An iterable of TOON lines (without trailing newlines). Can be an array of strings, a generator, or any synchronous iterable.
​
options
DecodeStreamOptions
Optional configuration object for decoding behavior.
​
indent
number
default:2
Number of spaces per indentation level. Must match the indentation used in the TOON input.
​
strict
boolean
default:true
When true, enforces strict validation of array lengths, tabular row counts, and blank line restrictions.
Path expansion not supported: The expandPaths option from decode() is not available in streaming mode. Streaming decoders emit raw events without post-processing transformations.

​
Return Value

​
events
Iterable<JsonStreamEvent>
A synchronous iterable that yields JSON stream events. Each event represents a structural element of the JSON data model.

​
Event Types

The decoder emits six types of events:
​
startObject
{ type: "startObject" }
Marks the beginning of a JSON object.
​
endObject
{ type: "endObject" }
Marks the end of a JSON object.
​
startArray
{ type: "startArray", length: number }
Marks the beginning of a JSON array. The length field indicates the declared array length from the TOON header.
​
endArray
{ type: "endArray" }
Marks the end of a JSON array.
​
key
{ type: "key", key: string, wasQuoted?: boolean }
Represents an object key. The optional wasQuoted field is true when the key was quoted in the TOON source.
​
primitive
{ type: "primitive", value: JsonPrimitive }
Represents a primitive value (string, number, boolean, or null).

​
Usage Examples

​
Basic Event Streaming

Process TOON lines event-by-event for custom handling: 
import { decodeStreamSync } from '@toon-format/toon'

const lines = [
  'name: Alice',
  'age: 30',
  'active: true'
]

for (const event of decodeStreamSync(lines)) {
  console.log(event)
}

// Output:
// { type: 'startObject' }
// { type: 'key', key: 'name' }
// { type: 'primitive', value: 'Alice' }
// { type: 'key', key: 'age' }
// { type: 'primitive', value: 30 }
// { type: 'key', key: 'active' }
// { type: 'primitive', value: true }
// { type: 'endObject' }

​
Array Processing

Handle array events with declared lengths: 
import { decodeStreamSync } from '@toon-format/toon'

const lines = [
  'users[2]:',
  '  - name: Alice',
  '  - name: Bob'
]

for (const event of decodeStreamSync(lines)) {
  if (event.type === 'startArray') {
    console.log(`Processing ${event.length} items...`)
  }
  console.log(event)
}

// Output includes:
// { type: 'startArray', length: 2 }

​
Custom Value Builder

Build values with custom logic by handling events manually: 
import { decodeStreamSync } from '@toon-format/toon'

function buildCustomValue(lines: string[]) {
  const events = decodeStreamSync(lines)
  const stack: any[] = []
  let current: any = null

  for (const event of events) {
    switch (event.type) {
      case 'startObject':
        const obj = {}
        if (current !== null) stack.push(current)
        current = obj
        break

      case 'key':
        stack.push(event.key)
        break

      case 'primitive':
        const key = stack.pop()
        current[key] = event.value
        break

      case 'endObject':
        if (stack.length > 0) {
          const parent = stack.pop()
          const parentKey = stack.pop()
          parent[parentKey] = current
          current = parent
        }
        break
    }
  }

  return current
}

const lines = ['name: Alice', 'score: 95']
console.log(buildCustomValue(lines))
// { name: 'Alice', score: 95 }

​
Filtering Values During Decode

Use streaming events to filter or transform data on-the-fly: 
import { decodeStreamSync } from '@toon-format/toon'

function* filterSensitiveKeys(lines: string[]) {
  let skipNext = false

  for (const event of decodeStreamSync(lines)) {
    if (event.type === 'key' && event.key === 'password') {
      skipNext = true
      continue // Skip the key event
    }

    if (skipNext && event.type === 'primitive') {
      skipNext = false
      continue // Skip the value event
    }

    yield event
  }
}

const lines = [
  'username: alice',
  'password: secret123',
  'email: [email protected]'
]

for (const event of filterSensitiveKeys(lines)) {
  console.log(event)
}
// password key and value are omitted

​
Reading from a Generator

Work with lazy line generators for memory efficiency: 
import { decodeStreamSync } from '@toon-format/toon'

function* readLinesFromFile(filePath: string) {
  // Simplified example - actual implementation would read line-by-line
  const content = readFileSync(filePath, 'utf-8')
  yield* content.split('\n')
}

const lineGenerator = readLinesFromFile('data.toon')
const events = decodeStreamSync(lineGenerator)

for (const event of events) {
  // Process events as they're decoded
  if (event.type === 'primitive') {
    console.log('Value:', event.value)
  }
}

​
Custom Indentation

Decode TOON with non-standard indentation: 
import { decodeStreamSync } from '@toon-format/toon'

const lines = [
  'user:',
  '    name: Alice',      // 4-space indent
  '    profile:',
  '        bio: Engineer' // 8-space indent
]

const events = decodeStreamSync(lines, { indent: 4 })

for (const event of events) {
  console.log(event)
}

​
Strict vs. Non-Strict Mode

Control validation behavior with the strict option: 
import { decodeStreamSync } from '@toon-format/toon'

// Strict mode (default): throws on mismatched counts
const strictLines = [
  'items[3]:', // Declares 3 items
  '  - apple',
  '  - banana'  // Only 2 items provided
]

try {
  for (const event of decodeStreamSync(strictLines)) {
    // Will throw: expected 3 items, got 2
  }
} catch (error) {
  console.error(error.message)
}

// Non-strict mode: allows mismatched counts
const events = decodeStreamSync(strictLines, { strict: false })
for (const event of events) {
  console.log(event) // Processes without error
}

​
Event Stream Lifecycle

Understanding the event sequence for different TOON structures: 
const lines = ['key: value']
// Events:
// startObject → key → primitive → endObject

​
Performance Considerations

Memory Efficient

Only the current event is in memory at any time, enabling processing of arbitrarily large TOON files.

Early Termination

Stop iteration at any point to avoid processing unnecessary data.

No Buffering

Events are yielded as soon as they’re parsed with zero buffering overhead.

Generator-Friendly

Works seamlessly with generator functions and lazy iterables.

​
Error Handling

Errors are thrown synchronously when invalid TOON syntax is encountered: 
import { decodeStreamSync } from '@toon-format/toon'

const invalidLines = [
  'key: value',
  '  invalid indentation' // Unexpected indent
]

try {
  for (const event of decodeStreamSync(invalidLines)) {
    console.log(event)
  }
} catch (error) {
  console.error('Decode error:', error.message)
  // Handle syntax errors
}

decodeStream

Async variant for streaming sources like file readers and network responses

decodeFromLines

Decode lines directly to a value (supports path expansion)

decode

Standard decoder that builds the complete value tree

Event Builder Guide

Learn how to build values from event streams

​
Type Definitions

type JsonStreamEvent
  = | { type: 'startObject' }
    | { type: 'endObject' }
    | { type: 'startArray', length: number }
    | { type: 'endArray' }
    | { type: 'key', key: string, wasQuoted?: boolean }
    | { type: 'primitive', value: JsonPrimitive }

type JsonPrimitive = string | number | boolean | null

interface DecodeStreamOptions {
  indent?: number      // Default: 2
  strict?: boolean     // Default: true
  expandPaths?: never  // Not supported in streaming mode
}

Build docs developers (and LLMs) love

Get started for freeTalk to us