Overview
The JsonlMirror class provides append-only JSONL storage for captured events with automatic indexing for efficient querying.
Class: JsonlMirror
Constructor
constructor(
rootDir: string = ".codaph",
options?: {
indexWriteMode?: "immediate" | "batch";
autoFlushEveryEvents?: number;
}
)
Root directory for mirror storage
options.indexWriteMode
'immediate' | 'batch'
default:"immediate"
Index write strategy:
immediate: Flush indexes after every eventbatch: Buffer index updates for better performance
options.autoFlushEveryEvents
Auto-flush indexes after N events (only applies to batch mode)
Methods
appendEvent
Appends an event to the appropriate daily segment.
async appendEvent(
event: CapturedEventEnvelope
): Promise<MirrorAppendResult>
event
CapturedEventEnvelope
required
Event to append
Relative path to segment file (e.g., events/abc123/2026/03/03/segment-20260303.jsonl)
Line number within segment
16-character SHA-256 hash of event line
True if event was deduplicated (already exists)
appendRawLine
Appends a raw JSONL line to the session’s raw log.
async appendRawLine(
sessionId: string,
line: string
): Promise<void>
Raw JSONL line (without trailing newline)
{rootDir}/runs/{sessionId}/raw-codex.ndjson
flush
Flushes all buffered writes and indexes.
async flush(): Promise<void>
Index Structures
Manifest
Tracks all daily segments.
Location: {rootDir}/index/{repoId}/manifest.json
interface RepoManifest {
repoId: string;
segments: Record<string, SegmentMeta>;
}
interface SegmentMeta {
id: string; // e.g., "20260303"
relativePath: string; // Segment file path
from: string; // Earliest event timestamp
to: string; // Latest event timestamp
eventCount: number; // Total events in segment
}
Sparse Index
Tracks sessions, threads, and actors.
Location: {rootDir}/index/{repoId}/sparse-index.json
interface SparseIndex {
repoId: string;
sessions: Record<string, SparseSessionIndex>;
threads: Record<string, SparseThreadIndex>;
actors: Record<string, SparseActorIndex>;
}
interface SparseSessionIndex {
from: string;
to: string;
eventCount: number;
segments: string[]; // Segment paths containing this session
threads: string[]; // Thread IDs in this session
actors: string[]; // Actor IDs in this session
}
interface SparseThreadIndex {
sessionId: string;
from: string;
to: string;
eventCount: number;
segments: string[];
}
interface SparseActorIndex {
from: string;
to: string;
eventCount: number;
sessions: string[]; // Sessions by this actor
segments: string[];
}
Event ID Index
Enables deduplication and event lookup.
Location: {rootDir}/index/{repoId}/event-ids.json
interface EventIdIndex {
repoId: string;
events: Record<string, EventIdIndexEntry>;
}
interface EventIdIndexEntry {
segment: string; // Segment containing event
ts: string; // Event timestamp
sessionId: string;
actorId: string | null;
}
Helper Functions
readManifest
async function readManifest(
rootDir: string,
repoId: string
): Promise<RepoManifest>
readSparseIndex
async function readSparseIndex(
rootDir: string,
repoId: string
): Promise<SparseIndex>
readEventIdIndex
async function readEventIdIndex(
rootDir: string,
repoId: string
): Promise<EventIdIndex>
readEventsFromSegments
async function readEventsFromSegments(
rootDir: string,
segmentPaths: string[]
): Promise<CapturedEventEnvelope[]>
Storage Layout
.codaph/
├── events/
│ └── {repoId}/
│ └── 2026/
│ └── 03/
│ └── 03/
│ └── segment-20260303.jsonl
├── runs/
│ └── {sessionId}/
│ └── raw-codex.ndjson
└── index/
└── {repoId}/
├── manifest.json
├── sparse-index.json
└── event-ids.json
Usage Example
import { JsonlMirror } from './lib/mirror-jsonl';
import { createCapturedEvent } from './lib/core-types';
const mirror = new JsonlMirror('.codaph', {
indexWriteMode: 'batch',
autoFlushEveryEvents: 100,
});
// Append events
for (let i = 0; i < 200; i++) {
const event = createCapturedEvent({
source: 'codex_sdk',
repoId: 'abc123',
sessionId: 'session-1',
threadId: 'thread-1',
ts: new Date().toISOString(),
sequence: i,
eventType: 'item.completed',
payload: { item: { type: 'reasoning', text: `Step ${i}` } },
});
const result = await mirror.appendEvent(event);
if (result.deduplicated) {
console.log('Duplicate event:', event.eventId);
}
}
// Flush at end
await mirror.flush();
Buffering
Use batch mode for high-throughput scenarios:
const mirror = new JsonlMirror('.codaph', {
indexWriteMode: 'batch',
autoFlushEveryEvents: 50,
});
Deduplication
The mirror automatically deduplicates based on eventId. Duplicate appends return deduplicated: true without writing to disk.
Daily Segments
Events are automatically partitioned by UTC date, balancing:
- File size: Prevents segments from growing unbounded
- Query efficiency: Enables date-range filtering at segment level