Interactive TUI

Overview

RCLI’s interactive TUI (Terminal User Interface) provides a rich dashboard for voice and text interaction, model management, action browsing, benchmarking, and real-time performance monitoring.

Push-to-Talk

Hold SPACE to record, release to process

Live Metrics

Real-time hardware and performance stats

Model Browser

Download and switch AI models on the fly

Action Manager

Browse, enable/disable, and test 43 macOS actions

Launching the TUI

rcli

The TUI launches immediately and displays:

Conversation area - Chat history with user/RCLI messages
Input prompt - Type text commands or press SPACE for voice
Status bar - Pipeline state, model info, performance metrics
Hardware monitor - CPU, GPU, RAM, audio level (live updates)

Keyboard Shortcuts

Main Interface

Key	Action	Description
SPACE	Push-to-talk	Hold to record voice, release to transcribe and process
ENTER	Send text	Submit typed text command to LLM
ESC	Stop/Quit	Stop current processing or quit TUI
↑/↓	History	Navigate command history (like shell)
T	Toggle trace	Enable/disable tool call trace (inline debugging)

Panel Shortcuts

Key	Panel	Description
M	Models	Browse, download, and switch LLM/STT/TTS models
A	Actions	Browse, enable/disable, and execute macOS actions
B	Benchmarks	Run performance benchmarks (STT, LLM, TTS, E2E, tools, RAG)
R	RAG	Ingest documents, clear index, view stats
D	Cleanup	Delete unused models to free disk space
X	About	Show version, engine info, license
Q	Quit	Exit TUI

All panels are non-blocking. Press ESC to close a panel and return to the conversation.

Push-to-Talk Workflow

Press and Hold SPACE

The status bar shows [RECORDING] and a live waveform animation appears.Audio is buffered to the capture ring buffer while you speak.

Speak Your Command

Speak naturally. The waveform visualizer shows your voice activity in real-time.Silero VAD runs continuously to detect speech and filter silence.

Release SPACE

Recording stops. The TUI shows [TRANSCRIBING…].Audio is sent to Whisper/Parakeet for offline transcription (~44ms latency).

Transcript Appears

The transcribed text is displayed in the conversation area as a user message.Status changes to [THINKING…] as the LLM processes your query.

LLM Response Streams

The LLM’s response is displayed token-by-token as it generates.If tool calls are detected, they execute and results are fed back to the LLM.

TTS Playback

Status changes to [SPEAKING]. The response is synthesized and played.You can press ESC to interrupt playback at any time.

Waveform Visualizer

While recording, the TUI displays a live ASCII waveform:

[RECORDING] █▄▀█▄▀█▄ 0.4s

The waveform shows audio level (RMS) and recording duration.

The waveform helps you confirm your mic is working and adjust speaking volume.

Text Input

Type a command at the prompt and press ENTER:

> open Safari

The LLM processes the text (no STT), executes tool calls, and responds.

Interactive Commands

Special commands for TUI interaction:

Command	Action
`help`	Show keyboard shortcuts
`clear`	Clear conversation history
`models`	Open Models panel
`actions`	Open Actions panel
`bench`	Open Benchmarks panel
`info`	Show engine information
`quit`	Exit TUI

Models Panel (M)

Press M to open the Models browser:

↑/↓ - Navigate model list
←/→ - Switch between LLM/STT/TTS tabs
ENTER - Download or switch to selected model
D - Delete selected model (if installed)
ESC - Close panel

Model States

State	Indicator	Description
Active	`[✓]`	Currently loaded model
Installed	`[*]`	Downloaded but not active
Available	`[ ]`	Not yet downloaded

Hot-Swap

Select a model and press ENTER to switch at runtime:

Current LLM is unloaded
New model is loaded from disk
Model profile is detected
System prompt is re-cached
Selection is persisted to config

Hot-swap takes 500-1500ms depending on model size. Conversation history is preserved.

Actions Panel (A)

Press A to browse and manage the 43 macOS actions:

↑/↓ - Navigate action list
SPACE - Toggle enable/disable
ENTER - Execute action (prompts for arguments)
I - Show action details (parameters, examples)
ESC - Close panel (changes are auto-saved)

Action States

State	Indicator	Description
Enabled	`[✓]`	Available to LLM
Disabled	`[ ]`	Hidden from LLM

Direct Execution

Select an action and press ENTER to execute it directly:

TUI prompts for JSON arguments
Action executes synchronously
Result is displayed (success/fail + output)

Use this to test actions without involving the LLM.

Benchmarks Panel (B)

Press B to run performance benchmarks:

Available Suites

Measures transcription latency and real-time factor:

Load test audio (10 seconds)
Transcribe 100 times
Report avg latency, RTF, throughput

Measures tool-calling accuracy:

100 test queries with expected tools
Measure precision, recall, latency
Compare across models with --all-llm

Running Benchmarks

Select a suite (or “All”)
Press ENTER to start
Progress bar shows completion
Results are displayed in a table
Export to JSON with S (save)

RAG Panel (R)

Press R to manage document indexing:

Options

Ingest Documents - Select directory to index
Clear Index - Delete RAG index and unload retriever
Index Status - Show chunk count, index size, memory usage

Drag-and-Drop

In the TUI, drag a file or folder from Finder into the terminal:

TUI detects the drop event (reads file path from stdin)
Automatically runs rag ingest on the path
Loads the index for immediate querying

This is the fastest way to query a document - just drag, drop, and ask.

Cleanup Panel (D)

Press D to free disk space by deleting unused models:

Shows all installed models with sizes
Mark models for deletion with SPACE
Press ENTER to confirm deletion
Models are removed from ~/Library/RCLI/models/

Deleted models must be re-downloaded to use again. The active LLM/STT/TTS cannot be deleted.

Status Bar

The bottom status bar shows:

Left Side

Pipeline State: IDLE / LISTENING / THINKING / SPEAKING / INTERRUPTED
Active Models: LLM: qwen3-0.6b | STT: whisper-base | TTS: piper-lessac

Right Side

Performance: TTFT: 22ms | Tok/s: 159 | Latency: 131ms
Context: Tokens: 1,234 / 4,096
Audio Level: Mic: ███░░░░ (0.4)

Hardware Monitor

The TUI displays live hardware stats:

CPU: [=========>     ] 64%  (8P + 4E cores)
GPU: [============>  ] 82%  (Metal: 30 cores)
RAM: [======>        ] 42%  (15.2 / 36.0 GB)
Mic: [====>          ] 28%  (RMS level)

Updates every 500ms using:

CPU: sysctl hw.activecpu, kern.cp_time
GPU: Metal Performance Counters (IOKit)
RAM: mach_vm_statistics
Mic: Ring buffer RMS from CoreAudio

Tool Call Trace (T)

Press T to toggle tool call tracing. When enabled, every tool call is shown inline:

> open Safari
  ~ [TRACE] Tool call: open_app({"app_name": "Safari"})
  ~ [TRACE] open_app -> OK: {"success": true, "output": "Opened Safari"}
  RCLI: Done! Safari is now open.

Use Cases:

Debug why an action failed
Understand how the LLM routes requests
Evaluate tool-calling accuracy
Learn model-specific tool formats

Combine with rcli bench --suite tools to benchmark tool-calling across all LLMs.

TUI Architecture

The TUI is built with FTXUI v5.0.0, a modern C++ terminal UI library.

Key Components

// From src/cli/tui_app.h
class TUIApp {
    ftxui::ScreenInteractive screen_;  // Terminal screen
    ftxui::Component input_;           // Text input field
    ftxui::Component conversation_;    // Scrollable chat area
    ftxui::Component status_bar_;      // Bottom status
    ftxui::Component panels_;          // Modal panels (models, actions, etc.)
};

Event Loop

The TUI runs a non-blocking event loop:

while (running) {
    // Handle keyboard input
    if (space_pressed) start_recording();
    if (enter_pressed) send_message();
    
    // Update hardware stats (500ms interval)
    update_cpu_gpu_ram();
    
    // Render frame
    screen_.PostEvent(ftxui::Event::Custom);
}

The TUI is fully responsive and never blocks. You can interrupt LLM generation or TTS playback with ESC at any time.

Next Steps

Interactive Mode

Complete guide to TUI keyboard shortcuts

Models

Learn about all 20+ AI models available

Actions

Browse all 43 macOS actions

Benchmarking

Run performance benchmarks

Get Started

Core Features

Commands

Models

Actions

Advanced

Development

​Overview

Push-to-Talk

Live Metrics

Model Browser

Action Manager

​Launching the TUI

​Keyboard Shortcuts

​Main Interface

​Panel Shortcuts

​Push-to-Talk Workflow

​Waveform Visualizer

​Text Input

​Interactive Commands

​Models Panel (M)

​Navigation

​Model States

​Hot-Swap

​Actions Panel (A)

​Navigation

​Action States

​Direct Execution

​Benchmarks Panel (B)

​Available Suites

​Running Benchmarks

​RAG Panel (R)

​Options

​Drag-and-Drop

​Cleanup Panel (D)

​Status Bar

​Left Side

​Right Side

​Hardware Monitor

​Tool Call Trace (T)

​TUI Architecture

​Key Components

​Event Loop

​Next Steps

Interactive Mode

Models

Actions

Benchmarking

Build docs developers (and LLMs) love

Overview

Launching the TUI

Keyboard Shortcuts

Main Interface

Panel Shortcuts

Push-to-Talk Workflow

Waveform Visualizer

Text Input

Interactive Commands

Models Panel (M)

Navigation

Model States

Hot-Swap

Actions Panel (A)

Navigation

Action States

Direct Execution

Benchmarks Panel (B)

Available Suites

Running Benchmarks

RAG Panel (R)

Options

Drag-and-Drop

Cleanup Panel (D)

Status Bar

Left Side

Right Side

Hardware Monitor

Tool Call Trace (T)

TUI Architecture

Key Components

Event Loop

Next Steps