Skip to main content

​
Overview

Streaming allows your application to receive and display AI responses in real-time as they’re generated, rather than waiting for the complete response. This creates a much better user experience, especially for long responses.

​
Quick Start

​
Basic Streaming

Use the stream() method instead of send() to enable streaming: 
use Mateffy\Magic;

$messages = Magic::chat()
    ->prompt('Write a short story about a robot')
    ->stream();

// The response is streamed in real-time
// Access the complete text after streaming finishes
echo $messages->text();

​
Difference: stream() vs send()

// Without streaming - waits for complete response
$messages = Magic::chat()
    ->prompt('Explain quantum computing')
    ->send(); // Blocks until complete

// With streaming - receives response in chunks
$messages = Magic::chat()
    ->prompt('Explain quantum computing')
    ->stream(); // Returns chunks as they arrive

​
Real-time Progress

​
onMessageProgress Callback

Receive chunks as they arrive: 
Magic::chat()
    ->prompt('Write a long article about AI')
    ->onMessageProgress(function ($message) {
        // Called for each chunk during streaming
        echo $message->text();
        flush(); // Send to browser immediately
    })
    ->stream();

​
Complete Message Callback

Get notified when complete messages are received: 
Magic::chat()
    ->prompt('Explain machine learning')
    ->onMessage(function ($message) {
        // Called when a complete message is received
        logger()->info('Complete message received', [
            'type' => get_class($message),
            'content' => $message->text(),
        ]);
    })
    ->stream();

​
Streaming in Web Applications

​
Laravel HTTP Streaming

Stream responses directly to the browser: 
use Illuminate\Http\Request;
use Mateffy\Magic;

Route::get('/api/chat/stream', function (Request $request) {
    return response()->stream(function () use ($request) {
        $prompt = $request->input('prompt');
        
        Magic::chat()
            ->prompt($prompt)
            ->onMessageProgress(function ($message) {
                echo "data: " . json_encode([
                    'text' => $message->text(),
                ]) . "\n\n";
                
                if (ob_get_level() > 0) {
                    ob_flush();
                }
                flush();
            })
            ->stream();
        
        echo "data: [DONE]\n\n";
    }, 200, [
        'Content-Type' => 'text/event-stream',
        'Cache-Control' => 'no-cache',
        'X-Accel-Buffering' => 'no',
    ]);
});

​
JavaScript Client (Server-Sent Events)

Consume the stream in your frontend: 
const eventSource = new EventSource('/api/chat/stream?prompt=' + encodeURIComponent(prompt));
const outputDiv = document.getElementById('output');

eventSource.onmessage = (event) => {
    if (event.data === '[DONE]') {
        eventSource.close();
        return;
    }
    
    const data = JSON.parse(event.data);
    outputDiv.textContent += data.text;
};

eventSource.onerror = (error) => {
    console.error('Stream error:', error);
    eventSource.close();
};

​
React Example

Use streaming in a React component: 
import { useState, useEffect } from 'react';

function ChatInterface() {
    const [output, setOutput] = useState('');
    const [loading, setLoading] = useState(false);
    
    const streamResponse = async (prompt) => {
        setLoading(true);
        setOutput('');
        
        const eventSource = new EventSource(
            `/api/chat/stream?prompt=${encodeURIComponent(prompt)}`
        );
        
        eventSource.onmessage = (event) => {
            if (event.data === '[DONE]') {
                eventSource.close();
                setLoading(false);
                return;
            }
            
            const data = JSON.parse(event.data);
            setOutput(prev => prev + data.text);
        };
        
        eventSource.onerror = () => {
            eventSource.close();
            setLoading(false);
        };
    };
    
    return (
        <div>
            <div className="output">{output}</div>
            {loading && <div className="loading">Streaming...</div>}
        </div>
    );
}

​
Streaming with Tool Calls

Streaming works seamlessly with tool calls: 
use Mateffy\Magic;
use Mateffy\Magic\Chat\Messages\Step;

Magic::chat()
    ->messages([
        Step::user('What is the weather in Paris?'),
    ])
    ->tools([
        'get_weather' => function (string $city) {
            // This executes during the stream
            return WeatherAPI::get($city);
        },
    ])
    ->onMessageProgress(function ($message) {
        // Shows both text and tool call progress
        echo $message->text();
        flush();
    })
    ->stream();

​
Tool Call Progress

Monitor tool execution during streaming: 
Magic::chat()
    ->messages([Step::user('Search for flights to Paris')])
    ->tools([
        'search_flights' => function (string $destination) {
            echo "[Searching flights to {$destination}...]\n";
            flush();
            
            $results = FlightAPI::search($destination);
            
            echo "[Found {$results->count()} flights]\n";
            flush();
            
            return $results;
        },
    ])
    ->onMessageProgress(function ($message) {
        echo $message->text();
        flush();
    })
    ->stream();

​
Advanced Streaming

​
Custom Data Packets

Access raw streaming data: 
Magic::chat()
    ->prompt('Write a story')
    ->onDataPacket(function ($packet) {
        // Raw data from the LLM provider
        logger()->debug('Stream packet', ['packet' => $packet]);
    })
    ->stream();

​
Token Statistics During Streaming

Track token usage as the response streams: 
Magic::chat()
    ->prompt('Explain quantum physics')
    ->onTokenStats(function ($stats) {
        echo "Tokens used: {$stats['total_tokens']}\n";
        flush();
    })
    ->stream();

​
Error Handling

Handle errors during streaming: 
try {
    Magic::chat()
        ->prompt('Your prompt here')
        ->onMessageProgress(function ($message) {
            echo $message->text();
            flush();
        })
        ->stream();
} catch (\Throwable $e) {
    echo "Stream error: {$e->getMessage()}\n";
    logger()->error('Stream failed', [
        'error' => $e->getMessage(),
        'trace' => $e->getTraceAsString(),
    ]);
}

​
Complete Streaming Example

Here’s a complete Laravel controller with streaming: 
use Illuminate\Http\Request;
use Mateffy\Magic;
use Mateffy\Magic\Chat\Messages\Step;

class ChatController extends Controller
{
    public function stream(Request $request)
    {
        $validated = $request->validate([
            'messages' => 'required|array',
            'model' => 'string|nullable',
        ]);
        
        return response()->stream(function () use ($validated) {
            $messages = collect($validated['messages'])
                ->map(fn($msg) => match($msg['role']) {
                    'user' => Step::user($msg['content']),
                    'assistant' => Step::assistant($msg['content']),
                })
                ->toArray();
            
            try {
                Magic::chat()
                    ->model($validated['model'] ?? 'google/gemini-2.0-flash-lite')
                    ->messages($messages)
                    ->tools([
                        'search' => function (string $query) {
                            $this->sendEvent('tool_call', [
                                'tool' => 'search',
                                'query' => $query,
                            ]);
                            
                            $results = SearchService::search($query);
                            
                            $this->sendEvent('tool_result', [
                                'tool' => 'search',
                                'count' => count($results),
                            ]);
                            
                            return $results;
                        },
                    ])
                    ->onMessageProgress(function ($message) {
                        $this->sendEvent('message', [
                            'text' => $message->text(),
                        ]);
                    })
                    ->onTokenStats(function ($stats) {
                        $this->sendEvent('tokens', $stats);
                    })
                    ->stream();
                
                $this->sendEvent('done', ['status' => 'complete']);
            } catch (\Throwable $e) {
                $this->sendEvent('error', [
                    'message' => $e->getMessage(),
                ]);
            }
        }, 200, [
            'Content-Type' => 'text/event-stream',
            'Cache-Control' => 'no-cache',
            'X-Accel-Buffering' => 'no',
        ]);
    }
    
    private function sendEvent(string $type, array $data): void
    {
        echo "event: {$type}\n";
        echo "data: " . json_encode($data) . "\n\n";
        
        if (ob_get_level() > 0) {
            ob_flush();
        }
        flush();
    }
}

​
Frontend for Complete Example

class ChatClient {
    constructor(endpoint) {
        this.endpoint = endpoint;
        this.eventSource = null;
    }
    
    stream(messages, callbacks = {}) {
        const params = new URLSearchParams({
            messages: JSON.stringify(messages),
        });
        
        this.eventSource = new EventSource(`${this.endpoint}?${params}`);
        
        this.eventSource.addEventListener('message', (e) => {
            const data = JSON.parse(e.data);
            callbacks.onMessage?.(data);
        });
        
        this.eventSource.addEventListener('tool_call', (e) => {
            const data = JSON.parse(e.data);
            callbacks.onToolCall?.(data);
        });
        
        this.eventSource.addEventListener('tool_result', (e) => {
            const data = JSON.parse(e.data);
            callbacks.onToolResult?.(data);
        });
        
        this.eventSource.addEventListener('tokens', (e) => {
            const data = JSON.parse(e.data);
            callbacks.onTokens?.(data);
        });
        
        this.eventSource.addEventListener('done', (e) => {
            this.eventSource.close();
            callbacks.onDone?.();
        });
        
        this.eventSource.addEventListener('error', (e) => {
            const data = JSON.parse(e.data);
            this.eventSource.close();
            callbacks.onError?.(data);
        });
    }
    
    stop() {
        this.eventSource?.close();
    }
}

// Usage
const client = new ChatClient('/api/chat/stream');

client.stream(
    [
        { role: 'user', content: 'Tell me about Paris' },
    ],
    {
        onMessage: (data) => {
            document.getElementById('output').textContent += data.text;
        },
        onToolCall: (data) => {
            console.log('Tool called:', data.tool, data.query);
        },
        onToolResult: (data) => {
            console.log('Tool result:', data);
        },
        onTokens: (data) => {
            document.getElementById('token-count').textContent = 
                `Tokens: ${data.total_tokens}`;
        },
        onDone: () => {
            console.log('Stream complete');
        },
        onError: (data) => {
            console.error('Stream error:', data.message);
        },
    }
);

​
Best Practices

Always Use Streaming

Use stream() for better UX, especially for responses longer than a few sentences.

Flush Output

Call flush() after echoing content to ensure immediate browser delivery.

Handle Errors

Wrap streaming in try-catch and provide user feedback on errors.

Set Headers

Use proper headers for Server-Sent Events: text/event-stream and no-cache.

​
Performance Considerations

1

Disable Output Buffering

Set X-Accel-Buffering: no header to prevent proxy buffering.
2

Monitor Connection

Check if the client disconnects to stop unnecessary processing.
3

Optimize Chunk Size

Balance between responsiveness and overhead - very small chunks increase overhead.
4

Rate Limiting

Implement rate limiting on streaming endpoints to prevent abuse.

​
Troubleshooting

​
Stream Not Updating in Browser

Make sure you’re flushing output: 
echo $content;
if (ob_get_level() > 0) {
    ob_flush();
}
flush();

​
Nginx Buffering

Disable nginx buffering for streaming endpoints: 
location /api/chat/stream {
    proxy_pass http://backend;
    proxy_buffering off;
    proxy_cache off;
    proxy_set_header Connection '';
    proxy_http_version 1.1;
    chunked_transfer_encoding off;
}

​
Large Responses Timeout

Increase PHP execution time for long streams: 
set_time_limit(300); // 5 minutes

Magic::chat()
    ->prompt('Write a very long article')
    ->stream();

​
Next Steps

Chat API

Learn more about the Chat API features

Extraction

Extract structured data from documents

Embeddings

Generate embeddings for semantic search

API Reference

Explore the complete API documentation

Build docs developers (and LLMs) love

Get started for freeTalk to us