OpenAI Streaming API - Real-time GPT-4, Claude responses with Server-Sent Events. Implement streaming chat, reduce time-to-first-token for better UX in AI applications.

Streaming

Stream responses token-by-token for real-time output using Server-Sent Events (SSE).

Enable Streaming

Set stream: true in your request:

stream = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Tell me a story"}],
    stream=True
)

Example Request

from openai import OpenAI

client = OpenAI(
    base_url="https://api.uniterouter.ai/v1",
    api_key="your_api_key",
)

stream = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "user", "content": "Write a haiku about coding"}
    ],
    stream=True
)

for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.uniterouter.ai/v1",
  apiKey: process.env.UNITEROUTER_API_KEY,
});

const stream = await client.chat.completions.create({
  model: "gpt-4o",
  messages: [{ role: "user", content: "Write a haiku about coding" }],
  stream: true,
});

for await (const chunk of stream) {
  const content = chunk.choices[0]?.delta?.content;
  if (content) {
    process.stdout.write(content);
  }
}

curl https://api.uniterouter.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $UNITEROUTER_API_KEY" \
  -N \
  -d '{
    "model": "gpt-4o",
    "messages": [{"role": "user", "content": "Write a haiku"}],
    "stream": true
  }'

Stream Chunk Format

Each chunk is a Server-Sent Event:

data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","created":1699000000,"model":"gpt-4o","choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]}

data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","created":1699000000,"model":"gpt-4o","choices":[{"index":0,"delta":{"content":" world"},"finish_reason":null}]}

data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","created":1699000000,"model":"gpt-4o","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]

Collecting Full Response

full_response = ""
for chunk in stream:
    if chunk.choices[0].delta.content:
        full_response += chunk.choices[0].delta.content

print(full_response)

Streaming with Tools

Function calls are also streamed:

stream = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "What's the weather in Tokyo?"}],
    tools=tools,
    stream=True
)

tool_calls = []
for chunk in stream:
    delta = chunk.choices[0].delta
    
    if delta.tool_calls:
        for tool_call in delta.tool_calls:
            if tool_call.index >= len(tool_calls):
                tool_calls.append({"id": "", "function": {"name": "", "arguments": ""}})
            
            tc = tool_calls[tool_call.index]
            if tool_call.id:
                tc["id"] = tool_call.id
            if tool_call.function.name:
                tc["function"]["name"] = tool_call.function.name
            if tool_call.function.arguments:
                tc["function"]["arguments"] += tool_call.function.arguments

Stream Options

Get usage statistics with streaming:

stream = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello"}],
    stream=True,
    stream_options={"include_usage": True}
)

for chunk in stream:
    if chunk.usage:
        print(f"Tokens: {chunk.usage.total_tokens}")

Error Handling

try:
    stream = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": "Hello"}],
        stream=True
    )
    
    for chunk in stream:
        if chunk.choices[0].delta.content:
            print(chunk.choices[0].delta.content, end="")
            
except Exception as e:
    print(f"Stream error: {e}")

Web Integration

React Example

async function streamChat(message: string) {
  const response = await fetch("https://api.uniterouter.ai/v1/chat/completions", {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "Authorization": `Bearer ${API_KEY}`,
    },
    body: JSON.stringify({
      model: "gpt-4o",
      messages: [{ role: "user", content: message }],
      stream: true,
    }),
  });

  const reader = response.body?.getReader();
  const decoder = new TextDecoder();

  while (true) {
    const { done, value } = await reader!.read();
    if (done) break;
    
    const chunk = decoder.decode(value);
    const lines = chunk.split("\n").filter(line => line.startsWith("data: "));
    
    for (const line of lines) {
      const data = line.slice(6);
      if (data === "[DONE]") return;
      
      const json = JSON.parse(data);
      const content = json.choices[0]?.delta?.content;
      if (content) {
        // Update UI with content
        setResponse(prev => prev + content);
      }
    }
  }
}

Benefits

Benefit	Description
Lower Latency	First token appears immediately
Better UX	Users see progress in real-time
Memory Efficient	Process tokens as they arrive
Cancelable	Stop generation mid-stream

Chat Completions - Base API
Function Calling - Tool use with streaming

Streaming

On this page