# Streaming LLM Responses to the Browser with Server-Sent Events

> Stream LLM tokens from a Next.js route handler to the browser as server-sent events: SSE format, a fetch-based client, heartbeats, cancellation and fixes.

Source: https://www.epifive.com/blog/stream-llm-responses-server-sent-events · Published 2026-10-25 · EpiFive

## Key takeaways

- An SSE response is a text/event-stream body where each message is lines of field colon value, ended by a blank line.
- Return a Web ReadableStream from a Next.js route handler to stream tokens without extra libraries.
- Use fetch with a stream reader for POST-based chat, and EventSource only when a plain GET URL is enough.
- Send periodic comment lines to keep the connection alive and cancel the upstream call when the client disconnects.
- Without HTTP/2, browsers cap SSE connections at six per domain, so many open tabs can exhaust the limit.

To stream an LLM response to the browser with server-sent events (SSE), return a `text/event-stream` response from a Next.js route handler, write each token as an `event` or `data` message followed by a blank line, and read the stream in the browser with `EventSource` or with `fetch` and a stream reader. The user sees text appear as the model produces it instead of waiting for the full answer.

## Who this is for and what you will have

This guide is for developers building a chat or summarisation feature in Next.js who want tokens to appear progressively. By the end you will have a route handler that streams model output as SSE, a browser client that parses the stream and handles errors, a heartbeat to keep connections open, cancellation when the user leaves, and a list of the usual reasons streams arrive in one lump.

The examples use TypeScript, the Next.js App Router and the Anthropic TypeScript SDK as the model source. The streaming layer is provider-neutral: any source of text chunks can be wrapped the same way.

## Prerequisites

- A Next.js App Router project. Route Handlers are documented in the `route.md` reference that ships with Next.js, and recent versions differ from older tutorials, so read the reference for your installed version.
- An API key for your model provider, stored in an environment variable on the server only.
- For the Claude example: `npm install @anthropic-ai/sdk` and `ANTHROPIC_API_KEY` set.

## How SSE works

The Mozilla documentation defines the format. The server responds with the MIME type `text/event-stream`, and the stream must be UTF-8. Messages are separated by a pair of newline characters, in other words a blank line. Each line is `field: value`.

| Field | Meaning |
| --- | --- |
| `data` | The payload. Several consecutive `data:` lines are joined with newline characters. |
| `event` | An event type name. Typed events go to listeners added with `addEventListener()`; untyped messages go to `onmessage`. |
| `id` | Sets the last event ID of the `EventSource` object. |
| `retry` | The reconnection time in milliseconds, as an integer. |

A line that starts with a colon is a comment and is ignored, which makes comments useful as keep-alive pings. A stream that carries two token messages and a final marker looks like this:

```text
event: token
data: {"text":"Hello"}

event: token
data: {"text":" world"}

event: done
data: {}

```

SSE is one-way: the client cannot send events to the server over the same connection. Browsers can read it natively with `EventSource`, and any HTTP client that exposes the response body as a stream can read it too.

## Step 1: Create the route handler

In the App Router, a route handler lives in `route.ts` and exports functions named after HTTP methods. The handler below returns a `Response` whose body is a `ReadableStream`, a pattern the Next.js documentation shows for streaming with the underlying Web APIs.

Create `app/api/chat/route.ts`:

```ts
import Anthropic from "@anthropic-ai/sdk";

export const maxDuration = 60; // seconds; deployment platforms may apply this limit

const client = new Anthropic();
const encoder = new TextEncoder();

function sse(event: string, data: unknown) {
  return encoder.encode(`event: ${event}\ndata: ${JSON.stringify(data)}\n\n`);
}

export async function POST(request: Request) {
  const { prompt } = await request.json();

  let heartbeat: ReturnType<typeof setInterval>;

  const stream = new ReadableStream({
    cancel() {
      clearInterval(heartbeat); // the client disconnected
    },
    async start(controller) {
      heartbeat = setInterval(() => {
        controller.enqueue(encoder.encode(": keep-alive\n\n"));
      }, 15000);

      try {
        await client.messages
          .stream(
            {
              model: "claude-sonnet-5-5",
              max_tokens: 1024,
              messages: [{ role: "user", content: String(prompt) }],
            },
            { signal: request.signal },
          )
          .on("text", (text) => {
            controller.enqueue(sse("token", { text }));
          });
        controller.enqueue(sse("done", {}));
      } catch (err) {
        if (!request.signal.aborted) {
          controller.enqueue(sse("error", { message: "The model request failed." }));
        }
      } finally {
        clearInterval(heartbeat);
        if (!request.signal.aborted) controller.close();
      }
    },
  });

  return new Response(stream, {
    headers: {
      "Content-Type": "text/event-stream",
      "Cache-Control": "no-cache",
      "X-Accel-Buffering": "no",
    },
  });
}
```

Points worth noting:

- `maxDuration` is a route segment option for the maximum execution time in seconds. The Next.js documentation says deployment platforms can use the value from the build output to add execution limits, so check what your platform allows for streaming responses.
- The `.on("text", ...)` helper and the `messages.stream` method come from the Anthropic SDK streaming documentation, where awaiting the stream object waits for it to finish. The Claude streaming page does not show cancellation. The second argument that carries `signal` is a request option, so confirm in the SDK reference for your installed version that it accepts an abort signal, and use the SDK's own cancellation method if it does not.
- The `cancel()` callback on the `ReadableStream` runs when the reader cancels the stream, so the heartbeat timer stops and the handler never writes to a closed stream.
- The headers match the MDN example: `text/event-stream` is required, `Cache-Control: no-cache` is used in the example, and `X-Accel-Buffering: no` is set there too, although MDN notes these two are not listed as requirements.
- Errors are sent as a named `error` event with a safe message. Never forward raw provider errors or keys to the browser.

### Swapping in another provider

The streaming layer only needs an async source of strings. If your provider returns an async iterable, replace the `.on("text")` call with a `for await` loop that enqueues each chunk. Keep the surrounding `ReadableStream`, headers, heartbeat and cancellation the same.

## Step 2: Read the stream in the browser

`EventSource` is the built-in client. Its constructor takes a URL and an optional `withCredentials` flag, which means you cannot send a JSON body with it. Chat requests carry a prompt, so reading the stream with `fetch` is more practical. The client below parses the SSE format itself.

```ts
export async function streamChat(
  prompt: string,
  onToken: (text: string) => void,
  signal?: AbortSignal,
) {
  const res = await fetch("/api/chat", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ prompt }),
    signal,
  });
  if (!res.ok || !res.body) throw new Error(`Request failed: ${res.status}`);

  const reader = res.body.pipeThrough(new TextDecoderStream()).getReader();
  let buffer = "";

  while (true) {
    const { value, done } = await reader.read();
    if (done) break;
    buffer += value;

    let boundary: number;
    while ((boundary = buffer.indexOf("\n\n")) !== -1) {
      const raw = buffer.slice(0, boundary);
      buffer = buffer.slice(boundary + 2);

      let event = "message";
      const data: string[] = [];
      for (const line of raw.split("\n")) {
        if (line.startsWith(":")) continue; // comment / heartbeat
        if (line.startsWith("event:")) event = line.slice(6).trim();
        if (line.startsWith("data:")) data.push(line.slice(5).trimStart());
      }
      if (!data.length) continue;

      const payload = JSON.parse(data.join("\n"));
      if (event === "token") onToken(payload.text);
      if (event === "error") throw new Error(payload.message);
      if (event === "done") return;
    }
  }
}
```

Two parsing rules from the MDN format matter here: messages end at a blank line, and multiple `data:` lines in one message are joined with newline characters. Network chunks do not line up with message boundaries, so the loop accumulates a buffer and only parses complete messages.

Wire it into a client component with an `AbortController` so the user can stop generation:

```tsx
"use client";
import { useRef, useState } from "react";
import { streamChat } from "./stream-chat";

export function Chat() {
  const [text, setText] = useState("");
  const controller = useRef<AbortController | null>(null);

  async function send(prompt: string) {
    setText("");
    controller.current = new AbortController();
    try {
      await streamChat(prompt, (t) => setText((s) => s + t), controller.current.signal);
    } catch (e) {
      if ((e as Error).name !== "AbortError") setText((s) => s + "\n[error]");
    }
  }

  return (
    <section>
      <button onClick={() => send("Explain SSE in two sentences.")}>Ask</button>
      <button onClick={() => controller.current?.abort()}>Stop</button>
      <pre>{text}</pre>
    </section>
  );
}
```

### When EventSource is enough

If your endpoint can take its input from the URL, for example a job ID for a report already queued, `EventSource` gives you reconnection for free. By default, if the connection closes, the browser restarts it, and the `retry` field controls the wait. Call `.close()` when finished, otherwise the browser will keep reconnecting after your server ends the stream. For named events use `addEventListener`, and handle failures in `onerror`.

## Step 3: Keep the connection alive and clean up

### Heartbeats

Idle connections can be closed by proxies and load balancers. MDN recommends sending periodic comment lines, starting with a colon, to prevent timeouts. The route handler above sends `: keep-alive` every 15 seconds; choose an interval shorter than the idle timeout of the shortest hop on your path.

### Cancellation

When the user closes the tab or aborts the request, your server should stop paying for tokens. In the route handler, `request.signal` is the standard abort signal of the incoming `Request`. Pass it to the upstream call where the SDK accepts a signal, and avoid enqueuing after the stream is closed. MDN makes the same point for other languages: the server loop should notice that the client disconnected and stop, otherwise it keeps running.

### Connection limits

MDN warns that without HTTP/2, SSE is limited by the browser's maximum number of open connections, which is very low at 6 per browser and domain. Opening several tabs can exhaust it, and Chrome and Firefox have marked that as "Won't fix". With HTTP/2 the number of simultaneous streams is negotiated and defaults to 100. Serve your site over HTTP/2 when you expect users to keep several streaming tabs open.

## Step 4: Verify the stream

Test the endpoint outside the browser first, with buffering turned off:

```bash
curl -N -X POST http://localhost:3000/api/chat \
  -H "Content-Type: application/json" \
  -d '{"prompt":"Say hello in five words."}'
```

You should see `event: token` messages arrive progressively, then `event: done`. If the whole body appears at the end, the problem is on the server or an intermediary, not in your React code. In the browser, open the Network tab, select the request and watch the response body grow, or use the EventStream view where available.

## Troubleshooting

| Symptom | Likely cause | Fix |
| --- | --- | --- |
| All tokens appear at once at the end | A proxy or layer is buffering | Test with `curl -N`; set `X-Accel-Buffering: no`; check proxy and compression settings |
| Stream stops after about a minute | Platform or proxy timeout, or `maxDuration` | Raise `maxDuration` within your platform limit; keep heartbeats; reconnect or resume |
| Client receives garbled JSON | Message split across network chunks | Buffer until a blank line before parsing, as in the client above |
| Tokens keep generating after the tab closes | Upstream call not cancelled | Pass `request.signal` and stop on abort |
| Only the first few tabs stream | Six-connection limit without HTTP/2 | Serve over HTTP/2 or reduce open streams |
| `EventSource` reconnects forever | Stream ended and the browser restarts it | Send a `done` event and call `.close()` on receipt |
| Cached or repeated responses | Intermediary caching the stream | Send `Cache-Control: no-cache` |

If compression is the culprit in a self-hosted setup, the Next.js `compress` option controls gzip when you use `next start`. The documentation recommends leaving it enabled unless compression is configured elsewhere, for example in nginx.

## Checklist

- The response uses `text/event-stream`, UTF-8 and `Cache-Control: no-cache`.
- Each message ends with a blank line, and typed events use an `event:` field.
- The client buffers input and parses only complete messages.
- A heartbeat comment is sent more often than the shortest idle timeout.
- The upstream call is cancelled when the request aborts.
- Errors are sent as safe, named events, with no secrets or raw provider output.
- You tested with `curl -N` before debugging the UI.

For wiring the model side with tools, see [Claude Tool Use in Python: Build a Simple Agent Loop Step by Step](https://www.epifive.com/blog/claude-tool-use-python-agent-loop), and for the 429 errors you may hit when many users stream at once, see [Handling LLM API Rate Limits: Retry, Backoff and Queues in Python](https://www.epifive.com/blog/llm-api-rate-limits-retry-backoff-python).

## References

- Using server-sent events (MDN): https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events
- Streaming messages (Claude API): https://platform.claude.com/docs/en/build-with-claude/streaming
- Next.js route handlers (`route.md` in the installed documentation): https://nextjs.org/docs/app/api-reference/file-conventions/route
- Next.js `maxDuration` route segment config: https://nextjs.org/docs/app/api-reference/file-conventions/route-segment-config/maxDuration
- Next.js `compress` option: https://nextjs.org/docs/app/api-reference/config/next-config-js/compress

## Frequently asked questions

### What Content-Type does a server-sent events response need?

The response must use the MIME type text/event-stream and UTF-8 encoding. It is also common to send Cache-Control no-cache so that nothing between the server and the browser stores the stream.

### Should I use EventSource or fetch to read an LLM stream?

EventSource reconnects automatically but takes only a URL and a withCredentials option, so you cannot attach a JSON body. For chat requests that carry a prompt, a POST with fetch and a stream reader is the practical choice, and you parse the same event format yourself.

### Why do my events arrive all at once instead of token by token?

Something between your code and the browser is buffering the response. Check for a proxy or compression layer, send a flush-friendly response such as a ReadableStream, and set the X-Accel-Buffering no header, which MDN shows in its server example for nginx-style proxies.

### How do I stop generating tokens when the user closes the tab?

Listen for the request abort signal in the route handler and cancel the upstream model call when it fires. Without this, your server keeps paying for tokens nobody will read.