Streaming server rendering

The problem: one slow call holds the whole page hostage

Picture a product page on a shop. To render it, the server needs four things: the product itself (20 ms from a cache), the price for this user (40 ms), the stock level (60 ms), and the reviews (900 ms on a bad day, because the reviews service is slow). With classic server rendering, which the lesson Server-side rendering covered, the server waits for all four before it sends a single byte of HTML.

So the user stares at a blank tab for about a second. The browser can't start downloading the CSS, the fonts, or the hero image, because it hasn't seen the head yet. The product name and photo were ready after 20 ms, but they sit in server memory, waiting for reviews that most users never scroll down to read.

All or nothing, twice

Classic server rendering is all or nothing in two places. On the server, nothing is sent until everything is fetched and rendered. In the browser, nothing is interactive until everything is hydrated: hydration (attaching event handlers and state to the server HTML) runs as one pass over the whole page, after the JavaScript for the whole page has downloaded.

• Server side: the slowest data call sets the time to first byte (TTFB, the time until the first byte of the response reaches the browser) for everyone.
• Browser side: the biggest component sets the time until the Buy button works, even if the Buy button itself is tiny.
• Both waits add up. A 900 ms reviews call plus a 400 ms hydration pass means the user can click Buy only after about 1.5 seconds, even on a fast phone.

The usual workaround was to drop the slow part from the server render and fetch it in the browser after load. That works, but now the reviews arrive through a second round trip, after the JavaScript runs, and you write two data paths for one page. Streaming gives you a cleaner answer: send what is ready now, and send the rest on the same response when it's ready.

What streaming changes

Streaming server rendering sends the HTML in pieces. The first piece holds the head and the page shell with the fast content, plus placeholders (such as a skeleton) where slow parts will go. The connection stays open. When the reviews are ready, the server sends their HTML as another piece, with a tiny inline script that swaps it into the placeholder.

On the browser side, selective hydration splits the single hydration pass into many small ones. Each streamed piece hydrates on its own, as soon as its code and HTML are both there, and the piece the user is clicking on hydrates first. The Buy button doesn't wait for the reviews.

The mental model: a shell with holes, filled on one open line

Keep one picture in your head: the page is a shell with holes. The shell is everything that is ready quickly: the head, the navigation, the product title and photo, the price. Each hole is a part that waits on slow data: the reviews, the recommendations. The server sends the shell at once, with a placeholder in each hole, and then fills the holes one by one over the same open response.

A restaurant works the same way. It doesn't hold your bread and salad until the slow-roasted main is done. It sets the table, brings each course when it's ready, and you start eating in minutes. The place settings are the placeholders: they show where each dish will go, so nothing on the table has to move when it arrives.

Three ideas that fit together

Streaming server rendering is really three separate ideas. They are often mixed up in interviews, so keep them apart:

1. Sending HTML in chunks. HTTP lets a server send a body bit by bit and keep the connection open. Browsers parse and paint HTML as it arrives. This part is plain HTTP and has worked since the 1990s.
2. Out-of-order streaming. The server sends each hole's content as soon as its data is ready, in whatever order that happens, and a tiny inline script moves it into the right hole. Without this, a slow part near the top would block everything below it.
3. Selective hydration. In the browser, each hole hydrates on its own, as soon as its HTML and its JavaScript are both there. If the user clicks a part that isn't hydrated yet, that part jumps the queue.

In React, one component ties all three together: Suspense. A Suspense boundary marks a hole. On the server it says where the stream may send a fallback and come back later; in the browser it says where hydration may be split. You draw the boundaries once and get both benefits.

What streaming does not change

Streaming doesn't make any data call faster. The reviews still take 900 ms, and the last byte of the page still arrives after about 900 ms plus rendering time. What changes is what the user sees during that time: the useful parts of the page, instead of a white tab.

It also doesn't remove the cost of hydration. The browser still downloads and runs the same JavaScript for the same components. Selective hydration only splits that work into smaller tasks and puts the right ones first. If you want to send less JavaScript, that is a different tool: see the lessons Islands and partial hydration and React Server Components, which build on streaming.

How HTML travels in pieces: chunks, frames, and flushing

Streaming needs nothing new from HTTP. A response doesn't have to declare its size up front, and a server can write the body in as many pieces as it likes. What differs between HTTP versions is how those pieces are framed on the wire.

HTTP/1.1: chunked transfer encoding

Over HTTP/1.1, a response normally has a Content-Length header so the browser knows where the body ends. A streamed page can't know its length in advance, so the server sends Transfer-Encoding: chunked instead. Each chunk is its size in hexadecimal, a line break, the bytes, and another line break. A chunk of size zero marks the end.

HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked

1f4
<!doctype html><html><head><link rel="stylesheet" href="/app.css">...
...<main><h1>Trail Runner 3</h1>...<template id="B:0"></template>...

9c
<div hidden id="S:0"><section>4.6 stars...</section></div>
<script>$RC("B:0","S:0")</script>

0

HTTP/2 and HTTP/3: DATA frames

HTTP/2 doesn't use chunked encoding at all. The specification forbids the Transfer-Encoding header, because every message is already split into frames. The headers travel in a HEADERS frame, the body in one or more DATA frames on the same stream, and the last frame carries an END_STREAM flag. HTTP/3 works the same way over QUIC.

So a streamed page over HTTP/2 is simply a stream whose DATA frames arrive with gaps between them. You don't set anything: your server code writes bytes, and the HTTP layer frames them. A common setup is an HTTP/1.1 app server behind a proxy or CDN that speaks HTTP/2 to the browser: the proxy drops the Transfer-Encoding header and passes each chunk on as DATA frames.

The browser parses as bytes arrive

Browsers have always parsed HTML incrementally. The parser builds the DOM from each chunk as it lands, and the preload scanner, a fast second parser that looks ahead for URLs, starts downloads for stylesheets, scripts, fonts, and images it finds. The browser can paint the content it has as soon as the stylesheets in the head have loaded, without waiting for the end of the document.

That is why flushing the head early is the first win of streaming, even before any Suspense. If the very first chunk contains the head, with its link rel="stylesheet", font preloads, and the preload for the hero image, those downloads start while the server is still fetching data. By the time the shell's HTML arrives, the CSS is often already there.

• Put render-blocking CSS and link rel="preload" for fonts and the hero image in the head, so they are in the first chunk.
• Load the client bundle without blocking the parser. React adds async to the scripts you pass as bootstrapScripts, so later chunks keep being parsed while the bundle downloads.
• Don't put a synchronous script in the shell: it stops the parser until it has downloaded and run, and later chunks wait behind it.

Suspense boundaries: where the stream may pause

Chunks are the transport. Something still has to decide what goes in which chunk. In React that is the Suspense boundary: a component with a fallback that wraps a part of the tree. When a component inside it is waiting for data, the server renders the fallback in its place, carries on with the rest of the page, and comes back to the boundary when the data is ready.

export default function ProductPage({ id }: { id: string }) {
  return (
    <Layout>
      <ProductHero id={id} />
      <BuyBox id={id} />
      <Suspense fallback={<ReviewsSkeleton />}>
        <Reviews id={id} />
      </Suspense>
      <Suspense fallback={<RailSkeleton />}>
        <AlsoBought id={id} />
      </Suspense>
    </Layout>
  );
}

The shell is everything outside a boundary

React calls the part of the page outside every Suspense boundary the shell. The shell must finish rendering before the first chunk can go out, because there is no fallback to show in its place. So any data the shell needs sets the TTFB. If ProductHero waits 300 ms for its data, the first chunk waits 300 ms too.

Boundaries nest. A boundary inside another boundary can't appear before its parent: the browser first gets the parent's content (with the inner fallback inside it), then the inner content. Nesting is how you build a sensible reveal order, such as the review summary first and the review list a moment later.

The server APIs: renderToPipeableStream and renderToReadableStream

React has two streaming server APIs. renderToPipeableStream is for Node.js streams. renderToReadableStream returns a web ReadableStream, for runtimes such as Deno, Bun, Cloudflare Workers, and other edge platforms. Both stream the same HTML; they differ in how they report progress.

import { renderToPipeableStream } from "react-dom/server";

app.get("/p/:id", (req, res) => {
  const { pipe, abort } = renderToPipeableStream(<App url={req.url} />, {
    bootstrapScripts: ["/client.js"],
    onShellReady() {
      res.statusCode = 200;
      res.setHeader("Content-Type", "text/html; charset=utf-8");
      pipe(res); // first chunk leaves now
    },
    onShellError() {
      res.statusCode = 500;
      res.send("<h1>Something went wrong</h1>");
    },
    onError(error) {
      console.error(error); // also fires for errors inside boundaries
    },
  });
  setTimeout(() => abort(), 10_000); // give up on slow boundaries
});

On the client, hydrateRoot(document, <App />) picks up the server HTML. You rarely write this code yourself: Next.js, React Router, and other frameworks call these APIs for you. But the callbacks explain behaviour you will see, such as why a 404 status is impossible after the shell is sent.

Out-of-order streaming: filling holes in any order

HTML is a stream of text from top to bottom. The simplest kind of streaming, called in-order streaming, sends the page in document order and flushes whenever it can. That helps the head, but a slow part in the middle still blocks everything below it: the footer can't be sent before the reviews, because the footer comes after them in the text.

Out-of-order streaming removes that limit. The server sends the shell with a marked placeholder for each hole, and then sends each hole's content at the end of the document, as soon as it's ready, in whatever order the data arrives. A tiny inline script moves each piece into its hole. eBay's Marko framework did this years before React; React 18 made it mainstream.

What React actually sends

Here is a simplified view of the two chunks for the product page. React marks a pending boundary with HTML comments and an empty template that has an id. Later, it sends the finished content inside a hidden div with a matching id, followed by a call to a small helper.

<!-- chunk 1 (about 120 ms): the shell -->
<main>
  <h1>Trail Runner 3</h1>
  <p>$129</p>
  <!--$?--><template id="B:0"></template>
  <div class="reviews-skeleton"></div><!--/$-->
</main>

<!-- chunk 2 (about 900 ms): appended at the end of body -->
<div hidden id="S:0">
  <section><h2>Reviews</h2><p>4.6 stars from 1,280 reviews</p></section>
</div>
<script>$RC("B:0", "S:0")</script>

• <!--$?--> means the boundary is still pending. After the swap it becomes <!--$-->, which tells hydration the content is real.
• The template element is an invisible anchor: it renders nothing, but $RC can find it by id.
• The content arrives in a div with the hidden attribute, so it never flashes at the bottom of the page before it's moved.
• The helper function is defined inline once, in the first chunk that needs it, so later chunks only carry the short call.

Step through it below. Notice that the recommendations rail finishes before the reviews, so its chunk arrives first and fills its hole first, even though it sits lower on the page.

The life of one boundary

Each boundary moves through a small set of states. On the server it is pending until its data resolves; then its content is streamed and swapped in, and in the browser it later hydrates. If its component throws on the server, React sends no content for it. It marks the boundary for client rendering (the comment becomes <!--$!-->), and the browser renders that part itself when the JavaScript loads.

Selective hydration: interactive piece by piece

Streaming gets pixels on screen early. But a button in server HTML does nothing until hydration has attached its click handler. Classic hydration was the second all-or-nothing wait, and streaming alone doesn't fix it. Selective hydration does.

How hydration used to work

Before React 18, ReactDOM.hydrate walked the whole tree in one go. It could start only after the JavaScript for every component on the page had loaded, and once started it ran as one long task: on a mid-range phone, 300 to 800 ms for a big page during which taps and scrolls had to wait. A tap on Buy before the code loaded did nothing at all, and a tap during the long task was handled only when the whole page had finished.

What changes with Suspense boundaries

• Hydrate before all the HTML has arrived. The shell hydrates as soon as its code is loaded, while the reviews are still streaming. Pending boundaries are skipped and hydrated later.
• Hydrate before all the code has arrived. If a boundary's component is code split with lazy, the rest of the page hydrates without waiting for that chunk of JavaScript.
• Hydrate in small tasks. React hydrates one boundary at a time and gives the main thread back to the browser between them, so scrolling and typing stay responsive.
• Hydrate what the user touches first. If the user clicks inside a boundary that isn't hydrated yet, React hydrates that boundary right away, ahead of the others, and then replays the click so it isn't lost.

The last point is the clever one. React listens for events at the root of the app. When a click lands on DOM that belongs to a dehydrated boundary, React raises that boundary to the top of its queue, hydrates it and the boundaries on the path above it synchronously, and then dispatches the click to the newly attached handler. Continuous events, such as hovering over a boundary or moving focus into it, also raise its priority, but less urgently than a click.

Try it in the lab. Pick a part to click while hydration is running and watch the queue change.

What it doesn't fix

Selective hydration changes the order of the work, not the amount. The browser still downloads, parses, and runs every component's code, and every boundary still hydrates in the end. If the page ships 400 KB of JavaScript, it still ships 400 KB. To ship less, you need islands or React Server Components, which leave static parts out of the bundle entirely.

Mismatches still matter, too. If the browser's first render doesn't match the server HTML (a date formatted in a different time zone, a random id), React throws away that boundary's server HTML and renders it again in the browser. With boundaries, the damage stays inside one boundary, but it costs time and can shift the layout.

Status codes, headers, and errors after the first flush

Streaming has one rule that surprises everyone the first time: the status code and headers leave with the first chunk, and after that they are locked. In HTTP/1.1 they are the first lines of the response; in HTTP/2 they are the HEADERS frame that comes before any DATA frame. The browser may already be painting your shell. There is no way to take them back.

• You can't turn a 200 into a 404 because the product turned out not to exist.
• You can't send a 302 redirect with a Location header because the user turned out to be logged out.
• You can't add a Set-Cookie header, for example to refresh a session, from a component that renders later.
• You can't change Cache-Control because a late boundary turned out to be personalised.

HTTP does have trailers, headers sent after the body, but browsers don't use them for status or cookies. So the fix is a design rule, not a header trick: anything that decides the status, redirects, cookies, or caching must be known before the shell is flushed.

Decide early: the usual pattern

Do the deciding checks before rendering starts, or in the shell. Frameworks give you a place for this: middleware, route loaders, or the page component itself, outside any boundary. The checks are usually fast (does this id exist, is there a valid session), so they cost little TTFB. The slow, decorative data goes inside boundaries, where a failure can only affect its own section.

// Next.js App Router: the existence check is in the shell
export default async function Page({ params }: { params: Promise<{ id: string }> }) {
  const { id } = await params;
  const product = await getProduct(id); // fast, cached
  if (!product) notFound(); // runs before the first flush: real 404

  return (
    <>
      <ProductHero product={product} />
      <Suspense fallback={<ReviewsSkeleton />}>
        <Reviews id={id} /> {/* slow, after the flush */}
      </Suspense>
    </>
  );
}

When the decision comes too late

Sometimes the deciding fact only shows up inside a boundary, and frameworks then fall back to in-page tools. Next.js, for example, documents that streamed responses keep their 200 status. A redirect() that runs after the flush inserts a meta tag that redirects in the browser, and a notFound() renders the not-found UI with a meta name="robots" content="noindex" tag so search engines leave the URL out. That works for users, but anything that only reads status codes, such as monitoring, sees a success, and some crawlers report a soft 404.

Errors inside and outside boundaries

What breaks streaming in production: buffers on the path

Streaming works only if every hop between your render code and the browser passes bytes on as they come. Any hop that collects the whole body before sending it, a buffer, quietly turns your streamed page back into a classic one. Nothing errors. The page just arrives all at once, and the TTFB is as slow as before.

The usual suspects

• Reverse proxies. nginx buffers proxied responses by default (proxy_buffering on). Turn it off for streamed routes with proxy_buffering off, or send the response header X-Accel-Buffering: no from the app, which nginx honours per response.
• Compression. Gzip and Brotli encoders hold data until they have enough to compress well. A compression middleware must flush after each write. Node's compression middleware adds a res.flush() method, and React's pipe calls flush() on the destination when it exists.
• Middleware that reads the body. Anything that rewrites, minifies, hashes, or logs the full HTML must wait for the end of it. That includes some CDN features that rewrite HTML at the edge.
• Serverless platforms. A function that returns a whole response object can't stream. AWS Lambda, for example, needs its separate response streaming mode for this; other platforms differ, so check the docs for your host.
• Networks you don't control. Some corporate proxies and antivirus tools buffer pages to scan them. You can't fix those, so the page must still work, just later, when everything arrives at once.

# nginx: let streamed pages through as they are written
location / {
  proxy_pass http://app;
  proxy_http_version 1.1;
  proxy_buffering off;
}

# Check from a terminal: -N turns off curl's own buffering
curl -N -s https://shop.example/p/42 | while IFS= read -r line; do
  printf '%s %s\n' "$(date +%T.%3N)" "${line:0:60}"
done

How you notice

In Chrome DevTools, open the Network panel and click the document request. In the Timing tab, a streamed page has a short "Waiting for server response" (that is the TTFB) and a long "Content Download" that matches your slowest boundary. A buffered page shows the opposite: a long wait and a download of a few milliseconds. A filmstrip from WebPageTest or the Performance panel shows it too: skeletons first, then sections filling in.

The cost on the server

A streamed response holds its connection and its render state open until the slowest boundary finishes. With classic rendering the server also waited, but the request still took the same total time, so the cost doesn't change much. What does change is that a hanging dependency now holds open responses that users are already looking at. Always set a time limit with abort(), so a stuck reviews service turns into client-rendered fallbacks instead of thousands of half-finished pages.

Effects on TTFB, FCP, LCP, CLS and INP, plus security and accessibility

Streaming moves several metrics at once, and not always in the same direction. Here is the product page from chapter 1 (shell data ready by 60 ms, reviews at 900 ms) on a mid-range phone, classic against streamed. The numbers are illustrative but realistic.

The LCP traps

A great TTFB doesn't guarantee a good LCP. Three things commonly go wrong. First, the LCP element is inside a slow boundary, so the first chunk paints only skeletons, and LCP is as late as before. Second, the hero image's img tag arrives in the shell, but nothing tells the browser it matters, so it downloads at low priority: add fetchpriority="high", or a link rel="preload" in the head.

Third, and least obvious: LCP is the largest element painted so far, and the browser keeps updating it until the user interacts. If a late boundary streams in a block bigger than the hero, such as a long review list above the fold, that block becomes the new LCP element and the metric moves to 900 ms. Keep late content below the fold or smaller than the main content.

Layout shift

Every swap replaces a fallback with real content. If the skeleton is 120 px tall and the reviews are 480 px, everything below jumps by 360 px. Size each fallback like the content it stands for, use min-height or aspect-ratio for variable parts, and avoid boundaries above the main content, where a shift moves what the user is reading.

Security: inline scripts and CSP

Out-of-order streaming depends on inline scripts like $RC(...). A strict Content Security Policy blocks inline scripts unless they carry a matching nonce, a random value created for each response. React accepts a nonce option in renderToPipeableStream and renderToReadableStream and adds it to every script it writes. Send the same value in the Content-Security-Policy: script-src 'nonce-...' header, which is easy, because the headers go out before any script does.

Caching changes too. If any boundary is personalised, the whole response is personalised, because it is one HTTP response. A plain HTTP cache can't store the shell and let your server stream the personal parts into the same response. Either keep the page public and fetch personal parts separately, accept Cache-Control: private, or use a framework feature built for this split, such as Partial Prerendering (covered in the last chapter).

Accessibility

• Mark each skeleton as loading: give its region an accessible name such as "Loading reviews" and aria-busy="true", so screen reader users know something is coming.
• Don't announce every chunk through a live region. Three sections arriving means three interruptions. Announce only content the user asked for, such as search results.
• Never move focus when a section arrives. The swap puts content in its document position, so reading order stays correct without extra work.
• Respect prefers-reduced-motion for shimmering skeletons.

In real products, frameworks, and the interview

You will rarely call renderToPipeableStream yourself. Every major meta-framework now streams, each with its own way to mark the holes. Knowing the names helps in interviews, because the interviewer may use their framework's words.

Where the idea came from

Facebook's BigPipe (2010) split the home page into pagelets such as the news feed, friend requests, and ads. The server flushed the page layout first, then each pagelet as a script block carrying its HTML and the CSS and JavaScript it needed, and the browser placed each one in its slot. Facebook's 2010 post reported that, in its own lab tests, this roughly halved the latency users perceived in most browsers. Today's out-of-order streaming is the same idea, driven by components instead of hand-written pagelets.

A static shell with streamed holes

The newest step combines streaming with static generation. Next.js's Partial Prerendering (part of its Cache Components model in recent versions) builds the shell of a route ahead of time, serves it from the cache, and streams the dynamic holes, such as the cart or personal recommendations, into the same response. On a host that supports it, you get a CDN-speed TTFB for the shell and fresh data for the holes. The lesson Static generation and incremental regeneration covers the static half.

Using it in a system design interview

Bring streaming up when the page you are designing mixes fast, important data with slow, secondary data: product pages, news articles with comments, dashboards with many widgets, search results with facets. Say it in one sentence first: "I'd server render and stream: the shell with the main content goes out in the first chunk, and the slow sections stream into Suspense boundaries."

1. Draw the page as a wireframe and circle the boundaries: what's in the shell, what's in each hole, and which element is likely the LCP element.
2. Name the data behind each part with a rough latency, so the reason for each boundary is visible.
3. Say what happens on the wire: head first so CSS and fonts start, shell next, holes out of order with swap scripts.
4. Cover the browser side: selective hydration, the Buy button in the shell, heavy widgets lazy loaded in their own boundaries.
5. Name the traps before the interviewer does: status codes locked at the flush, buffering proxies, layout shift from skeletons, CSP nonces.