Choosing a real-time technique

The problem: five tools, one feature

Your team is adding three features to a project management app in the same quarter. A small badge shows the number of unread notifications, a board view shows cards moving as teammates drag them, and a new AI assistant streams its answer word by word. A product manager asks one simple question: "Do we need WebSockets?"

The honest answer is "for which feature?". The badge changes a few times an hour and nobody minds if it is 30 seconds late. The board needs updates within a second, mostly from server to browser, and the assistant streams one long answer to one request. Three features, three different shapes of traffic, and very possibly three different techniques.

The five tools you already know

The earlier lessons in this module covered each technique on its own. This lesson does not teach them again. It puts them side by side and gives you a way to choose. A quick reminder of what each one is:

• Short polling (see the short polling lesson): the browser asks "anything new?" on a timer, for example every 10 seconds. Plain HTTP, simplest of all, and wasteful when nothing changes.
• Long polling (see the long polling lesson): the browser asks, and the server holds the request open until there is news or a timeout, usually 20 to 30 seconds. Near-instant delivery over plain HTTP.
• Server-sent events, or SSE (see the server-sent events lesson): one GET response that stays open and carries a stream of text events from server to browser. EventSource reconnects on its own and resumes with Last-Event-ID.
• WebSockets (see the WebSockets lesson): one upgraded connection where both sides send messages at any time, with 2 to 14 bytes of framing each. You own the protocol, the reconnects, and the scaling.
• WebRTC (covered in a later lesson): direct browser-to-browser connections for audio, video, and data, built for very low latency media. It is the only option here that can skip your server for the data itself.

Two neighbors also come up in these conversations. Webhooks (see the webhooks lesson) are server-to-server callbacks, so they never reach a browser directly. Push notifications (covered in a later lesson) reach a device even when your page is closed. Both often sit next to the five above in a real design.

How teams choose badly

Most bad choices come from picking the tool first and the requirement second. Three patterns show up again and again:

• WebSockets for everything. A notification badge gets a stateful connection layer, sticky load balancing, a pub/sub bus, and a reconnect protocol, for data that changes twice an hour. Polling every 30 seconds would have cost a few lines of code.
• Polling for everything. A live board polls every second to feel fast. With 20,000 open tabs, that is 20,000 requests per second, almost all of them answered with "nothing new".
• Ignoring the network in between. A WebSocket design works in the office and fails at a customer whose corporate proxy blocks the upgrade, with no fallback planned.

The fix is a short list of questions asked in a fixed order: who sends data, how often, how fresh it must be, how many clients there are, and what the network and device allow. The next chapter turns those questions into one picture.

The mental model: describe the traffic, then pick the pipe

Think of the five techniques as pipes of different sizes and shapes. You would not choose a pipe before you know what flows through it, how much, and in which direction. So the first step is always to describe the traffic of one feature in plain numbers. The choice usually falls out of that description.

Five questions that describe any real-time feature

1. Who sends? Only the server, only the client, or both. Count the messages per minute in each direction for one active user.
2. How fresh must it be? The longest delay a user would accept before the feature feels broken: 100 ms for a cursor, 1 second for a chat message, 30 seconds for a badge, minutes for a report.
3. How many receive the same data? One user (their own order), a small group (a chat channel), or a huge audience (a live score). This is called fan-out.
4. Where is the other end? Your server, or another user's device (audio, video, files between people).
5. Must it reach a closed page? If the user should hear about it with no tab open, no in-page technique can help.

Write the answers as a small profile. For the three features from the last chapter it looks like this:

The two axes that matter most

Of the five questions, two separate the techniques most clearly: how much the client sends and how fresh the data must be. Plot a feature on those two axes and you land in the zone of one technique. The other three questions then confirm the choice or move you to a neighbor.

Read the map from the bottom left. Data that can be 30 seconds or more late, with little sent by the client, belongs to short polling. Make it fresher, down to a few seconds or under one, and you move to SSE, with long polling as its fallback.

Add frequent messages from the client and you move to WebSockets. Audio and video between people need delays of tens of milliseconds, and WebRTC is the tool built for that.

"The client sends something" is not enough for a WebSocket

A common slip is to pick WebSockets because the client sends anything at all. Almost every feature has some client actions: posting a comment, moving a card, voting. If those happen a few times a minute, a normal POST is fine, and the updates can still come down an SSE stream.

The question is whether the client sends often, many times a minute, in small pieces where per-request headers would dominate. Typing indicators, cursor positions, and game inputs pass that test. A comment form does not.

The same goes for freshness. "Real-time" in a product spec often means "within a few seconds". Ask what delay would actually hurt. If the answer is 10 seconds, the cheapest technique that meets 10 seconds wins, and that is often polling.

The decision tree

The map gives a feeling for the zones. The decision tree turns it into a fixed order of questions you can walk in an interview, out loud, in under a minute. The order matters: the first questions catch the special cases that no amount of tuning can fix, and the last ones pick between close neighbors.

Step 0: does the server need to speak first at all?

Before the tree, check that you need real-time delivery. A product page's stock count only matters when someone looks at it or adds it to the cart, so a fresh fetch at that moment is enough. A settings page needs nothing. Real-time pays off only when something changes while the user is watching and seeing it late has a cost you can name: a missed chat message, a stale price, two people editing the same card.

Steps 1 and 2: the special cases

• Media or data between users' devices? Voice, video, screen sharing, or large files sent person to person need WebRTC. It is built for media, can send over UDP so a lost packet does not stall the ones behind it, and can connect browsers directly. The WebRTC lesson later in this module covers it. You still need a server channel (often a WebSocket) for signaling, the messages that set the call up.
• Must it reach a closed page? Use web push, covered in the push notifications lesson. It is never the only channel: once the user opens the page, an in-page technique takes over for live updates.

Steps 3 to 5: the main branch

• Does the client send often, many times a minute? Pick WebSockets. One connection carries both directions with a few bytes of framing per message.
• Must updates arrive within a few seconds? Pick SSE. It streams server-to-client over plain HTTP, reconnects on its own, and resumes with Last-Event-ID. Keep long polling as the fallback for networks that buffer streams.
• Otherwise, when a delay of tens of seconds is fine, pick short polling, with an interval that matches the freshness need, ETag checks so unchanged answers are a cheap 304, and a pause in hidden tabs.

Two adjustments come after the tree. If the same data goes to a huge audience, consider polling a CDN-cached URL even when the tree said SSE, because the cache absorbs the fan-out (the scale chapter shows the math). And if events start on another company's server, they come to your backend by webhook first and then reach the browser by whichever technique the tree picked.

Walking the tree for three features

Note the AI row. EventSource only sends GET requests with no body and no custom headers, so it cannot send a long prompt. AI chat clients usually send a POST with fetch and read the response body as a stream in the same text/event-stream format. It is still server-sent events on the wire, just without the EventSource object and its automatic reconnect.

Side by side: the comparison table

The tree tells you where to start. The table tells you what you are signing up for. Each column is a technique, and each row is a question an interviewer, or your future on-call self, will ask about it.

Reading the rows that people get wrong

Delay is not only the network trip. With short polling, an event that happens just after a poll waits the whole interval, so the average delay is half the interval and the worst is a full one. With long polling, the server answers at once, but there is a gap while the next request travels.

SSE and WebSockets push the moment the server knows, so their delay is close to one network trip, often 20 to 100 ms. The race below delivers the same four server events with each technique; change the polling interval and watch requests trade against waiting.

Server cost is a different kind of cost per column. Polling costs requests per second: CPU, logs, auth checks, and database reads, even when the answer is empty. Persistent techniques cost open connections: memory per connection (a few KB to tens of KB on an event-loop server), file descriptors, and load balancer slots. Neither is free, and which one is cheaper depends on how often data changes compared with how often you would ask.

Reconnect and resume is where the work hides. EventSource reconnects by itself and sends the last event id it saw. A raw WebSocket does neither: when it closes, it stays closed until your code opens a new one and tells the server where it left off. That code is easy to get subtly wrong, and it is the code that runs during your worst outages.

What the table leaves out

• Binary data. SSE is text only (UTF-8). If you need binary, encode it (base64 adds about 33 percent) or use WebSockets, which carry binary frames.
• Message ordering. Within one response or one connection, every technique here delivers in order, except WebRTC data channels, which can be set to unordered for lower delay. Across reconnects, ordering is yours to restore with sequence numbers.
• Caching. Only short polling plays well with HTTP caches and CDNs. A held request or an open stream is never cached.
• Authentication. Polling and long polling send cookies and headers like any request. EventSource sends cookies (cross-origin only with withCredentials: true) but no custom headers, so it relies on the session cookie or a short-lived ticket in the URL. WebSocket sends cookies on the handshake but no Authorization header, so teams use the cookie with an Origin check, a short-lived ticket in the URL, or a token in the first message (all covered in the WebSockets lesson).

Scale: requests, connections, and fan-out

At small scale every technique works, and the choice is about simplicity. At large scale the cost model decides. Polling turns users into requests per second, and persistent techniques turn users into open connections. A design review usually starts with one quick calculation of each.

The two numbers to compute

• Polling load = online clients divided by the interval. 100,000 tabs polling every 10 seconds is 10,000 requests per second. If data changes once a minute, 5 out of 6 of those answers say "nothing new".
• Connection count = online clients, all the time. 100,000 tabs means 100,000 open connections. An event-loop server (Node.js, Go, Java with Netty, async Python) can typically hold tens of thousands each (an assumption to measure on your own stack), so that is a handful of machines, plus the pub/sub layer that tells each machine which events its clients care about.

Connections are often cheaper than they sound, and requests are often more expensive. Each poll pays for TLS work, auth, routing, logging, and usually a database or cache read. An idle open connection costs memory and a heartbeat every 20 to 30 seconds. Try the numbers yourself:

Fan-out changes everything

Fan-out is how many clients receive the same message. A direct message goes to 2 people, a channel message to 50, and a goal in a cup final to 2 million people at the same second. The same technique can be cheap at a fan-out of 50 and painful at 2 million, because every message must be written to every open connection.

For huge fan-out of identical data, polling a CDN can beat any push technique. Put the score at GET /match/481/score with Cache-Control: public, max-age=2. Two million clients polling every 5 seconds send 400,000 requests per second, but they all land on the CDN.

Turn on request collapsing (the CDN sends one origin request for many clients waiting on the same expired URL), and each CDN location asks your origin about once every 2 seconds. Your servers then see a few hundred requests per second. The cost is a delay of up to about 7 seconds (2 seconds of cache age plus the 5 second interval), which a score page can accept and a chat cannot.

Reconnect storms

Persistent connections have a cost that polling does not: they all fail together. A deploy, a load balancer restart, or a short network cut closes 50,000 connections at once, and 50,000 clients try to reconnect in the same second. Each reconnect pays for a TLS handshake, an auth check, and a resubscribe, so the fleet can fall over just as it comes back.

The defenses are the same for every technique: exponential backoff with random jitter so reconnects spread over 10 to 30 seconds, a server that can ask clients to wait (retry: in SSE, Retry-After on HTTP), gradual deploys that drain connections a slice at a time, and resume by cursor so each client asks only for what it missed. The retries and backoff lesson covers the formulas.

The network in between: proxies, firewalls, and timeouts

Your connection does not go straight from the browser to your server. It may pass a home router, a carrier's network, a corporate proxy, a CDN, and your own load balancer. Each of them was built mostly for short request and answer traffic, and each can quietly break a long-lived one. This is the part of the choice that a demo on office Wi-Fi never shows.

Three ways the middle breaks real-time traffic

• Dropped upgrades. A WebSocket starts with an HTTP request carrying Upgrade: websocket. An old or strict proxy that does not understand it may strip the header or reject the request, and the socket never opens.
• Buffering. Some proxies, and some server setups such as nginx with its default proxy_buffering on, collect a response before passing it on. A stream of small SSE events then arrives in clumps, or not until the buffer fills. Set X-Accel-Buffering: no for nginx, and test through your real proxy chain.
• Idle timeouts. Load balancers and proxies close connections that send nothing for a while; 60 seconds is a common default, for example on AWS Application Load Balancers and in nginx's proxy_read_timeout. A quiet stream or a long-poll held too long is cut without warning.

Each break has a matching fix. Heartbeats every 15 to 30 seconds keep idle timeouts from firing: an SSE comment line, a WebSocket ping, or a long-poll timeout answered before 30 seconds. Disabling buffering fixes clumps. And for dropped upgrades, the main fix is encryption.

Why TLS fixes most of it

A proxy can only change what it can read. Over https:// and wss://, the traffic is encrypted end to end, so a proxy in the middle sees an opaque stream on port 443 and passes it along. That is why wss:// gets through far more networks than ws://. Always use wss://; plain ws:// is both unsafe and less likely to get through.

There is one exception. Many companies run TLS-inspecting proxies that decrypt traffic with their own certificate, which the company installs on its laptops. Those proxies can read and change everything, so they can still block upgrades, buffer streams, or cut connections after a fixed time. If your product sells to large companies, assume some customers sit behind one.

WebRTC and blocked UDP

WebRTC prefers UDP, because media can skip a lost packet instead of waiting for it. Many corporate and hotel networks block outgoing UDP, and some routers block direct paths between devices. When no direct path works, WebRTC uses a TURN relay: a server you run (or rent) that forwards the media.

The last resort is TURN over TCP or TLS on port 443, which looks like ordinary HTTPS to a firewall. It works almost anywhere, at the cost of more delay and your bandwidth bill. The WebRTC lesson covers STUN and TURN in detail.

Phones, batteries, and background tabs

On a laptop plugged into the wall, a request every few seconds costs nothing anyone notices. On a phone it can cost a visible share of the battery. The reason is not the bytes. It is the radio.

The radio tail

A phone's cellular radio has a low-power idle state and a high-power active state. Sending or receiving anything moves it to the active state, and after the transfer it stays active for several seconds in case more data comes, before it drops back to idle. That waiting period is called the tail. Its length is set by the mobile network and varies by carrier and generation (one widely cited textbook example uses 10 seconds), so a 300-byte poll can cost seconds of full-power radio time.

So the rule on mobile is fewer, larger wake-ups. Ten small messages spread over a minute cost far more than the same ten sent together. This applies to every technique: a poll every 15 seconds, a WebSocket ping every 15 seconds, and an SSE heartbeat every 15 seconds all wake the radio just as often. A persistent connection is not automatically kinder to the battery than polling; what matters is how often anything moves.

Moves that save battery

• Batch on the server. Send presence and counter updates at most once every few seconds, merged, instead of one message per change.
• Stretch heartbeats. Use the longest heartbeat your proxies allow. If the load balancer cuts at 60 seconds, 25 to 30 seconds is enough; 5 seconds is waste.
• Back off when idle. If the user has not touched the page for a minute, poll less often or slow the stream's updates.
• Stop in hidden tabs. Close the stream or pause polling when the page is hidden, and catch up with one request when it is visible again.
• Use push for closed pages. The device or browser already keeps one shared connection to its push service for all apps and sites, so web push adds no connection of yours.

Background tabs: the browser decides for you

Browsers already limit work in tabs the user cannot see. Timers in hidden tabs are slowed, in some cases to once a minute, and on phones a background tab may be frozen or discarded entirely. A polling loop built on setTimeout slows down without telling you. A WebSocket or SSE stream may stay open, be frozen, or be closed by the system, and none of this is consistent across browsers, so do not rely on it.

Take control instead. Listen for visibilitychange, pause on hidden, and on visible do one catch-up request from your cursor before resuming the live channel. This works the same for every technique, which is a good reason to put it in your shared real-time client rather than in each feature:

let hiddenTimer: ReturnType<typeof setTimeout> | undefined;

document.addEventListener("visibilitychange", () => {
  if (document.hidden) {
    hiddenTimer = setTimeout(() => realtime.pause(), 30_000);
  } else {
    clearTimeout(hiddenTimer);
    realtime.resume({ after: store.lastEventId }); // catch up, then go live
  }
});

Fallbacks: a ladder of transports

Because the failures live in the network, a production real-time layer rarely bets on one technique. It keeps a ladder: the best transport first, then simpler ones that get through more networks. The usual order is WebSocket, then SSE, then long polling, and short polling as the very last step. A one-way feature starts the ladder at SSE.

How to tell that a rung failed

You cannot ask the browser "will this work here?". You find out by trying, with deadlines:

• Open timeout. If a WebSocket has not fired open within about 5 to 10 seconds, treat it as blocked.
• First-message timeout. Have the server send a hello event right after connecting. If an SSE stream opens but no hello arrives within a few seconds, something is buffering it.
• Repeated early drops. If the connection opens but dies after the same short time, again and again, a proxy is cutting it. Two or three such drops in a row means step down.

When a rung fails, step down and remember it for the session, for example in sessionStorage, so every page load does not pay the same 10-second timeout again. Try the better rung again later, perhaps on the next session, because the user may have moved from the office network to home Wi-Fi.

One message format on every rung

A fallback is only safe if switching rungs never loses or repeats a message. That needs the same rules on every transport: each event has an increasing id, the client keeps the last id it handled, and every rung can resume with "give me everything after id N". With that, a step down is just a reconnect on a different pipe. Hide all of it behind one small interface, so components never know which rung is in use:

type RealtimeEvent = { id: number; topic: string; data: unknown };

interface Transport {
  connect(after: number): Promise<void>; // resume after this event id
  onEvent(handler: (e: RealtimeEvent) => void): void;
  send?(topic: string, data: unknown): void; // WebSocket only; others use fetch
  close(): void;
}

// Components only ever see this:
realtime.subscribe("board:42", (e) => applyBoardEvent(e.data));

Build, use a library, or buy

In real products

Real products mix techniques, one per feature, each the cheapest that meets that feature's needs. The table shows typical choices for well-known kinds of product. Companies change their stacks over time and rarely document every detail, so read these as common, publicly described patterns rather than exact current internals.

One app, many pipes

Go back to the project management app from the first chapter and put all its features together. A realistic design ends up with four techniques, each behind the same shared client, and only one stateful connection fleet:

• Badge and inbox: short polling every 60 seconds, paused when hidden, plus web push for mentions when the page is closed.
• Board: an SSE stream per open board, with moves sent by POST. If the team later adds live cursors, this one feature moves to a WebSocket.
• AI assistant: a streamed fetch response per question, closed when the answer ends.
• Integrations: webhooks from GitHub or a calendar arrive at the backend, which publishes them to the board's topic, so they reach the browser through the same SSE stream.

Lab: set the requirements, get the pick

Set the requirements of a feature you know, or one from an interview prompt. The lab walks the same tree as this lesson, names a primary technique and a fallback, and shows which answer decided it. Try changing one control at a time to find where the pick flips.

Using it in a system design interview

Almost every frontend system design prompt has a real-time part: a chat, a feed, notifications, a dashboard, a collaborative editor. Interviewers rarely care which technique you name. They care whether you can justify it from requirements, say what it costs, and say what happens when it fails.

When to bring it up

Ask the traffic questions during requirements, before you draw anything: "How fresh must this be? Does the user send often, or mostly watch? How many people see the same update? Should they hear about it with the tab closed?". The answers set up the choice, and asking them shows you know the choice depends on them.

Then make the choice in the API or interface part of the design, one feature at a time. Say it as a sentence with a reason and a cost: "Board updates go over SSE: they flow one way, need about a second, and SSE reconnects and resumes by itself. Moves are plain POSTs. If the corporate proxy buffers the stream, we fall back to long polling."

What to draw

• The browser's real-time client as one box that all features use, with the transport inside it.
• The connection tier (gateway servers holding the connections) separate from the API servers, with a pub/sub box between them and the services that produce events.
• An arrow for resume: the client sends its last event id on reconnect, and the server answers from a short event log.

How deep to go by level