Long polling

The problem: polling is either slow or wasteful

You are building a support chat widget. A customer types a question, and an agent answers from a dashboard. The first version uses short polling (the previous lesson): every 3 seconds the widget sends GET /api/chats/88/messages?after=m-41 and the server answers at once, usually with an empty list.

It works, and the numbers are not great. A reply waits on average half the interval, 1.5 seconds, before the next poll picks it up, and up to 3 seconds in the worst case. In a chat, that pause feels slow. Meanwhile almost every request comes back empty: a conversation with one message every 30 seconds wastes 9 of every 10 polls.

Short polling gives you one knob, the interval, and both ends hurt. Poll every 500 ms and 10,000 open chats send 20,000 requests per second, almost all "nothing new". Poll every 10 seconds and replies arrive up to 10 seconds late.

One change: don't answer until there is news

Long polling removes that trade-off with one change on the server. When a request arrives and there is nothing new, the server does not answer "nothing". It keeps the request open and waits.

The moment a message arrives, the server answers the waiting request with that message. If nothing comes for a while, say 25 seconds, it answers with an empty response so the request does not hang forever.

The browser does its part by asking again immediately after every answer. There is no interval to tune. At almost every moment, each open chat has exactly one request waiting on the server, ready to carry the next message the instant it exists.

The result for our chat: replies arrive about one network trip after they are saved, often under 100 ms, and a quiet chat costs one request every 25 seconds instead of one every 3. You get delivery faster than any polling interval could give, with nearly as few requests as polling every 25 seconds.

Why learn it today

Long polling is older than WebSockets and server-sent events, and most new real-time features use those. So why a whole lesson on it?

• It is still the fallback. Socket.IO and SignalR both keep it for networks where streams and sockets fail.
• It is still a public API style. Telegram's Bot API and Dropbox's API offer long-poll endpoints, because any HTTP client can call them.
• Its problems are everyone's problems. Gaps between requests, missed and duplicated messages, held connections, proxy timeouts, and reconnect storms show up in every real-time system. Long polling shows them most clearly.

This lesson builds on the short polling lesson and does not repeat how to pause in hidden tabs or how ETag and 304 work. It covers what is new: holding the request, timeouts, ordering and missed messages, server cost, and where long polling fits today.

The mental model: always one request waiting

Picture a shop counter. Short polling is phoning the shop every hour to ask whether your parcel has arrived.

Long polling is walking to the counter and saying "I will wait here until it comes in". The moment the box arrives, the clerk hands it over, and you get straight back in line for the next one. If nothing comes by closing time, the clerk says "nothing today" and you line up again.

Keep two facts from that picture. First, the client always has one request parked on the server: an open request the server has not answered yet. Second, the parked request is the only way the server can reach the client. When it is answered, the server has no way to reach that client again until the next request arrives.

Every long poll ends one of three ways

A long-poll request is an ordinary HTTP request. What is special is only how long the server waits before it answers. Each one ends in exactly one of three ways, and the client handles each differently.

The state picture above is the whole client. It sends a request, waits, and handles one of the three endings. Two of them go straight back to sending; one goes through a backoff wait first. If you can draw this, you can write the client.

The wait moved from the client to the server

With short polling, the waiting happens in the browser: a setTimeout between requests. With long polling, the waiting happens on the server: a request sits in memory until there is news. That move is the source of every benefit and every cost in this lesson.

• Benefit: latency. News leaves the server the moment it exists. The delay is one trip from server to client, not half an interval.
• Benefit: fewer empty answers. A quiet client sends one request per timeout, not one per interval.
• Cost: held connections. The server keeps one open request per online client, all the time.
• Cost: gaps. Between an answer and the next request, nothing is parked. News created then needs somewhere to wait.

It also helps to see what long polling is not. It is not a stream: each answer ends the response, and the next event needs a new request. It is not two-way: the client sends its own messages with separate POST requests.

How it works, step by step

Let's follow one support chat for 40 seconds, with real values. The customer's widget has already shown every message up to ID 41, so it remembers 41 as its cursor: the ID of the newest message it has.

1. 0 ms. The widget sends GET /api/chats/88/events?after=41.
2. 55 ms. The request reaches the server (one trip takes about 55 ms on this network). The server checks its store for events in chat 88 after 41. There are none, so it parks the request in a list of waiters for chat 88 and starts a 25 second timer.
3. 12,400 ms. The agent sends a reply with POST /api/chats/88/messages. The server saves it as event 42, then looks at the waiters for chat 88 and finds our parked request.
4. 12,405 ms. The server answers the parked request: 200 with event 42 and cursor: 42. It clears the timer.
5. 12,460 ms. The answer reaches the browser. The widget shows the reply and saves 42 as its cursor.
6. 12,462 ms. The widget sends GET /api/chats/88/events?after=42 straight away. About 55 ms later the server finds nothing new and parks it with a fresh 25 second timer.
7. About 37,520 ms. Nothing happened for 25 seconds, so the timer fires and the server answers 204 No Content. The answer reaches the widget about 55 ms later, and it sends the next request at once.

The agent's reply reached the customer about 60 ms after it was saved. With 3 second short polling, the same reply would have arrived anywhere from 0 to 3 seconds later. And in those 40 seconds the widget sent 3 requests, where 3 second short polling would have sent 14.

What it looks like on the wire

Nothing in the request says "this is a long poll". It is a normal GET; the only clue is timing. In the browser's Network panel, it shows as a request that stays "pending" for seconds.

GET /api/chats/88/events?after=41 HTTP/1.1
Host: support.example.com
Accept: application/json
Cookie: session=...

(12.4 seconds pass)

HTTP/1.1 200 OK
Content-Type: application/json
Cache-Control: no-store

{"events":[{"id":42,"from":"agent","text":"Hi! Checking your order now."}],"cursor":42}

The server: park, wake, or time out

On the server, a long-poll handler does three jobs. If there is data after the cursor, answer at once; if not, park the request in a list keyed by topic (here, the chat ID). Then end it later in one of three ways: a new event arrives, the timer fires, or the client goes away. Here is a compact version for Node.js with an Express-style API.

type Waiter = { after: number; reply: (evs: ChatEvent[]) => void };
const parked = new Map<string, Set<Waiter>>(); // chat id -> waiting requests

app.get("/api/chats/:id/events", async (req, res) => {
  const chatId = req.params.id;
  const after = Number(req.query.after ?? 0);
  res.set("Cache-Control", "no-store");

  const waiters = parked.get(chatId) ?? new Set<Waiter>();
  parked.set(chatId, waiters);
  const finish = (send: () => void) => {
    if (!waiters.delete(waiter)) return; // already answered
    clearTimeout(timer);
    send();
  };
  const reply = (evs: ChatEvent[]) => finish(() => res.json({ events: evs, cursor: evs.at(-1)!.id }));
  const waiter: Waiter = { after, reply };
  waiters.add(waiter); // park first, so a wake during the read below is not missed
  const timer = setTimeout(() => finish(() => res.status(204).end()), 25_000);
  res.on("close", () => finish(() => {})); // sent, or the client left

  const ready = await events.after(chatId, after);
  if (ready.length) reply(ready); // news already waiting: answer at once
});

// Call this after a new event is saved, never before.
async function wake(chatId: string) {
  for (const w of parked.get(chatId) ?? []) {
    const evs = await events.after(chatId, w.after);
    if (evs.length) w.reply(evs);
  }
}

Three details matter more than they look. wake runs only after the event is saved, so a request that arrives a moment later finds the event in the store instead of parking and missing it. The handler parks the request before it reads the store, so an event saved during that read still wakes it. And finish checks that the waiter is still in the set, because the timer, a new event, and a closed connection can race, and a response can only be sent once.

The client: a loop with no sleep

The client is a loop. Unlike short polling, there is no setTimeout between healthy requests. It only sleeps after an error.

async function listen(chatId: string, stop: AbortSignal) {
  let cursor = loadCursor(chatId);
  let failures = 0;
  while (!stop.aborted) {
    try {
      const signal = AbortSignal.any([stop, AbortSignal.timeout(35_000)]);
      const res = await fetch(`/api/chats/${chatId}/events?after=${cursor}`, { signal });
      if (res.status === 200) {
        const body = await res.json();
        applyEvents(body.events); // ignores IDs it already has
        cursor = Math.max(cursor, body.cursor);
        saveCursor(chatId, cursor);
      } else if (res.status === 401 || res.status === 403) {
        return onSignedOut(); // the session ended: retrying will not help
      } else if (res.status !== 204) {
        throw new Error(`HTTP ${res.status}`);
      }
      failures = 0; // 200 or 204: go straight round again
    } catch {
      if (stop.aborted) return;
      await sleep(backoff(failures++)); // about 1 s, 2 s, 4 s, up to 30 s, with jitter
    }
  }
}

Try the numbers yourself. The lab below compares short polling and long polling for the same chat. Change how often messages arrive and watch both the delay and the request count.

Timeouts and the immediate re-request

Why does the server answer an empty 204 after 25 seconds at all? Why not hold the request until there is news, even if that takes an hour? Because the request does not travel straight from the browser to your code. It passes through things that do not like quiet connections.

Everything in the middle has an idle timeout

Between the browser and your handler sit load balancers, reverse proxies, CDNs, corporate proxies, and sometimes a hosting platform's router. Each one closes a connection that has been silent for too long, to free its own memory. A held long poll is silent by design, so it is exactly what they close.

If any hop closes the request first, the client sees an error instead of a clean empty answer, backs off, and the next message arrives seconds late. So your server must always speak first.

The timeout chain

Set three timeouts in a fixed order, and write the order down, because each is owned by a different team or config file.

1. Server hold time, the shortest: how long the handler waits before answering 204. Usually 20 to 30 seconds.
2. Every hop's idle timeout, longer than the hold time by a clear margin: 60 seconds or more on the load balancer and proxies you control.
3. Client request timeout, longer than the hold time plus a slow network: for a 25 second hold, about 35 seconds. It catches requests that silently died, for example when a laptop sleeps or the network changes.

A hold time that is too short only costs requests: at 5 seconds, a quiet client sends 12 a minute instead of 2 or 3. One that is too long costs reliability, because some proxy will cut it. That is why 20 to 30 seconds is the common choice.

Add a little randomness to the hold time on the server, like 20 to 30 seconds instead of exactly 25. Otherwise, clients that connected at the same moment (after a deploy, say) all time out at the same moment, forever, and the server sees a wave of requests every 25 seconds.

The empty answer: 204 or an empty list

The timeout answer can be 204 No Content with no body, or 200 with { "events": [], "cursor": 41 }. Both work; pick one and keep it.

204 is smaller and clearly means "nothing", but remember it has no body, so calling res.json() on it throws. Check the status first, as the client loop does. An empty 200 is simpler for clients that always parse JSON, and it lets the server send a fresh cursor or a hint such as pollAfterMs, like the server hint in the short polling lesson.

Asking again at once, and when not to

After a 200 or a 204, the client asks again with no delay. Any pause is time with nothing parked on the server, so it adds straight to the delay of the next message. This is the main habit to unlearn from short polling, where a delay between requests is the whole design.

Errors are the opposite. Group them into three kinds and treat each differently:

• Retry with backoff: network errors, 500, 502, 503, 504, and the client's own timeout. Wait about 1, 2, 4, 8 seconds, up to a cap like 30, with random jitter; reset after a success.
• Retry when told: 429 or 503 with a Retry-After header. Wait at least that long.
• Don't retry: 401 or 403 means the session ended; refresh the login or stop. 400 would only send the same bad request again.

One small optimization on the server side: when an event arrives, some servers wait a few milliseconds, say 20 to 50 ms, before answering, to collect any events that arrive right behind it. A burst of 10 messages then goes out in one response instead of 10 answer-and-re-request cycles. It trades a tiny bit of delay for far fewer round trips.

The gap: ordering and missed messages

Long polling has a blind spot. After the server answers, there is a short time when no request is parked: the answer travels to the browser, the browser handles it, and the next request travels back.

On a good connection that gap is 50 to 150 ms. On a train, it can be several seconds. What happens to an event created in the gap?

The naive server loses it

Imagine a server that only pushes events to whoever is waiting right now: when an event arrives, it hands it to every parked request and forgets it. The client's next request just says "give me what's new". An event that arrives while nobody is parked is handed to nobody.

The next request parks and waits for the event after it. The message is gone, and nothing reports an error.

Step through it in the lab below, first without a cursor and then with one. Watch event 43, which the agent sends while the answer for 42 is still on its way to the browser.

The fix: a cursor and a buffer

The fix has two halves, and you need both. The client sends a cursor with every request, ?after=42, meaning "I have everything up to 42". The server keeps recent events instead of forgetting them, and answers with everything after the cursor. Event 43 sits in the buffer during the gap, and the next request collects it at once, without parking.

The same pair fixes much bigger gaps. A laptop that slept for 10 minutes wakes up, sends its old cursor, and gets all 37 events it missed in one response. A tab that crashed and reloads can read its cursor from sessionStorage and pick up where it stopped. One mechanism covers a 50 ms gap and a 10 minute one.

A buffer cannot keep everything forever. Pick a retention window, such as the last 1,000 events per chat or the last 24 hours, and decide what happens when a cursor is older than the oldest event kept.

The server must not quietly return what it has, because the client would never know it missed events in between. Instead it answers with a clear signal, like 410 Gone or { "resync": true }. The client then fetches the full current state with a normal request and continues from the cursor that came with it.

What makes a good cursor

• Increasing, from one source. Use a sequence assigned in one place per topic: a database sequence, a Redis INCR, or a log offset. If two servers each count on their own, their numbers collide.
• Not a timestamp. Server clocks differ by a few milliseconds or more, and two events can share the same millisecond. ?after=1739871234567 will one day skip an event written in that same millisecond, or one written by a server whose clock was behind.
• Opaque for public APIs. A public API can hand out a string the client only echoes back, like Dropbox's cursor. The server can then change what is inside it, say from one number to a number per shard, without breaking old clients. In your own app, a plain number is simpler and lets the client compare cursors.

Duplicates are normal

Cursors stop events from being lost, but they also mean the same event can arrive twice. Suppose the server sends event 42, and the connection drops before the response reaches the browser. The client never saw it, so it retries with ?after=41 and gets 42 again. That is correct behavior: if an answer can be lost on the way, the only safe design is to resend it.

This is at-least-once delivery: every event arrives, sometimes more than once. The network alone cannot promise exactly once.

What you can do is make applying an event idempotent, meaning applying it twice has the same effect as once. Store messages in a map keyed by ID, and ignore an ID you already have. At-least-once delivery plus idempotent handling looks like exactly once to the user.

Out-of-order answers

Ordering breaks when a client has two long polls open at once: a retry fires while the old request is still alive, or a component mounts twice. The answers can arrive in either order, and an older answer can move the cursor backward.

• Run one loop per topic and abort the old request before starting a new one (the request races lesson covers AbortController patterns).
• Sort by the server's sequence number, never by arrival time.
• Only move the cursor forward: cursor = max(cursor, body.cursor).
• Give the user's own messages a client-generated ID when you POST them, so the copy that comes back through the long poll replaces the pending one instead of showing twice.

What a held connection costs the server

Short polling's cost is requests per second. Long polling's cost is concurrent requests: how many are open at the same moment. Because every online client keeps one request parked almost all the time, the number of open requests equals the number of online clients. 50,000 people with the page open means 50,000 requests sitting on your servers, all day.

Thread-per-request servers run out of workers

Many classic servers give each request its own worker, a thread or a process, until the response is sent. Tomcat's default pool is 200 threads; a PHP-FPM pool is often a few dozen processes. For normal requests that finish in 50 ms, 200 workers serve thousands of requests per second.

Now park long polls on them. Each held request keeps a worker busy doing nothing for 25 seconds. At 200 online users the pool is full, and every login, page load, and POST waits in the queue. The site looks down while the CPU sits almost idle, which makes this failure confusing the first time you see it.

Event-loop servers park requests cheaply

Servers built on an event loop or lightweight tasks, such as Node.js, Go, Elixir, Rust with async, or Python with asyncio, do not tie a thread to a waiting request. A parked request is a small object in memory: the socket, its buffers, and a callback. That is roughly kilobytes, not a whole thread with its own stack, so tens of thousands of parked requests per machine is ordinary. Java's virtual threads (Java 21) and async servlets bring the same benefit to the JVM.

Even then, the limits move rather than disappear. Each open connection is a file descriptor, and many Linux systems default to 1,024 per process, so raise the limit. Memory per connection adds up: at a rough estimate of 20 KB each (the real figure depends on your runtime and buffer sizes, so measure it), 50,000 connections take about 1 GB. And the load balancer in front holds the same number of connections, so check its limits and pricing too.

The cost per message

There is a second cost, paid per event. Every event ends a response, and the client sends a new request with all its headers and cookies, often 500 to 800 bytes. A 60 byte chat message can travel inside a kilobyte of HTTP. Over a stream or a socket, it costs its own bytes plus a few bytes of framing.

For a support chat with one message every 30 seconds, nobody will notice. For a live sports feed with 5 updates per second, long polling turns into a request every 200 ms per client, which is short polling at its worst. Long polling is efficient when events are rare compared with the round trip, and wasteful when they are frequent.

The cost in the browser: six connections

Over HTTP/1.1, browsers open at most 6 connections per host. A parked long poll holds one of them the whole time. Open the app in 6 tabs and every connection to that host is busy waiting, so images, API calls, and navigations to the same host stall until a long poll ends. Users report "the app freezes when I open many tabs".

• Serve it over HTTP/2 or HTTP/3. Many requests share one connection as separate streams, so a parked request does not block others. This is the real fix, and most CDNs and load balancers support it today.
• Share one long poll across tabs. Elect one tab to run the loop and pass events to the others with BroadcastChannel, or run the loop in a SharedWorker.
• Use a separate host name for the long-poll endpoint, such as events.example.com, so it uses its own pool of 6. An older trick that still works on HTTP/1.1, though the endpoint then needs CORS and its own cookie setup.

More than one server

With one server, wake finds the parked request in memory. With ten servers behind a load balancer, the agent's POST may land on server 3 while the customer's long poll is parked on server 7. Server 3 has nobody to wake.

The usual answer is a pub/sub channel between servers, such as Redis pub/sub, NATS, or Postgres LISTEN/NOTIFY. After saving the event, server 3 publishes "chat 88 has news". Every server subscribed to chat 88 wakes its own waiters, and each waiter reads the store with its own cursor. The pub/sub message only says "look again"; the data still comes from the store.

This design has a nice property. If a pub/sub message is lost, nothing is lost for good: the waiter still times out within 25 seconds, the client re-requests with its cursor, and the store returns the event. The cursor turns a missed wake-up into a delay instead of a missing message.

Failure modes in production

Long polling fails in a few repeatable ways. Each has a clear symptom, so you can spot it in your metrics before users report it.

The reconnect storm after a deploy

When a server restarts, every request parked on it is cut at the same moment. With 20,000 clients on that server, 20,000 requests fail together, and every client retries. If the first retry has no jitter, they all arrive in the same second, each with a cursor that may need a backlog read from the database. The servers that remain get a burst many times their normal request rate.

• Drain instead of cutting. On shutdown, stop accepting new long polls and answer the parked ones with 204 spread over 10 to 20 seconds. Clients see a normal timeout and reconnect to another server, a few at a time.
• Jitter the first retry too. Even the first retry after an error should wait a random delay, spread over a few seconds. The more clients a server holds, the wider the spread needs to be.
• Let the server push back. Under load, answer with 503 and Retry-After, and have clients honor it.

Connections that die silently

A laptop lid closes. A phone switches from Wi-Fi to mobile data. The TCP connection is dead, but neither side was told. The browser's fetch waits for an answer that will never come, and the server keeps a waiter for a client that is gone.

Both timeouts from the timeout chain clean this up. The server's 25 second hold timer eventually fires on the dead request and frees it. The client's 35 second AbortSignal.timeout aborts the dead fetch and starts a fresh one.

Without the client timeout, a tab can stay silently stuck until the user reloads. You can also listen for the online event and restart the loop at once when the network returns.

Caches and buffering proxies

A long poll is a GET, and GET responses can be cached. If a CDN or a proxy caches one answer, every client gets the same old event list forever, and the cursor never moves. Send Cache-Control: no-store on every long-poll response, and keep the CDN from caching the path.

Buffering is the opposite story, and here long polling wins. Some proxies and antivirus tools hold back a response until it is complete, which breaks streams like server-sent events because they never complete. A long-poll answer is a complete, normal response, so buffering only delays it by the time the buffering takes. This is one of the reasons long polling is still the fallback.

Expired sessions in the middle of a hold

Sessions expire while requests are parked, and the next request gets a 401. A loop that treats every error the same retries it forever. Handle 401 on its own path: refresh the token or send the user to sign in, then restart the loop.

Symptoms and causes

Where long polling fits today

In 2026, you rarely pick long polling as the main transport for a new feature. Server-sent events give you server-to-client streaming over plain HTTP, and WebSockets give you a two-way channel; both are supported in every current browser. Long polling lives on in three roles: the fallback, the public API, and the notify-then-fetch pattern.

The fallback

Real-time libraries keep long polling because it works wherever HTTP works. Two well-known ones use it in opposite ways.

• Socket.IO starts every connection with HTTP long polling, then tries to upgrade to a WebSocket in the background. Messages flow from the first moment, even if the upgrade later fails behind a strict proxy.
• ASP.NET Core SignalR goes the other way: it tries WebSockets first, then server-sent events, then long polling, and uses the first one that works.

Either way, your product design is the same: one message model, one cursor or ID scheme, and a transport that can change underneath. If you build your own, keep the event format and the cursor identical across transports, so the fallback is a different pipe, not a different protocol.

Public APIs

Long polling is a friendly shape for a public API, because any HTTP client in any language can use it, with no socket library and no streaming parser.

• Telegram Bot API. getUpdates takes an offset (the cursor: the next update ID you want) and a timeout in seconds. With a timeout above zero, the server holds the request until an update arrives. Asking with a higher offset confirms the earlier updates.
• Dropbox API. files/list_folder/longpoll takes a cursor and a timeout between 30 and 480 seconds (default 30), and Dropbox adds random jitter to the wait. It answers only changes: true or false, sometimes with a backoff field: the number of seconds to wait before the next long poll.

Notify, then fetch

Dropbox's endpoint shows a useful variant. The long poll does not carry the data. It only says "something changed", and the client then calls a normal endpoint (files/list_folder/continue) with its cursor to get the changes. The long-poll server stays tiny and stateless, holding only cheap waits, while the data path stays an ordinary, cacheable, rate-limited API.

You can use the same split: a long poll answers { "changed": ["inbox"] }, and the client refetches that query through its normal data layer.

Using it in a system design interview

Long polling comes up in two ways. Sometimes you choose it, for a notification badge or a public API. More often, you mention it as the fallback after choosing server-sent events or WebSockets, and the interviewer asks "what if a message arrives between two polls?". Either way, depth shows in the details, not the definition.

What to draw

1. A short sequence: client sends GET ?after=cursor, server holds, event arrives, server answers, client asks again at once. Label the 25 s timeout and the 204.
2. The cursor and buffer: a row of event IDs with the client's cursor and the oldest event kept, and what happens when the cursor is older than that.
3. If there are several servers: browsers, load balancer, long-poll servers, a pub/sub channel, and the store, with the "wake, then read with cursor" arrow.

What to say

• The trade. "Delivery in about one network trip, one request per 25 seconds when quiet, and one held request per online user."
• The timeout chain. "The server answers before any proxy gives up; the client gives up later, to catch dead connections."
• The gap. "A cursor and a buffer; at-least-once delivery with dedupe by ID; a resync for cursors that are too old."
• Server and retries. "An event-loop service of its own; data and timeouts loop at once, errors back off with jitter, 401 goes to sign-in."

How deep to go by level

Match the depth to the role. A junior answer that explains the mechanics clearly is a good answer. A senior answer is judged on what happens at scale and under failure, so spend your time there and keep the basics to one sentence.