Design a client-side rate-limited action queue

Requirements: who sets the limit, and what must never happen

A rate limit is a rule on the server: this caller may make at most N requests in some window of time. Break the rule and the server answers 429 Too Many Requests, and some APIs block you for a while longer.

Real numbers vary a lot. GitHub allows a signed-in user 5,000 REST requests per hour (60 without signing in). Notion gives each connection 180 requests per minute on most plans and 600 on Business and Enterprise. HubSpot lets a privately distributed app make 100 to 190 requests every 10 seconds, depending on the plan, and Shopify's legacy REST Admin API gives each app a bucket of 40 requests per store that drains at 2 per second.

A user clicking makes a few requests a minute, so most apps never notice. The problem appears when the client makes many requests on its own: a bulk edit, an import, a sync after being offline, a fast typist in a chat. Then the browser needs a queue that holds actions, a scheduler that decides when each one may go, and a way to show the user what is waiting and why.

Nearby guides own the rest, so we link instead of repeating: the progress bar queue guide (item states, concurrency K, progress bars), the offline sync engine guide (queueing writes made offline), the analytics SDK guide (batching and sending on page hide), the logging SDK guide (rate limiting error events) and the file uploader guide (chunked uploads and resume). This guide is about one thing: staying under a rate limit you do not control, without losing or doubling any action.

Questions to ask first

Functional requirements

• Accept actions from any part of the app (enqueue), at any rate, and never drop one silently.
• Send them no faster than the limit, and with no more than K requests in flight at once.
• Read the server's signals (429, Retry-After, rate limit headers) and slow everyone down, not just the request that failed.
• Retry the errors that are worth retrying, with backoff, and give up on the rest with a clear reason.
• Run a user's direct action (save this one contact) ahead of a bulk job, without starving the bulk job.
• Keep edits to the same record in order, and merge edits that make earlier ones pointless.
• Keep the queue across reloads, and share one limit across tabs.
• Show status: queued, sending, waiting for the limit (with a countdown), done, failed. Let the user cancel queued work.

Non-functional requirements

• Never above the limit. Aim for at most 1 429 per 1,000 requests in steady state. Each 429 is wasted time and, on some APIs, a longer ban.
• Close to the limit. A bulk job of 2,000 actions should finish in under 4 minutes (about 220 seconds at 90% of 10 per second), not 10 minutes. Being too careful is also a bug.
• Fast feedback. A single user action should start within 100 ms if there is room in the budget, and the UI should update at once with an optimistic value.
• No lost work and no doubled work. Every action ends as done, failed with a reason, or cancelled by the user. A create is applied at most once.
• Cheap. No polling: one timer, set to the next moment a send is allowed.

Out of scope

• Enforcing the limit. The server enforces it; the client only tries to stay under it. A client limiter is a courtesy and a speed-up, never a security control.
• The progress bar UI itself (see the progress bar queue guide) and the offline sync protocol (see the offline sync engine guide).
• Server-side rate limiting of our own API. We mention its algorithms because the client mirrors them, but we do not design a server limiter.

What makes this hard

Start with the code most people write first. The user picks 2,000 contacts, and the app runs Promise.all(contacts.map(updateOwner)). The browser opens as many connections as it can (about 6 per host on HTTP/1.1, many more streams on HTTP/2), and the server sees hundreds of requests in the first second.

Against a limit of 10 per second, about 10 succeed and the rest get 429. If each failed call retries at once, the next second looks the same. This pattern is called a retry storm: the client spends its whole budget on requests that are rejected.

Fixing that is not just adding a delay. Six things make this problem hard, and every later chapter answers one of them.

1. Two limits, not one. Servers limit how many requests you start per window and how many are open at once (GitHub documents both). Counting only one still gets 429 from the other.
2. The client cannot see the true count. Other tabs, devices and server jobs may spend the same budget. The client's counter is a guess; the server's headers are the truth.
3. Time is tricky. Windows reset when the server decides, clocks differ, and background tabs have their timers slowed.
4. Retries can do harm. Fast retries deepen overload, a retried create can make a duplicate, and a retried 400 never works.
5. Not all actions are equal. One edited field should save now, even during a bulk job, while a background sync can wait. Two edits to one record must not swap order.
6. The browser is not a server. Reloads wipe memory, and two tabs that each run a queue send twice the allowed rate.

First question: can we send fewer requests?

Before pacing requests, try to need fewer of them. Many APIs with tight limits offer batch endpoints: Salesforce's sObject Collections resource updates up to 200 records in one call, and HubSpot has batch endpoints for contacts. As 10 calls of 200, the 2,000 contacts take 1 second instead of 200. The Google Sheets guide applies the same idea to a huge paste: one range operation instead of thousands of cell edits.

Batching does not remove the queue. Batches still count against the limit, some records in a batch fail while others succeed, and many endpoints (create one page, send one message) have no batch form. So the design treats a batch as one job with many results; the rest of this guide uses single requests to keep the examples clear.

Rate and concurrency: two different limits

A rate limit counts requests you *start* in a window: 10 per second, 5,000 per hour. A concurrency limit counts requests *open at the same moment*: at most 4 in flight. The progress bar queue guide is built around the second. A rate-limited queue needs both, and they interact.

The link between them is a rule called Little's law: the average number of requests in flight equals the rate times the average time each one takes. At 10 requests per second and 300 ms per request, about 3 are in flight on average.

Now the server slows down and each request takes 2 seconds. To keep 10 per second you would need 20 in flight. With a concurrency cap of 4, the real rate drops to 4 divided by 2 seconds, which is 2 per second. Which limit is in charge changes from minute to minute with latency.

The scheduler loop

The scheduler is a small loop that runs when a job is added, a request finishes, or its timer fires. It never polls: when it cannot start anything, it sets one setTimeout for the next moment a start becomes possible.

function pump() {
  clearTimeout(wakeTimer);
  while (inFlight < maxConcurrent && queue.hasReady(now())) {
    const waitMs = limiter.msUntilAllowed(now());
    if (waitMs > 0) {
      wakeTimer = setTimeout(pump, waitMs);
      return;
    }
    const job = queue.takeNext(now());
    limiter.consume(now());
    inFlight++;
    send(job).finally(() => {
      inFlight--;
      pump();
    });
  }
  const nextReady = queue.nextReadyAt(); // a job waiting for its retry time
  if (nextReady) wakeTimer = setTimeout(pump, nextReady - now());
}

Three details matter. First, now() is injected rather than calling Date.now() directly, so tests can move time by hand (in production, wall-clock time suits values saved across reloads, like a retry time, while performance.now() suits short intervals because it never jumps). Second, the limiter is consumed when the request *starts*, because that is when the server counts it. Third, a finished request calls pump() again, so a freed slot is used at once.

Choosing the algorithm: windows, logs and buckets

The rate gate answers one question: may I start a request now, and if not, how long until I may? The client does best when it mirrors the server's algorithm, so we look at each one and its behaviour at the edges, then pick.

Fixed window

Cut time into windows on the clock and keep one counter per window; allow while it is under N. GitHub's hourly limit looks like this to a client: a count and a reset time, x-ratelimit-reset. Notion's per-connection limit is also a budget per 60-second window.

The weakness is the window edge. With 100 per minute, a client can send 100 at 0:59 and 100 more at 1:00: 200 requests in 2 seconds, while each window still sees only 100.

A server that uses fixed windows accepts that. A server that uses a sliding window rejects half of it. So a fixed-window client facing a sliding-window server can get a burst of 429s whenever its traffic bunches up around a rollover.

Sliding window log and sliding window counter

A sliding window log keeps the start time of each recent request. To decide, drop timestamps older than the window, then allow if fewer than N remain. The answer is exact: no stretch of length W ever holds more than N requests, wherever you put it. The cost is one number per request in the window: 10 numbers for 10 per second, 5,000 for 5,000 per hour, still tiny for a browser.

A sliding window counter keeps two counters, this window and the last, and estimates the sliding count as the current count plus the previous count times the share of the previous window still inside. Cloudflare described it for its edge limiter because it needs little memory per key, a saving a browser does not need.

Token bucket

A token bucket holds up to C tokens (the capacity) and gains r tokens per second (the refill rate). Each request takes one token; with no token, you wait. A full bucket allows a burst of C at once, then a steady r per second, so two numbers describe both how bursty and how fast. That is why it is the most common choice for client libraries.

class TokenBucket {
  private tokens: number;
  private last: number;
  constructor(private capacity: number, private perSec: number, now: number) {
    this.tokens = capacity;
    this.last = now;
  }
  private refill(now: number) {
    const gained = ((now - this.last) / 1000) * this.perSec;
    this.tokens = Math.min(this.capacity, this.tokens + gained);
    this.last = now;
  }
  msUntilAllowed(now: number): number {
    this.refill(now);
    if (this.tokens >= 1) return 0;
    return Math.ceil(((1 - this.tokens) / this.perSec) * 1000);
  }
  consume(now: number) {
    this.refill(now);
    this.tokens -= 1;
  }
}

Note the trap when mirroring a window limit. A bucket with C = 10 and r = 10 per second can send 10 at time 0 and 9 more during the next 0.9 seconds: 19 inside one second, which a server allowing 10 per second answers with 429. In general, a token bucket allows up to C + r times W requests in any stretch of W seconds. To stay under N per W, keep C small (1 or 2) for smooth pacing, or use a sliding log.

Leaky bucket and GCRA

The leaky bucket has two versions. As a queue, requests drop into a bucket and leave through a hole at a constant rate: the output is perfectly smooth, and extra requests wait or overflow. As a meter, the bucket is a counter that drains at a constant rate; a request adds one, and if the bucket would overflow, the request is rejected. A meter-style leaky bucket allows exactly the same traffic as a token bucket with the same size and rate, just counted the other way round.

Shopify's legacy REST Admin API is the meter kind: 40 requests per app per store, draining at 2 per second (400 and 20 on Shopify Plus). It reports the level in a header such as X-Shopify-Shop-Api-Call-Limit: 32/40.

GCRA (the generic cell rate algorithm, from ATM networks) stores a leaky bucket as one timestamp: the theoretical arrival time of the next request. Servers like it for that single number per key; a client can treat it as a token bucket.

Try it below. Pick an algorithm and a traffic pattern, and watch which requests the server accepts. Then turn off Retry-After and see how a client that retries at once makes things worse.

Learning the limit from the server

The documented limit is a starting guess. The real budget depends on who else spends it, the customer's plan, and how busy the server is (Shopify says it may temporarily reduce limits). So the client corrects itself from three signals: the 429 status, the Retry-After header, and headers that report the remaining budget on every reply.

429 Too Many Requests and Retry-After

Status 429 was defined in RFC 6585 (2012): the user has sent too many requests in a given amount of time. The response may include Retry-After, which RFC 9110 defines in two forms: a number of seconds (Retry-After: 30) or an HTTP date (Retry-After: Wed, 21 Oct 2026 07:28:00 GMT).

The same header may come with 503 Service Unavailable, when the server is overloaded rather than limiting you on purpose. Some APIs use 403 for limits instead: GitHub may answer 403 or 429 when you hit a primary or secondary limit, so the client must check the headers and message, not only the status.

function retryAfterMs(res: Response): number | null {
  const raw = res.headers.get("retry-after");
  if (!raw) return null;
  if (/^\d+$/.test(raw.trim())) return Number(raw) * 1000;
  const until = Date.parse(raw);
  if (Number.isNaN(until)) return null;
  const serverNow = Date.parse(res.headers.get("date") ?? "") || Date.now();
  return Math.max(0, until - serverNow);
}

Headers that report the budget

Better than reacting to 429 is never getting it. Many APIs report the budget on every reply. There is no final standard yet, so the client needs small adapters:

Each adapter turns its headers into one shape: { remaining, resetInMs, limit, windowMs }. The limiter's sync method then corrects itself. If the server reports fewer remaining than the local bucket holds, lower the local count; if remaining is 0, pause until the reset. If the policy reports a quota and window, set the refill rate to 90% of quota divided by window.

Always take the more careful of the local and server views. The local view knows about requests still in flight that the server has not counted yet; the server knows about every other caller.

Back off globally, not per request

The most common bug: each request handles its own 429 by sleeping and retrying, while the scheduler keeps starting new ones that also fail. The limit belongs to the account, so the response must too. On the first 429, the whole limiter for that budget closes until the Retry-After time (or for a backoff delay when there is no header).

The job that failed goes back to the front of its lane with its attempt count unchanged, because it was never processed. Requests already in flight finish or fail on their own; their 429s extend the pause only if they ask for a later time.

When the pause ends, do not release everything at once. Clients that share a reset moment (every tab or user told to wait until the same time) would all resume together, so add a small random delay of 0 to 1 second to the resume time, and let the token bucket restart from empty so traffic ramps up instead of bursting.

Retries, backoff and idempotency

A 429 is handled by the global pause. Other failures need a per-job decision: retry, and when, or give up and tell the user. The retries and backoff lesson covers the basics; here a careless retry policy also spends the shared budget that other jobs need.

Which errors to retry

Exponential backoff with full jitter

Backoff means each retry waits longer: 0.5 s, 1 s, 2 s, 4 s, up to a cap. Jitter means the wait is random. Without it, every client that failed at the same moment retries at the same moment, and the spikes repeat.

The AWS Architecture Blog post "Exponential Backoff And Jitter" compared variants. Full jitter, a random wait anywhere between 0 and the exponential value, did the least total work and took only slightly longer than the fastest variant. Our queue has a second reason to jitter: one tab may retry 20 jobs that failed together during a short outage.

const BASE_MS = 500;
const CAP_MS = 30_000;

function backoffMs(attempt: number, retryAfter: number | null): number {
  if (retryAfter !== null) return retryAfter;
  const ceiling = Math.min(CAP_MS, BASE_MS * 2 ** attempt);
  return Math.random() * ceiling;
}

// attempt 0: 0 to 0.5 s, attempt 3: 0 to 4 s, attempt 6+: 0 to 30 s

A retry budget

Per-job limits are not enough. If the server is down, 2,000 jobs times 5 attempts is 10,000 requests that all fail, and each one spends rate budget. Google's SRE book describes a retry budget: retry only while retries are under about 10% of all requests, and cap attempts per request at about 3.

Past the budget, stop retrying non-429 errors, pause the queue, and tell the user the service is having trouble. The user, not the client, decides whether to try again later.

Idempotency keys: retrying a create safely

A request is idempotent if doing it twice has the same effect as once. Setting a contact's owner to Priya is idempotent; creating a note is not, because two creates make two notes. The danger is the unknown outcome: a request times out after the server applied it, the client retries, and now there are two notes. This happens more under a rate limit, because slow replies are common near the limit.

The fix is an idempotency key: a random id the client sends with the request, in a header such as Idempotency-Key. The server remembers the key and the first result, so a retry with the same key gets the saved result instead of a second create. The IETF has a draft that standardises the header.

Stripe's API is the well-known example. In its v1 API every POST accepts the header, keys can be up to 255 characters (Stripe suggests V4 UUIDs), keys may be pruned once they are at least 24 hours old, and reusing a key with different parameters is an error. The pruning matters for a queue that resumes on the next visit: a create retried days later is no longer protected by its key, so check whether the record exists first.

function enqueueCreate(note: NoteDraft) {
  return queue.add({
    kind: "create-note",
    method: "POST",
    url: "/api/notes",
    body: note,
    idempotencyKey: crypto.randomUUID(),
  });
}

async function send(job: Job) {
  return fetch(job.url, {
    method: job.method,
    headers: { "Idempotency-Key": job.idempotencyKey, "Content-Type": "application/json" },
    body: JSON.stringify(job.body),
    signal: AbortSignal.timeout(15_000),
  });
}

Without idempotency keys, there are two fallbacks. Let the client choose the record's id (many APIs accept a client-made id or an external id field), so a retry becomes an update of the same record. Or, before retrying a create with an unknown outcome, look it up by that external id and retry only if it is missing.

The job's life as a state machine

All these rules fit in one small state machine per job. Writing it down prevents bugs like a cancelled job being retried.

Priorities, ordering and coalescing

The bulk edit has 1,400 jobs left, about 2.5 minutes at 9 per second. The user fixes a typo in one contact's phone number. In a plain FIFO queue that save waits over two minutes, and the user assumes it is broken. The queue needs priorities, plus rules so nothing waits forever and nothing arrives out of order.

Lanes

Give every job one of three lanes, set by the caller at enqueue time so the queue never guesses. Interactive is one action the user just made and is watching (save this field, send this message). Bulk is work the user started but does not watch item by item (the 2,000-row edit, an import). Background is work the user did not ask for (refreshing caches, flushing logs).

Per-record ordering

Priorities create a new risk. The bulk job sets contact 42's owner to Sam; a second later the user opens contact 42 and sets the owner to Priya, interactively. If the interactive save jumps ahead and the bulk save lands after it, Sam overwrites Priya and the user's last action is lost.

The rule: every job has an ordering key (here contact:42). A job is ready only when no earlier job with the same key is waiting or in flight. Different keys still run in parallel and in priority order.

Coalescing and deduplication

While the queue is paused for a 429, a user edits a contact's title five times. Sending five requests spends five tokens to reach the state the last one sets. Coalescing merges jobs that are still waiting: if the newest job for that key is a waiting update of the same kind, copy the new fields into it instead of adding a job. A PATCH of title and a PATCH of phone to the same contact can merge into one PATCH with both.

Merge only into a job that has never been tried: a job that is in flight or already went out once has an idempotency key tied to its original body, so new values go into a new job behind it. And merge only into the newest job for the key: if a delete sits between two patches, folding the second patch into the first would move it ahead of the delete.

function enqueuePatch(key: string, fields: Record<string, unknown>, lane: Lane) {
  const newest = jobs.findLast((j) => j.key === key);
  if (newest && newest.kind === "patch" && newest.state === "queued" && newest.attempts === 0) {
    newest.body = { ...newest.body, ...fields };
    newest.lane = higherLane(newest.lane, lane);
    newest.coalesced += 1;
    return persist(newest);
  }
  return add({ key, kind: "patch", body: fields, lane, state: "queued" });
}

Note the lane upgrade: a background patch merged with an interactive one runs as interactive. Deduplication is the simpler cousin: two identical idempotent jobs (same method, URL and body) waiting at once become one, and both callers get the same result promise (the request deduplication lesson covers sharing in-flight GETs). Creates never merge, because two POSTs that each send a message are two messages. A create and a later delete of the same draft may cancel each other out, but only if neither has been sent.

Surviving reloads and sharing one limit across tabs

So far the queue lives in one tab's memory, and two everyday events break it. A reload halfway through the bulk edit makes 1,100 changes vanish without a word. Two open tabs each send 9 per second, 18 together against a limit of 10, and about half get 429.

Persist every job in IndexedDB

IndexedDB (see the IndexedDB lesson) is the only browser storage that is asynchronous, holds structured objects, and has room for thousands of jobs. localStorage blocks the main thread on every write and holds about 5 MB of strings. The rules for using it here:

• Write before you promise. enqueue resolves, and the UI says Queued, only after the job's IndexedDB transaction completes. A tab that dies a millisecond later loses nothing.
• Write every state change (queued, sending, retry-wait, done, failed, cancelled). An index on state lets a restarted queue find unfinished work quickly.
• Persist the limiter too. A 429 pause ends at a wall-clock time; store it, or a reload resets the pause and the restarted queue hits the limit again at once.
• Treat sending as unknown after a reload. A job that was sending when the page died may have been applied. Retry it with its saved idempotency key, or check first if it has none.
• Clean up. Delete done jobs after a short time (say an hour), so the store does not grow forever.

One tab owns the queue

const channel = new BroadcastChannel("rlq");

navigator.locks.request("rlq-leader", async () => {
  const scheduler = new Scheduler(await loadJobsFromIndexedDB());
  channel.onmessage = async (e) => {
    if (e.data.type === "job-added") scheduler.add(await readJobs(e.data.ids));
    if (e.data.type === "cancel") scheduler.cancel(e.data.id);
  };
  scheduler.onChange = (job) => channel.postMessage({ type: "status", id: job.id, state: job.state });
  scheduler.start();
  await new Promise(() => {}); // hold the lock for the life of this tab
});

export async function enqueue(job: NewJob) {
  const saved = await writeJob(job); // IndexedDB first
  channel.postMessage({ type: "job-added", ids: [saved.id] });
  return saved;
}

One subtle point: a BroadcastChannel object does not receive its own messages, so the leader never hears its own job-added. If the leader itself enqueues, it must also add the job to its own scheduler directly. Another: the leader's own reads and writes must go through the same queue as everyone else's, so that the rate is counted once.

When the browser closes

When the last tab closes, the queue stops. The Background Sync API gives a service worker a sync event later, when the device is online, even with no tab open. But it is Chromium-only (not Firefox or Safari), fires on connectivity rather than at a time you choose, and the handler's run time is limited (the background sync lesson and the offline sync engine guide go deeper).

For a rate-limited queue the plain answer is usually best: resume on the next visit, and warn before the tab closes if the user's own work is still waiting.

• On startup, any tab that becomes leader loads unfinished jobs and continues. A small banner says "Sending 1,100 changes left from your last visit", with Cancel.
• On beforeunload, if interactive or bulk jobs are waiting, call preventDefault() so the browser asks the user to confirm leaving (browsers show the prompt only if the user has interacted with the page). Do not do this for background jobs.
• If work must finish without the user (a nightly sync of 50,000 records), the client is the wrong place. Hand it to a server job with its own rate limiter.

Try both problems in the lab below: two tabs on one limit with and without a leader, and a reload in the middle with and without IndexedDB.

Architecture, data model and status UI

We now have every mechanism. This chapter puts them into parts with one job each, so each part can be tested alone and swapped when the API changes, then defines the data, the API and what the user sees. The guiding idea: every tab can enqueue and watch, only the leader sends, and IndexedDB sits between them as the single source of truth.

Each boundary buys something. The limiter knows nothing about jobs, so one limiter serves a chat send button and a bulk import, and the transport knows nothing about the queue, so rules like "is this 403 a rate limit?" live in one place per API. The scheduler never touches the network, so tests drive it with a fake transport and a fake clock. The UI talks only to the queue client, so it works the same in the leader and in a follower tab.

Keep one limiter per budget, not one per queue. A CRM may call two APIs (HubSpot and our own), and GitHub has separate budgets per resource family (core, search, graphql), named by the x-ratelimit-resource header. Each job carries a budget field so the scheduler checks the right limiter, and a pause on the search budget never stops core requests.

How data moves for one bulk edit

Follow the 2,000-row edit end to end. On Apply, the UI shows the new owner in all 2,000 rows at once, marked pending, and the queue client writes 2,000 jobs to IndexedDB in one transaction (one commit instead of 2,000) and posts one hint. The leader reads the new jobs, and the scheduler starts them as the limiter allows: 9 per second, 4 in flight. Each reply updates the job in IndexedDB and is broadcast, so every tab's rows turn from pending to saved.

Data model

The data model is small, and each field exists because of a decision earlier in the guide. If you cannot say which decision a field serves, it probably does not belong.

type Lane = "interactive" | "bulk" | "background";
type JobState = "queued" | "sending" | "retry-wait" | "done" | "failed" | "cancelled";

type Job = {
  id: string;              // crypto.randomUUID()
  budget: string;          // which limiter: "hubspot:acct-17", "github:core"
  lane: Lane;
  key: string | null;      // ordering key, "contact:42"
  kind: "patch" | "create" | "delete" | "custom";
  method: "POST" | "PATCH" | "PUT" | "DELETE";
  url: string;
  body: unknown;
  idempotencyKey: string;  // fixed at enqueue, reused on every attempt
  state: JobState;
  attempts: number;        // not counting 429s
  notBefore: number;       // epoch ms: retry time, or 0
  enqueuedAt: number;      // for aging and ETA
  coalesced: number;       // how many edits merged into this one
  groupId: string | null;  // the bulk action it belongs to
  error?: { status: number; message: string };
};

type BudgetState = {
  budget: string;
  perSec: number;          // learned refill rate, 90% of the server's
  capacity: number;
  maxInFlight: number;
  pausedUntil: number;     // epoch ms, 0 if not paused
  serverRemaining: number | null;
  serverResetAt: number | null;
};

The queue client's API

Callers should not see tokens or tabs. The same few calls cover the chat send button and the bulk edit.

interface ActionQueue {
  enqueue(job: NewJob): Promise<JobHandle>;           // resolves once saved
  enqueueMany(jobs: NewJob[], groupId: string): Promise<void>;
  cancel(idOrGroupId: string): Promise<void>;         // queued and retry-wait only
  retry(id: string): Promise<void>;                   // failed only, resets attempts
  subscribe(listener: (s: QueueSnapshot) => void): () => void;
}

type JobHandle = { id: string; done: Promise<Response> };

type QueueSnapshot = {
  counts: Record<JobState, number>;
  pausedUntil: number | null;     // for the countdown
  etaMs: number | null;           // waiting jobs / rate
  byGroup: Record<string, { done: number; failed: number; total: number }>;
};

Cancel works only on jobs that have not started. A job in flight may already be applied, so cancelling it would lie to the user; the UI disables Cancel for those rows. The done promise lets a caller like the chat composer await its own job, but it only exists in the tab that made the job and only until a reload. After a reload, status comes from subscribe.

Messages between tabs

type ChannelMessage =
  | { type: "job-added"; ids: string[] }
  | { type: "cancel"; id: string }
  | { type: "status"; id: string; state: JobState; error?: string }
  | { type: "budget"; budget: string; pausedUntil: number; etaMs: number | null };

Status UI: telling the user what is happening

A rate-limited queue makes the app slower on purpose, and users accept a wait they understand. The worst outcome is silent: a spinner that never ends, or a change that never saves. So the UI has one rule: every action the user made is visible until it is done, failed with a reason, or cancelled by them. The progress bar queue guide covers progress lists and bars; here we cover the wait itself, with countdowns and honest estimates.

Four states the user can see

In chat, messages over the limit appear at once, in order, with a clock mark and the line "You are sending quickly. These will go out in a few seconds." They are not shown as errors, because nothing failed.

Honest estimates and countdowns

The ETA is jobs waiting divided by the current rate, plus any pause left. Round it and update it every few seconds: "about 2 min" is honest, while "1 min 53 s" claims a precision we do not have, because the rate changes with latency and other tabs.

The pause countdown is different: it is a fixed time the server gave us. Compute it from the clock as pausedUntil - Date.now() in whole seconds, not by subtracting 1 each tick, because timers drift and are slowed in hidden tabs.

Accessibility

• Put the banner text in a polite live region (role="status"). Announce changes of state ("Paused for 20 seconds", "All 2,000 changes saved", "3 changes failed"), never each tick of the countdown, which would talk over everything else.
• The countdown itself sits outside the live region, so a screen reader user can read it when they choose.
• Cancel all and Retry failed are real buttons with clear labels ("Cancel 1,100 waiting changes"). After cancelling, move focus to the banner so the user hears the result.
• Row chips use text and an icon, not color alone, to tell waiting from failed.

Giving the user control

Let the user cancel the waiting part of a bulk job, and say exactly what that means: "900 changes were saved. 1,100 were not sent and are back to their old values." Rolling back the optimistic values of cancelled jobs is part of cancelling.

For an hourly budget, warn before a large job starts: "Labelling 6,000 issues needs more than your hourly limit of 5,000, so it will take more than an hour." The user can then pick a smaller selection instead of discovering the limit at item 5,001. On GitHub, requests that create content also face secondary limits (in general about 80 per minute and 500 per hour), so a write-heavy job may be slower still.

Testing with a fake clock and a strict fake server

Rate limiting bugs hide in timing: a window edge, a pause that ends early, two tabs that both think they lead. Real-time tests are slow and flaky, but because the scheduler takes the clock and the transport as inputs, tests can run hours of traffic in milliseconds. The unit and integration tests lesson covers the general setup; here are the tests that matter for this queue.

A fake server that enforces the limit

Write a fake transport that keeps a sliding log and answers 429 with Retry-After when the client goes over. Make it stricter than the real API on purpose, with a configurable latency to exercise both gates, and check only what the server saw.

test.prop([fc.array(fc.integer({ min: 0, max: 5_000 }), { maxLength: 300 })])(
  "never exceeds 10 per second, never loses a job",
  async (enqueueTimes) => {
    const clock = new FakeClock();
    const server = new StrictServer({ limit: 10, windowMs: 1000, latencyMs: 300, clock });
    const queue = createQueue({ clock, transport: server.transport, perSec: 9, maxInFlight: 4 });
    for (const t of enqueueTimes) clock.at(t, () => queue.enqueue(job()));
    await clock.runUntilIdle();
    expect(server.maxInAnyWindow()).toBeLessThanOrEqual(10);
    expect(server.count429()).toBe(0);
    expect(queue.snapshot().counts.done).toBe(enqueueTimes.length);
  },
);

A property test (here test.prop from the @fast-check/vitest package) generates hundreds of random inputs and checks rules that must hold for all of them. That is a good fit here: the rules (never above the limit, never lose a job, never reorder a key) are simple to state, and the inputs that break them are easy to miss when writing examples by hand.

Scenarios worth a named test

• Global pause. Four in flight, the first gets 429 with Retry-After: 20. Assert nothing starts for 20 fake seconds, then the failed job runs first.
• Both forms of Retry-After. Seconds, and an HTTP date with a client clock 3 minutes fast.
• Header sync. The server reports remaining: 2 while the client bucket holds 8; assert at most 2 sends before the reset.
• Idempotency. The server applies a POST, then the transport times out. Assert the retry carries the same key and one record exists.
• Ordering and coalescing. Five patches to one key while paused: assert one request with the last values. An interactive patch never overtakes an in-flight bulk patch to the same key.
• Starvation. Keep adding bulk jobs; assert a background job still runs within its aging bound.
• Reload. Stop the queue mid-run and build a new one on the same fake IndexedDB (the fake-indexeddb package runs in Node). Assert it resumes, keeps the pause, and retries jobs that were sending.
• Two tabs. Two queue instances with a shared fake lock and channel. Assert only one sends, and that killing it hands over without a gap or a double send.

Watching it in production

Ship a few counters through the app's analytics pipeline (the analytics SDK guide covers batching them): 429s per 1,000 requests (target under 1), achieved rate as a share of the documented limit (target 80 to 90%), queue wait at the 50th and 95th percentile per lane, attempts per job, and failed jobs grouped by status.

A rising 429 share means a header adapter broke or the API lowered its limit; a low achieved rate means the client is too careful. Jobs that vanish without a final state should number zero, so alert on any.

Running it in the interview

This question rewards structure. Many candidates jump to "use a token bucket" in the first minute and then have nothing to say when the interviewer asks about two tabs or a retried create. Spend the 45 minutes like this:

Draw the architecture early, even roughly, and add to it as you go: the queue client in every tab, IndexedDB below, the leader with scheduler, limiter and transport, the proxy, the API. A picture that grows with the conversation keeps you and the interviewer looking at the same design, and shows you are designing a system rather than reciting facts.

Follow-up questions to expect