Async patterns

Async work needs a few rules

Some work takes time, like asking a server for data. JavaScript doesn't stop and wait for it. Work like this is called asynchronous, or async for short.

async and await let you write async code that reads from top to bottom. But real apps still run into problems:

• The network can be slow.
• The server can be busy and say "try again later".
• The user can leave the page before the work ends.
• Eight jobs can all want the same server at the same time.

This lesson shows five small patterns for these problems. A pattern is a shape of code you can reuse again and again:

• Retry with backoff: try again after a failure, and wait longer each time.
• Timeouts: stop waiting after a time limit.
• Queues and pools: run only a few jobs at once.
• Polling: ask "Is it done yet?" again and again, with a pause in between.
• Sleep helpers: pause one async function for a set time.

All five follow one rule: every wait has a limit, and you can stop it when possible.

These patterns are built from things you met earlier: Promises, Timers, Error handling strategies, and The event loop. Go back to them if a word here is new to you.

A retry is another try after a failure. Backoff means each wait is longer than the one before. A common choice is to double it: 100 ms, then 200 ms, then 400 ms. (ms means milliseconds. 1000 ms is one second.)

The wait can't grow forever. A cap is the longest wait you allow, such as 2 seconds. Once the doubled wait reaches the cap, it stays there.

Many users can fail at the same moment, for example when a server goes down for a second. If they all wait exactly 100 ms, they all retry at the same moment too, and the server is too busy again. Jitter fixes this. It picks a random wait between 0 and the capped wait, so the retries spread out. This kind is called full jitter.

Retry only when both of these are true:

• The error will probably pass. A server status of 503 ("Service Unavailable") or 429 ("Too Many Requests") usually means "try again later".
• Doing the work twice is safe. Reading data is usually safe.

Writing data is harder. If a payment request runs twice, the customer pays twice. Work is idempotent when doing it twice has the same effect as doing it once. Some payment servers let you send an idempotency key with the request: a unique ID for it. If the same key arrives twice, the server does the work only once.

1async function retry(operation, options) {2  for (let attempt = 0; attempt < options.maxAttempts; attempt += 1) {3    try {4      return await operation();5    } catch (error) {6      if (!error.temporary) throw error;7      const uncapped = options.baseDelay * 2 ** attempt;8      const delay = Math.min(uncapped, options.maxDelay);9      const wait = options.random() * delay;10      await options.sleep(wait);11    }12  }13  throw new Error("No attempts left");14}

The lab passes in its own random function, which always gives 0.42, and its own short sleep. Do the same in tests. Pass in a fake sleep and a fake random, then check the waits and the number of tries. The test doesn't have to wait for real time.

A timeout is a time limit. In code, it's a promise that rejects (fails) when the limit passes.

Promise.race takes a list of promises. It settles (resolves or rejects) the same way as the first one to finish. So you race the real task against the timer. Whichever finishes first decides the result.

1function withTimeout(promise, ms) {Run2  let timer;3  const timeout = new Promise((_, reject) => {4    timer = setTimeout(() => reject(new Error("Timed out")), ms);5  });6  return Promise.race([promise, timeout])7    .finally(() => clearTimeout(timer));8}

Two details matter:

• Clear the timer in finally. finally runs after success and after failure. Clearing the timer there means it can't fire later. In Node.js, a timer that is still waiting also keeps the program from ending.
• If the task can stop, give it an AbortController signal. A signal is an object the task listens to for a "stop now" message. The previous lesson, Cancellation with AbortController, explains it in detail.

A pool runs at most N jobs at the same time. N is called the concurrency: how many jobs run at once. The other jobs wait in a queue, a line where the first to arrive is served first. Each time a job ends, the next job in the queue starts.

Say you have 100 images to upload. Starting 100 requests at once can slow down the browser, overload the server, and drain a phone's battery. A pool of four uploads them at a steady pace, four at a time.

1async function runPool(tasks, concurrency) {2  const results = [];3  let next = 0;4  async function worker() {5    while (next < tasks.length) {6      const index = next++;7      results[index] = await tasks[index]();8    }9  }10  await Promise.all(Array.from({ length: concurrency }, worker));11  return results;12}

Decide what happens when a job fails before you write the pool. There are two common choices:

• Collect the errors when the jobs don't depend on each other. For example, if one image thumbnail fails, the others can still be made.
• Fail fast, which means stop at the first error, when one failure makes the whole result useless. For example, an app can't start without its settings file.

Polling means asking the server "Is the job done?" again and again, with a pause between each question. You stop when the job is done, when it has failed, when the user cancels, or when a time limit passes.

Polling is useful when the server can't tell you on its own that the job is done. A later lesson covers WebSocket and Server-Sent Events. With those, the server sends you a message when something changes, so you don't need to ask.

1async function pollJob(checkStatus, { interval, timeout, signal, sleep }) {2  const deadline = Date.now() + timeout;3  while (Date.now() < deadline) {4    if (signal?.aborted) throw signal.reason;5    const status = await checkStatus();6    if (status.state === "done") return status.value;7    if (status.state === "failed") throw new Error(status.reason);8    await sleep(interval, { signal });9  }10  throw new Error("Polling timed out");11}

For network polling, use an async loop, not setInterval. setInterval calls a function every N ms, even if the last request hasn't come back yet. Then requests can pile up. An async loop awaits the request, and then awaits the sleep. So the next check starts only after the last one ended.

Browsers have no built-in sleep function, but you can make one. A timer can resolve a promise after some time. When you await that promise, only your async function pauses. The page and the event loop keep working.

1const sleep = (ms) => new Promise((resolve) => {2  setTimeout(resolve, ms);3});4 5await sleep(500);6console.log("half a second later");

Line 1 makes a new promise. Line 2 starts a timer that resolves the promise after ms milliseconds. Line 5 waits for that promise, so line 6 prints half a second later.

1function sleep(ms, { signal } = {}) {2  return new Promise((resolve, reject) => {3    if (signal?.aborted) return reject(signal.reason);4    const timer = setTimeout(resolve, ms);5    signal?.addEventListener("abort", () => {6      clearTimeout(timer);7      reject(signal.reason);8    }, { once: true });9  });10}

This version also takes a signal. Line 3 rejects right away if the signal has already stopped. Line 4 starts the timer. Lines 5 to 8 listen for a stop. When it comes, line 6 clears the timer and line 7 rejects with the signal's reason. { once: true } removes the listener after it runs once.

In Node.js, you can also import setTimeout from node:timers/promises and await it directly. In the browser, a small helper like this one shows the idea.

Where you'll use this

Real apps often use several patterns together. In this example, saving a draft does three things:

• It retries the save when it fails for a reason that may pass.
• It sends an idempotency key, so a repeated save doesn't make two drafts.
• It stops waiting for the reply after 2 seconds.

1async function saveDraft(draft, api) {2  const result = await retry(3    () => api.putDraft(draft, { idempotencyKey: draft.id }),4    { maxAttempts: 4, baseDelay: 100, maxDelay: 1000, sleep, random: Math.random },5  );6  return withTimeout(result.json(), 2000);7}

Line 2 calls the retry helper from the retry lab. Line 3 is the work to retry: save the draft, with the draft's ID as the idempotency key. Line 4 sets the limits: at most 4 tries, a first wait of 100 ms, and never more than 1000 ms. Line 6 reads the reply's JSON body, but waits at most 2000 ms.

Pages with buttons and search boxes need two more safety rules:

• Share a request that is in flight. "In flight" means started but not finished yet. If two parts of the page ask for the same data at once, give both the same promise. Then a double click doesn't start the same work twice.
• Ignore an old reply. Give each request a number. When a reply arrives, show it only if its number is still the newest.

1const inFlight = new Map();2function loadUser(id, fetchUser) {3  if (inFlight.has(id)) return inFlight.get(id);4  const request = fetchUser(id).finally(() => inFlight.delete(id));5  inFlight.set(id, request);6  return request;7}8 9let calls = 0;10const fetchUser = async (id) => {11  calls += 1;12  return { id, name: "Asha" };13};14const first = loadUser("42", fetchUser);15const second = loadUser("42", fetchUser);16console.log(first === second, calls);

inFlight is a Map: a list of keys, each with a value. Line 14 is the first call. The map has nothing for "42", so line 4 starts the fetch and line 5 saves its promise under that key. Line 15 is the second call. Line 3 finds the saved promise and returns it. So line 16 prints true 1: both callers got the same promise, and the fetch ran once.

The finally on line 4 removes the promise from the map when the request ends. This matters when a request fails. Without it, every later call would get the same failed promise.

1let latestRequest = 0;2function acceptLatest(request) {3  const requestId = ++latestRequest;4  return request().then((value) =>5    requestId === latestRequest ? "show " + value : "ignore stale",6  );7}8 9const oldResult = acceptLatest(async () => "old");10const newResult = acceptLatest(async () => "new");11Promise.all([oldResult, newResult]).then((results) => console.log(results.join(" | ")));

Search boxes often start a request for a, and then one for ab. If the reply for a comes back last, it could replace the newer results on the screen. An old reply like this is called stale.

Line 9 starts request 1, and line 10 starts request 2. Line 3 gives each one its number. When a reply arrives, line 5 compares its number with the newest number, which is 2. So the program prints ignore stale | show new.

1function debounce(task, wait) {2  let timer;3  return (...args) => {4    clearTimeout(timer);5    timer = setTimeout(() => task(...args), wait);6  };7}8 9const search = debounce((query) => console.log("search", query), 10);10search("a");11search("ab");12search("abc");13setTimeout(() => console.log("quiet"), 20);

Debounce waits until the calls stop for a moment. Each call to search clears the last timer (line 4) and starts a new 10 ms timer (line 5). Lines 10, 11, and 12 run quickly, one after another, so only the last timer is left. The program prints search abc, and then quiet.

Throttle is a similar tool. It allows at most one call in each time window, for example one call every 100 ms. It suits updates while the user scrolls or resizes the window. Neither tool stops a request that has already started. For that, add a timeout or an abort signal.

1function makeQueue() {2  let tail = Promise.resolve();3  return (task) => {4    const result = tail.then(task);5    tail = result.catch(() => {});6    return result;7  };8}9 10const queue = makeQueue();11const order = [];12queue(async () => order.push("first"));13queue(async () => order.push("second"));14queue(async () => console.log(order.join(" -> ")));

This queue starts each task only after the one before it ends. tail is the promise for the last task in line (line 2). Line 4 runs the new task after tail. Line 5 makes the new task the new tail. Lines 12 to 14 add three tasks at once, but they still run in order, so the program prints first -> second.

The empty catch on line 5 keeps the queue going after a task fails. The caller still sees the failure, because line 6 returns result, not tail.

Use this table to pick a pattern. Find what you need in the first column.

Common misconceptions

• "Retry every error." Retry only errors that may pass. A 400 (the request is wrong) or a 401 (not signed in) needs a fix, not the same request again. A payment with no idempotency key could charge twice.
• "Backoff can keep going forever." Always set a longest wait, a limit on tries, and a limit on total time.
• "Promise.race stops the slower promise." It only decides the race's result. The slower task keeps running, unless it listens for a stop signal.
• "setInterval is fine for polling." It starts a new check every N ms, even when the last request is still running. An async loop waits for each check to end.
• "A pool mixes up the order of the results." Jobs can finish in any order. But if you save each result at its job's place in the input, the results stay in input order.
• "sleep freezes JavaScript." Awaiting a sleep promise pauses one async function. The rest of the page keeps running.

Key takeaways

• Retry only errors that may pass, and only work that is safe to repeat. Double the wait each time, cap it, add jitter, and limit the number of tries.
• Promise.race gives you a time limit, but it doesn't stop the slower task. Clear the timer, and pass a signal when the work must stop.
• A pool runs at most N jobs at once. Decide what it does when a job fails.
• Poll with an async loop, so one check never overlaps the next.
• sleep is a promise around a timer. Make it stoppable when it may wait a long time.

Async patterns are reusable ways to wait. They give every wait a limit, keep the load on servers fair, and make the results easy to predict.

Up next: The browser environment.