Retries, timeouts, and backoff

The problem: failures that heal and failures that spread

Picture a food delivery app. A user on a train taps Track my order. The phone is between two cell towers, the request leaves the device, and the answer never comes back.

The spinner turns for 40 seconds, then the screen shows "Something went wrong". Ten seconds later the train reaches a station and the network is fine again.

Nothing was really broken. The failure was transient: a short problem that heals by itself in a few seconds. One quiet retry after a second or two would have shown the order status, and the user would never have known. Instead the app waited far too long, then gave up at the first try.

Now picture the opposite day. The order service has a bad deploy and slows down. Every one of the 200,000 phones that have the app open polls it, and each of them is coded to "retry right away until it works".

The service was handling 3,000 requests per second. Within a minute it sees 30,000, because every failed request comes back several times. The extra load keeps it down long after the bad deploy is rolled back.

Both stories are about the same few lines of client code. Retry too little and users see needless errors. Retry too much, too fast, or all at once, and clients turn a small outage into a long one.

Four questions every request has to answer

A client that handles failure well answers four questions for every request it sends. The rest of the lesson takes them one at a time.

1. How long do I wait? Every request needs a deadline, a timeout. Without one, a request that hangs keeps the spinner up forever, because fetch() has no timeout of its own.
2. Should I try again? Some failures heal (a dropped connection, a 503 Service Unavailable). Others never will (a 404 Not Found, a 400 Bad Request). Retrying those only wastes time.
3. Is it safe to try again? Reading data twice is harmless. Charging a card twice is not. A retry of a write needs an idempotency key so the server can spot the repeat.
4. When, and when do I stop? Retries are spaced out with exponential backoff (each wait longer than the last) and jitter (a random part, so clients do not retry in step). A cap on attempts, a retry budget, and a circuit breaker decide when to stop.

Where this logic lives

In most apps you write the retry loop once: in a small fetchWithRetry helper, in your API client, or in the settings of a server-state library such as TanStack Query. The point of this lesson is to know what those defaults do and when to change them. Request races covers cancelling unwanted requests, and REST API design covers idempotency keys and Retry-After from the server's side.

The mental model: a loop with a deadline and a brake

Think of phoning a busy restaurant to book a table. If the line is busy, you call back, but not every second: first after a minute, then after five. If a recording says "call back after 6 pm", you wait until 6.

If they tell you they are closed on Mondays, calling again will not help. And if the call drops halfway through the booking, you open the next call with "I'm the person who just booked for 8 pm", so they do not book two tables.

Every part of that story maps to a piece of retry logic. Waiting longer each time is backoff. "Call back after 6" is the server's Retry-After header. "Closed on Mondays" is a permanent error, such as a 404.

"I'm the person who just booked" is an idempotency key. And if every call for an hour fails, you stop calling and try a different restaurant: that is a circuit breaker.

One request as a small state machine

In code, a request with retries is a loop. It is easiest to understand as a handful of states, with the request moving between them. Each arrow in the next figure is a decision your code makes.

1. Sending. The attempt is on the wire with a timer running. It ends in one of three ways: a success, an error response, or the timer firing.
2. Classify. The code looks at what came back and decides: success, a failure that might heal (retryable), or one that will not (permanent).
3. Waiting. For a retryable failure with attempts left, the code sleeps for a backoff delay, then goes back to Sending. A Retry-After from the server sets the minimum wait.
4. Done or Failed. A success returns data. A permanent error, a used-up attempt count, or a total deadline ends in Failed, and the UI shows an error with a Retry button.

Three clocks, not one

Beginners often think of "the timeout" as one number. In practice three different clocks run, and mixing them up causes most of the bugs in this lesson.

The total deadline is the one users feel. Four attempts with an 8-second timeout and waits of up to 0.5, 1, and 2 seconds can add up to more than 35 seconds. That is too long for a spinner. Either cap the total, or lower the per-attempt timeout, so the sum fits what a person will sit through.

Which failures to retry

The first rule of retrying: only retry when a second try has a real chance of a different result. A failure caused by the request itself (a missing field, a wrong ID, no permission) fails the same way every time. Retrying it only delays the error message and adds load.

The second rule: only retry automatically when repeating the request is safe. Reading the same data twice is harmless. Creating an order twice is a real bug that costs real money.

How fetch reports a failure

fetch() has two different ways to fail, and your code has to handle both. This surprises people who are used to libraries like Axios, which throw on any status outside 200 to 299.

• The promise rejects. No HTTP response arrived. The error is a TypeError for network failures (connection dropped, DNS lookup failed, the server refused the connection) and also for CORS failures, which the browser deliberately makes look the same. If you passed a signal and it fired, the error is a DOMException named AbortError, or TimeoutError when the signal came from AbortSignal.timeout().
• The promise resolves with an error status. A response arrived, but response.ok is false because the status is outside 200 to 299. fetch treats this as a success at the network level. You must check response.ok or response.status yourself.

The decision for each kind of failure

Here is the policy this course uses everywhere, and the one the REST API design lesson uses for its error states. The middle column is for a request that is safe to repeat: a read, or a write that carries an idempotency key.

Is it safe to repeat?

A request is idempotent when sending it twice has the same effect on the server as sending it once. HTTP defines GET, HEAD, OPTIONS, PUT, and DELETE as idempotent. PUT /cart/items/42 with a quantity of 2 sets the quantity to 2 however many times it arrives. POST /orders creates a new order each time, and PATCH with "add one" adds one each time.

The hard case is a write that failed with a network error or a timeout. The browser cannot know whether the request reached the server. Maybe it was lost on the way there, or maybe the server finished the work and only the answer was lost.

So the default for a non-idempotent write is: do not retry automatically. Show the error and a Retry button, keep the user's input, and let a person decide. The idempotency key chapter shows how to make such a write safe to retry.

const RETRY_STATUS = new Set([408, 429, 502, 503, 504]);

function isRetryable(error: unknown, safeToRepeat: boolean): boolean {
  if (!safeToRepeat) return false;
  if (error instanceof DOMException) {
    return error.name === "TimeoutError"; // AbortError: someone cancelled
  }
  if (error instanceof TypeError) return true; // network failure
  if (error instanceof HttpError) return RETRY_STATUS.has(error.status);
  return false; // a bug in our own code: retrying will not fix it
}

Timeouts: decide how long to wait

fetch() has no timeout option. If a server accepts the connection and then never answers, or a proxy holds the request open, the promise can stay pending for minutes. Your spinner turns the whole time, and your retry logic never runs, because nothing has failed yet. A missing timeout is the most common reason "retries don't work".

So every request a person waits on gets a deadline. Modern browsers make that one line, with AbortSignal, the same object you already use to cancel requests in the request races lesson.

AbortSignal.timeout(ms)

AbortSignal.timeout(8000) returns a signal that aborts itself after 8 seconds. Pass it to fetch and the request is cancelled when time runs out. The promise rejects with a DOMException whose name is TimeoutError. A user cancelling gives AbortError instead, so your code can tell "too slow, maybe retry" apart from "not wanted, stop".

The deadline covers the whole response, not just the headers. If the headers arrive at 2 seconds but the body is still streaming at 8, the abort also makes response.json() reject. That is usually what you want: the user is waiting for the data, not the headers.

It is supported in every current browser (Chrome and Edge 103, Firefox 100, Safari 16) and in Node.js. One catch: Chrome and Edge before version 124 reported the timeout as an AbortError, not a TimeoutError, so on those old versions a timed-out request looks like a cancel.

AbortSignal.any: a timeout and a cancel button together

A real request usually needs two reasons to stop: its deadline, and the component unmounting or the user typing a new query. fetch takes only one signal. AbortSignal.any([a, b]) builds a signal that aborts as soon as any of its inputs does, with that input's reason.

It has been in all major browsers since 2024 (Chrome 116, Firefox 124, Safari 17.4). For older browsers, a small fallback with one AbortController and listeners on both signals does the same job.

async function getJson(url: string, cancel: AbortSignal, timeoutMs = 8_000) {
  // A new timeout for this attempt, merged with the caller's cancel signal
  const signal = AbortSignal.any([cancel, AbortSignal.timeout(timeoutMs)]);
  const response = await fetch(url, { signal });
  if (!response.ok) throw new HttpError(response.status, response.headers);
  return response.json(); // the same signal also limits reading the body
}

// Later, in the retry loop:
// TimeoutError  -> retryable (if safe to repeat)
// AbortError    -> stop at once, do not retry, do not show an error

Per attempt or in total?

Where you create the timeout decides what it limits. Create AbortSignal.timeout() inside the loop and each attempt gets a fresh 8 seconds. Create it once, before the loop, and it becomes a total deadline: the second attempt starts with whatever time is left, and the sleep between tries counts too.

Good clients use both: a short timeout per attempt, so one stuck connection does not eat the whole budget, and a total deadline, merged in with AbortSignal.any, that caps what the user waits. The sleep between attempts listens to the same signal.

Picking the numbers

Pick timeouts from real data, not from a round number that feels safe. Look at the latency percentiles for the endpoint: p50 is the time half of the requests beat, p99 is the time 99 in 100 beat. If p99 is 1.2 seconds, an attempt that has taken 6 seconds is almost certainly stuck, and a fresh attempt is likely to be faster than waiting.

• Small reads (a profile, a list page): a timeout of about 3 to 5 times p99, often 5 to 10 seconds.
• Search as you type: short, about 3 to 5 seconds, because a newer keystroke will replace the request soon anyway.
• Uploads and exports: no fixed short timeout. Use progress, or a timeout on "no progress for 30 seconds", and let the user cancel.
• Writes that charge or create: a generous timeout. Timing out a write does not undo it on the server; it only makes the outcome unknown.

Backoff and jitter: how long to wait between tries

Once you know a failure is worth retrying, the next question is when. Retrying at once rarely helps: whatever broke 50 ms ago is very likely still broken. And if the server is failing because it is overloaded, an instant retry adds load at the worst moment.

Exponential backoff

Exponential backoff doubles the wait after each failure in a row. With a base of 500 ms, the waits are 0.5 s, 1 s, 2 s, 4 s, 8 s. A cap stops the growth, say at 8 s or 30 s, so a long outage does not lead to hour-long waits. The formula is one line: min(cap, base * 2 ** attempt), where attempt counts from 0.

Why backoff alone is not enough: the thundering herd

Backoff spaces out the retries of one client. It does nothing about many clients that failed at the same moment. When a server goes down, every open tab sees its request fail within the same second.

With plain backoff they all wait exactly 0.5 s, all retry together, all fail together, all wait exactly 1 s, and so on. The load does not spread out: it arrives in sharp waves. This is the thundering herd.

The waves hurt most when the server comes back: a wave of 5,000 requests in the same 100 ms knocks it over again, and the clients back off in step once more. The outage outlasts the bug that caused it.

Jitter: let each client pick its own moment

Jitter adds randomness to each wait, so clients that failed together drift apart. The most common form, called full jitter, waits a random time between zero and the backoff value: random() * min(cap, base * 2 ** attempt). On the third failure the cap is 2 s, so one client waits 0.3 s, another 1.1 s, another 1.9 s. The waves flatten into a steady trickle.

Marc Brooker's 2015 post on the AWS Architecture Blog, "Exponential Backoff And Jitter", compared these strategies in a simulation of many competing clients. Backoff without jitter was clearly the worst; full jitter did the least total work, and decorrelated jitter finished slightly sooner. Full jitter is the standard answer in interviews.

Try the lab with 1,000 clients and the server down for 3 seconds, after which it can serve 1,000 requests per second. Without jitter, every retry wave lands in a single 100 ms slot, the server can take only a tenth of it, and the rest back off together again: the last client gets through after more than a minute. With full jitter, the same clients spread out, the busiest 100 ms gets about 270 requests instead of 1,000, and everyone is through in about 11 seconds.

Equal jitter finishes sooner in this lab, in about 7 seconds, but its busiest 100 ms gets about 420 requests, half again as many as full jitter. The AWS simulation found the opposite order, so the ranking between jitter styles changes with the workload. The gap between no jitter and any jitter does not.

When the server tells you how long: Retry-After

A server that sends 429 Too Many Requests or 503 Service Unavailable can add a Retry-After header. It holds either a number of seconds (Retry-After: 30) or an HTTP date. The short polling lesson shows how to parse both.

The rule for the client is simple: never retry sooner than Retry-After says. Use it as the minimum wait, and still add your own jitter on top if many clients get the same value. If Retry-After is longer than a person will wait, say 120 seconds, do not sleep behind a spinner. End the loop, say when to try again, and disable the button until then.

const BASE_MS = 500;
const CAP_MS = 8_000;

const backoffDelay = (attempt: number) =>
  Math.random() * Math.min(CAP_MS, BASE_MS * 2 ** attempt); // full jitter

function sleep(ms: number, signal: AbortSignal): Promise<void> {
  return new Promise((resolve, reject) => {
    signal.throwIfAborted();
    const id = setTimeout(resolve, ms);
    signal.addEventListener("abort", () => { clearTimeout(id); reject(signal.reason); }, { once: true });
  });
}

async function withRetry<T>(
  run: (signal: AbortSignal) => Promise<T>,
  { signal, safeToRepeat, attempts = 4 }: { signal: AbortSignal; safeToRepeat: boolean; attempts?: number },
): Promise<T> {
  for (let attempt = 0; ; attempt++) {
    try {
      return await run(signal);
    } catch (error) {
      if (attempt + 1 >= attempts || !isRetryable(error, safeToRepeat)) throw error;
      // retryAfterMs reads Retry-After from a 429 or 503 (seconds or a date), else 0.
      // It is the minimum wait; the jittered backoff on top keeps clients apart.
      await sleep(retryAfterMs(error) + backoffDelay(attempt), signal);
    }
  }
}

Making writes safe to retry: idempotency keys

The scariest failure in this lesson is a timeout on a write. The user taps Pay 42 euros. The request reaches the server, the card is charged, and then the response is lost: the phone switched from Wi-Fi to 4G at that exact moment. The browser only knows that it did not hear back.

Retry, and you might charge the card twice. Do not retry, and the user sees an error for a payment that worked, taps Pay again, and is charged twice anyway. Neither choice is safe until the server can recognize a repeat.

How a key makes a repeat harmless

An idempotency key gives one user action a name. The client makes a random ID, such as crypto.randomUUID(), when the user starts the action. It sends that ID in an Idempotency-Key header with the first attempt and with every retry.

The server stores the result of the first request under the key. When the same key arrives again, it does not charge again: it returns the stored result.

Stripe's API made this pattern well known, and an IETF working-group draft describes the same header. It is not yet an RFC, so the exact server behavior varies by API. The REST API design lesson covers the server side in depth: how long keys are stored, what happens when the same key arrives with a different body, and how a request still in progress is reported.

The client's rules

• One key per intent, made once. Create it when the user starts checkout or opens the form, not inside the retry loop. A new key per attempt makes every retry look like a new payment, which is the bug you wanted to prevent.
• Reuse it for the manual Retry button too. If the automatic retries give up and the user taps Retry, that is still the same intent. Same key.
• Keep it across a reload if the action matters. Put it next to the draft in sessionStorage or IndexedDB, so a refresh in the middle of checkout does not create a second key.
• Make a new key only for a new intent. After a success, or after the user changes the cart, the next Pay is a different action.
• Disable the button while a request is in flight. A key protects against duplicates, but a second click that makes a second key would not be caught.
• Wait and retry on "still in progress". If two attempts with the same key overlap, many APIs answer the second with 409 Conflict. That 409 is worth a retry after a short wait, unlike a 409 about the data itself.

When the API has no idempotency key

Many APIs do not support keys. Then the policy from the error table holds: a non-idempotent write does not retry automatically. On a network error or timeout, show a clear message, keep everything the user typed, and offer a Retry button. For high-stakes actions, check first: fetch the order list or the payment status, and only offer Retry if the first attempt really did not land.

Some writes can be redesigned to be naturally idempotent instead. "Set the quantity of item 42 to 2" (PUT) is safe to repeat, while "add one item 42" (POST) is not. Creating a resource under an ID the client chose, such as PUT /drafts/{uuid}, works the same way.

Knowing when to stop: retry budgets and circuit breakers

Backoff and jitter decide when one request tries again. They do not decide whether all of your retries, added up, are making things worse. Two tools do: a retry budget, which limits how many retries the whole client sends, and a circuit breaker, which stops calling a service that is clearly down.

Retries multiply across layers

A request from the browser rarely goes straight to the database. It passes through an API gateway, then a service, then maybe another service. If each layer makes up to 3 attempts, one click can become 3 times 3 times 3, which is 27 calls to a database that is already failing. Every layer thought it was being careful.

The fix is to retry in one layer only and make the others fail fast. In a web app the client is often the best place for reads, because it knows whether a person is still waiting and can show progress. If your gateway or backend already retries, the browser should retry less, or not at all. In an interview, asking "who else retries this call?" is a strong signal.

A retry budget

Even in one layer, retries add load exactly when the server is weakest. A retry budget caps that extra load. A common rule, described in Google's SRE book, is that retries may be at most about 10% of requests: when things are healthy you have plenty of room, and during an outage the client sends roughly its normal traffic plus 10%, not 4 times it.

The usual way to build it is a small token bucket. Each success adds a fraction of a token, each retry spends a whole token, and when the bucket is low, failures are reported at once without a retry. gRPC clients have this built in as retryThrottling with a tokenRatio setting, and AWS SDKs use a similar retry quota.

// Retries may be about 10% of successes; at most 10 saved up
const budget = { tokens: 10, max: 10, ratio: 0.1 };

function onSuccess() {
  budget.tokens = Math.min(budget.max, budget.tokens + budget.ratio);
}

function canRetry(): boolean {
  if (budget.tokens < 1) return false; // budget spent: fail fast
  budget.tokens -= 1;
  return true;
}

The circuit breaker

A circuit breaker watches every request to one service and, when too many fail, stops sending any for a while. The name comes from the switch in a fuse box: when something is wrong, it cuts the power instead of letting the wires overheat. Michael Nygard popularized it in the book "Release It!", and Martin Fowler's short article is the usual reference.

• Closed (normal): requests go through. The breaker counts failures over a recent window, such as the last 20 calls.
• Open: too many failed, for example 5 in a row or half of the last 20. Every call fails at once, without touching the network, for a cool-down such as 30 seconds. The UI shows "Recommendations are unavailable right now" instead of a spinner.
• Half-open: after the cool-down, one trial request goes through. If it succeeds, the breaker closes. If it fails, the breaker opens for another cool-down, often longer.

Breakers are most common between backend services, but they help in the browser too. Picture a dashboard where one of eight panels calls a reporting service that is down. Without a breaker, every refresh and route change waits for that service to time out again. With one, the next views show that panel's error at once while the other seven load normally.

• Keep one breaker per backend or host, in a module that every component shares, not one per component.
• Count only failures that mean "the service is unwell": network errors, timeouts, 502 to 504. A 404 or 422 says nothing about the service's health.
• Show cached or fallback data when open, with a short note, rather than an empty panel.

Offline: stop retrying and wait for the network

When the device has no network at all, every retry fails in a few milliseconds, burns battery, and fills your error tracker with noise. The better move is to notice that the device is offline, pause the retry timers, tell the user, and try again when the network returns.

What navigator.onLine can and cannot tell you

The browser exposes navigator.onLine, a boolean, and fires offline and online events on window when it changes. They are supported everywhere and cost nothing. But they answer a narrower question than their names suggest.

true can be wrong in many everyday ways: hotel Wi-Fi with a login page (a captive portal) that answers every request with its own page, a router whose internet link is down, a VPN that dropped, a company firewall that blocks your API, or a train connection so weak that nothing gets through. MDN says it plainly: you cannot assume that true means the browser can reach the internet.

So the reliable signal is your own requests. If the browser says online but three requests in a row fail with network errors, treat the app as offline for practical purposes. The first request that succeeds is the real proof that you are back.

Pausing and resuming the retry loop

function waitForOnline(signal: AbortSignal): Promise<void> {
  if (navigator.onLine) return Promise.resolve();
  return new Promise((resolve, reject) => {
    signal.throwIfAborted();
    const done = () => { cleanup(); resolve(); };
    const stop = () => { cleanup(); reject(signal.reason); };
    const cleanup = () => {
      window.removeEventListener("online", done);
      signal.removeEventListener("abort", stop);
    };
    window.addEventListener("online", done);
    signal.addEventListener("abort", stop);
  });
}

// In withRetry, before each new attempt:
// await waitForOnline(signal);  // does not use up an attempt while offline

• Do not count offline time as failures. Failures while navigator.onLine is false should not use up attempts, spend the retry budget, or open a circuit breaker. The service did nothing wrong.
• Resume with a little jitter. When an office's Wi-Fi comes back, hundreds of laptops fire online in the same second. A random delay of up to a second or two spreads them out.
• Refetch when the user comes back. Many apps also refetch stale data on online and when the tab becomes visible again. TanStack Query does both by default (refetchOnReconnect, refetchOnWindowFocus), and by default pauses queries and mutations while the browser reports offline.
• Queue writes that must not be lost. Save them, with their idempotency keys, in IndexedDB so they survive a reload, and send them in order when the network returns. The background sync lesson covers this in depth, including the Background Sync API, which only Chromium-based browsers support.

What the user should see

Being offline is a state of the app, not an error message. Keep showing the data you already have, mark it as possibly out of date, and put one calm banner at the top instead of an error in every panel. Buttons that need the network should say so, and actions the app can queue should say they will be sent later.

In real products, and what the user feels

You will rarely write all of this from scratch. Data libraries and SDKs ship retry policies, and the useful skill is knowing their defaults well enough to change them on purpose.

Defaults in common libraries

Note the TanStack Query row: its default delay has no randomness, and the default retry: 3 also retries a 404. Most teams pass a retry function that checks the status and a retryDelay such as (attempt) => Math.random() * Math.min(1000 * 2 ** attempt, 30_000).

The user's side of a retry

Retries are invisible when they work, and that is the goal. The first retry or two should be silent: the spinner keeps turning. If the wait goes past about 2 seconds, change the message to something honest like "Still loading..." or "Reconnecting...", so the user knows the app has not frozen. When the loop gives up, show an error that says what failed, keeps every field the user typed, and offers a Retry button.

• Keep stale data on screen. A list that failed to refresh should keep its old rows with a small "Could not refresh" note, not turn into an error page.
• Errors per panel, not per page. One failing widget should not blank the whole dashboard. This is where timeouts and circuit breakers pay off.

Accessibility

Screen reader users cannot see a spinner change to "Reconnecting". Put connection and loading status in a live region, an element with role="status" (which is polite: it waits for the screen reader to finish speaking). Announce changes of state, such as "You are offline", "Back online", or "Could not load orders", not every attempt.

Do not move focus to the error or to the Retry button on your own: the user may be in the middle of a form. Make Retry a real button, and when it is disabled during a 429 cool-down, say why in text.

Performance and security

• Battery and data. A tight retry loop on a phone keeps the radio awake. Backoff, the offline pause, and pausing background retries in hidden tabs (check document.visibilityState) all save battery.
• Do not retry login failures. A 401 from a login form means wrong credentials. Retrying can trip the account lockout or the server's brute-force protection.
• Respect rate limits as a contract. A client that ignores 429 and Retry-After may get its API key or IP blocked. The limit protects everyone else on the server.
• Keys must be unguessable. Use crypto.randomUUID(), not a counter or a timestamp, for idempotency keys, and the server should scope keys to the user who sent them.

Using it in a system design interview

Retries come up in almost every frontend design question, usually as "what happens if the request fails?" or "what if the API is slow?". It is rarely the main topic, so the skill is to give a complete, compact answer in a minute, and go deeper only if the interviewer asks.

When to bring it up

• In the API or data-fetching part of the answer, right after you list the endpoints: one sentence on timeouts and which calls retry.
• Whenever a write costs money or cannot be undone: payments, bookings, sending a message. Say "idempotency key" here, unprompted.
• When the design has many clients reconnecting at once: chat, live dashboards, polling. Say "backoff with jitter" and mention the thundering herd.

A one-minute answer

Here is a compact version you can adapt: "Every request has a timeout, set from p99 latency, with AbortSignal.timeout merged with the component's cancel signal. Reads retry up to 3 times on network errors, 408, 429, and 502 to 504, with exponential backoff and full jitter, and never sooner than Retry-After. Other 4xx errors show a message, and a 500 shows one with a Retry button."

Then the writes and the edge cases: "Writes retry only with an idempotency key made when the user starts the action. Retries pause while offline, and a circuit breaker per backend stops calls to a service that is down, so the rest of the page stays fast."

What to draw: the retry state machine from this lesson (send, classify, wait, done or failed) next to the client box, or a short sequence of a POST with an Idempotency-Key that times out and is retried. Either takes 30 seconds and shows you think about failure as part of the design.

How deep to go, by level

Mistakes that cost points

• Saying "we retry on failure" without saying which failures, how many times, or how long apart.
• Retrying a payment POST with no idempotency key, or making a new key for each retry.
• Fixed retry intervals for thousands of clients, with no mention of jitter.
• No timeout, so the retry logic never runs on a hung request.