Stale-while-revalidate cache

Write createSwrCache(fetcher, { maxAge }). fetcher(key) returns a promise of the data for key, like a function that fetches a user by ID. createSwrCache returns an object with one method, get(key), which always returns a promise of the value for key.

The first get for a key calls fetcher(key) and fulfills with its value once it arrives. The value is stored, and it's fresh for maxAge milliseconds, counted from when it arrived: while it's less than maxAge ms old, get fulfills with it without calling fetcher.

Once the stored value is maxAge ms old or more, it's stale. get still fulfills with it straight away, and also refreshes it in the background: it calls fetcher(key) during that get call, without waiting for the result. When the new value arrives, it replaces the old one and is fresh again.

Call fetcher at most once per key at a time. While a fetch for a key is in flight, a get for that key waits for it if there's no stored value yet, and gets the stale value without starting another fetch if there is.

If a background fetch fails, keep the stale value; the next get tries again. If the first fetch for a key fails, get rejects with its error, nothing is stored, and the next get calls fetcher again.

Keys are compared the way Map compares them, so 1 and "1" are different keys.

const users = createSwrCache((id) => fetchJson("/api/users/" + id), { maxAge: 60_000 });
 
await users.get(7); // nothing stored: fetches user 7, then fulfills with it
await users.get(7); // under a minute old: fulfills from the cache, no request
 
// a minute later:
await users.get(7); // stale: fulfills with the stored user at once, and refetches user 7 in the background
// once that request is done:
await users.get(7); // the refreshed user, fresh for another minute

What the tests check

1. 1fetches a missing key and fulfills with its value

   const clock = useFakeTimers();
   let version = 0;
   const fetcher = spy((key) => new Promise((resolve) => setTimeout(resolve, 100, key + " v" + ++version)));
   const cache = createSwrCache(fetcher, { maxAge: 1000 });
   let result;
   cache.get("a").then((value) => {
     result = value;
   });
   await clock.tickAsync(99);
   expect(result).toBeUndefined();
   await clock.tickAsync(1);
   expect(result).toBe("a v1");
   expect(fetcher).toHaveBeenCalledTimes(1);
   expect(fetcher).toHaveBeenCalledWith("a");
   

2. 2serves a fresh value from the cache, without fetching

   const clock = useFakeTimers();
   let version = 0;
   const fetcher = spy((key) => new Promise((resolve) => setTimeout(resolve, 100, key + " v" + ++version)));
   const cache = createSwrCache(fetcher, { maxAge: 1000 });
   cache.get("a");
   await clock.tickAsync(100); // "a v1" arrives
   await clock.tickAsync(999); // it's 999 ms old: still fresh
   const promise = cache.get("a");
   expect(promise).toBeInstanceOf(Promise);
   await expect(promise).resolves.toBe("a v1");
   expect(fetcher).toHaveBeenCalledTimes(1);
   

3. 3serves a stale value at once and refetches it in the background

   const clock = useFakeTimers();
   let version = 0;
   const fetcher = spy((key) => new Promise((resolve) => setTimeout(resolve, 100, key + " v" + ++version)));
   const cache = createSwrCache(fetcher, { maxAge: 1000 });
   cache.get("a");
   await clock.tickAsync(100); // "a v1" arrives
   await clock.tickAsync(1000); // it's 1000 ms old: stale
   let result;
   cache.get("a").then((value) => {
     result = value;
   });
   await flush();
   expect(result).toBe("a v1");
   expect(fetcher).toHaveBeenCalledTimes(2);
   

4. 4serves the refreshed value once the background fetch is done

   const clock = useFakeTimers();
   let version = 0;
   const fetcher = spy((key) => new Promise((resolve) => setTimeout(resolve, 100, key + " v" + ++version)));
   const cache = createSwrCache(fetcher, { maxAge: 1000 });
   cache.get("a");
   await clock.tickAsync(1100); // "a v1" arrived at 100 ms and is stale now
   cache.get("a"); // serves "a v1" and fetches "a v2"
   await clock.tickAsync(100);
   await expect(cache.get("a")).resolves.toBe("a v2");
   expect(fetcher).toHaveBeenCalledTimes(2);
   

5. 5starts one background fetch per key at a time

   const clock = useFakeTimers();
   let version = 0;
   const fetcher = spy((key) => new Promise((resolve) => setTimeout(resolve, 100, key + " v" + ++version)));
   const cache = createSwrCache(fetcher, { maxAge: 1000 });
   cache.get("a");
   await clock.tickAsync(1100); // "a v1" is stale
   const results = [];
   cache.get("a").then((value) => results.push(value));
   cache.get("a").then((value) => results.push(value));
   cache.get("a").then((value) => results.push(value));
   await flush();
   expect(fetcher).toHaveBeenCalledTimes(2);
   expect(results).toEqual(["a v1", "a v1", "a v1"]);
   

6. 6shares one fetch between gets for a missing key

   const clock = useFakeTimers();
   let version = 0;
   const fetcher = spy((key) => new Promise((resolve) => setTimeout(resolve, 100, key + " v" + ++version)));
   const cache = createSwrCache(fetcher, { maxAge: 1000 });
   const results = [];
   cache.get("a").then((value) => results.push(value));
   cache.get("a").then((value) => results.push(value));
   await clock.tickAsync(100);
   expect(fetcher).toHaveBeenCalledTimes(1);
   expect(results).toEqual(["a v1", "a v1"]);
   

7. 7keeps the stale value when a background fetch fails, and tries again

   const clock = useFakeTimers();
   let online = true;
   const fetcher = spy(async (key) => {
     if (!online) throw new Error("offline");
     return key + " v1";
   });
   const cache = createSwrCache(fetcher, { maxAge: 1000 });
   await cache.get("a");
   await clock.tickAsync(1000);
   online = false;
   await expect(cache.get("a")).resolves.toBe("a v1"); // stale; the background fetch fails
   await flush();
   await expect(cache.get("a")).resolves.toBe("a v1"); // still stale, so it fetches again
   await flush();
   expect(fetcher).toHaveBeenCalledTimes(3);
   

8. 8rejects when the first fetch fails, and fetches again next time

   const error = new Error("offline");
   let online = false;
   const fetcher = spy(async (key) => {
     if (!online) throw error;
     return key + " v1";
   });
   const cache = createSwrCache(fetcher, { maxAge: 1000 });
   await expect(cache.get("a")).rejects.toBe(error);
   online = true;
   await expect(cache.get("a")).resolves.toBe("a v1");
   expect(fetcher).toHaveBeenCalledTimes(2);
   

9. 9keeps keys apart, such as 1 and "1"

   const fetcher = spy(async (key) => typeof key + " " + key);
   const cache = createSwrCache(fetcher, { maxAge: 1000 });
   await expect(cache.get(1)).resolves.toBe("number 1");
   await expect(cache.get("1")).resolves.toBe("string 1");
   expect(fetcher).toHaveBeenCalledTimes(2);
   

Learn more

• Async patternsLesson
• Map & SetLesson