cf.completefrontend
My progress
FUNDAMENTALS · MODULE 9

In-memory caches

Data your app keeps as live JavaScript objects in the tab's memory, keyed and bounded, so that going back, switching tabs, or opening the same thing twice is instant: the fastest cache you have, and the easiest one to let grow out of control.

Why apps keep data in memory

Picture a mail app. You open your inbox, which loads 50 threads, then click a thread, read it, and press Back. Without an in-memory cache, the inbox component was unmounted, its state went with it, and the app fetches the 50 threads again. The user sees a spinner, the list jumps back into place, and their scroll position is lost.

The browser's HTTP cache does not save you here. Even on a fresh hit, the app still waits for an asynchronous read, then parses the JSON and builds its objects again. And many API responses are sent with Cache-Control: no-store or private, no-cache, so there is often nothing to hit at all.

An in-memory cache keeps the parsed result as ordinary JavaScript objects in the page's heap (the memory where the JavaScript engine stores objects). When the inbox mounts again, the app reads the 50 threads from a Map in well under a millisecond, paints them at once, and then quietly checks the server for anything new.

Where the heap cache sits among the other caches

A frontend has several caches stacked on top of each other. The heap cache is the top layer, the one closest to your components.

  • JS heap cache: live objects in a Map or a library store. Read in microseconds, synchronously. Lives as long as the tab and is visible only to it.
  • IndexedDB and other client storage: structured data on disk. Read in milliseconds, asynchronously, then copied back into objects. Survives reloads.
  • Service worker cache: whole Response objects stored by your worker. Survives reloads and works offline, but still costs an asynchronous read and a parse.
  • HTTP cache and CDN: responses stored by the browser and the edge, steered by headers. No code needed, but every read still pays for parsing.
A stack of four client-side caches, nearest first. In this tab only: the JS heap cache, parsed objects in a Map, read in under a millisecond and synchronously. Below it, in browser storage that survives a reload: IndexedDB, structured data on disk, read asynchronously and copied back into objects; the service worker cache, holding Response objects, read asynchronously and parsed; and the HTTP cache, bytes steered by headers, read asynchronously and parsed.This tab only: gone on reloadBrowser storage: survives reloadJS heap cacheparsed objects in a Mapunder 1 ms, syncIndexedDBstructured data on diskms, async + copyService worker cacheResponse objectsms, async + parseHTTP cachebytes, steered by headersms, async + parsefastestlasts longest
Only the top layer hands components objects that are ready to render, with no await and no parse. It is also the only layer that disappears when the tab closes or reloads.

Each layer down is slower and lasts longer. The heap cache is the only one that hands you objects that are ready to render, with no await and no parse. That is why the main data libraries (TanStack Query, SWR, Apollo Client, Relay, RTK Query) are each built around one.

What a heap cache buys you

  • Instant back and forward. Going back to a list shows the old data on the first frame, with no spinner or layout jump, so scroll restoration works.
  • Shared reads. Ten components that need the current user read one entry instead of firing ten requests (request deduplication covers the in-flight half).
  • Stale-while-revalidate in the UI. Show what you have now, fetch a fresh copy in the background, and swap it in when it arrives.
  • Cheaper derived work. Memoized results, such as a sorted view of 5,000 rows or a parsed Markdown document, are computed once per input.

What it costs

Memory is not free. Every entry you keep stays in the heap until you delete it, and the garbage collector cannot help you: a Map holds strong references, so anything in it is reachable and stays alive. A cache with no limit is a memory leak with a nice name.

The other cost is correctness. A heap cache can show data that is out of date, data from a user who just logged out, or data shaped for an older app version. Most of this lesson is about keeping the cache small and keeping it right.

Check your understanding · Exercise 1

A product page's API response is a fresh hit in the HTTP cache. Which of these still cost time on every visit if the app has no in-memory cache? Select all that apply.

The mental model: a map with rules

Every in-memory cache, from a ten-line helper to Apollo Client, is the same idea: a map from keys to entries, plus a few rules about what goes in, what comes out, and when things leave. Once you see that shape, each library is a different set of rules.

Think of a small desk next to a big filing room. The filing room (the server) has everything, but each trip takes a while, so the desk holds the folders you used recently. When the desk is full, you put one back, and the rule for which one is the eviction policy. How long a folder can sit there before you double-check it is the freshness rule.

What an entry holds

A useful entry carries a little metadata besides the value, so the rules can decide without guessing.

ts
type Entry<T> = {
  data: T;              // the parsed, ready-to-render value
  fetchedAt: number;    // when it came from the server (ms)
  lastUsedAt: number;   // for LRU: when a reader last asked for it
  hits: number;         // for LFU: how often it was read
  size: number;         // rough bytes, for a size budget
  observers: number;    // components using it right now
};

const cache = new Map<string, Entry<unknown>>();
The data plus the facts the policies need: age, recency, frequency, size, and who is watching.

Not every cache needs every field. A memo cache for a pure function needs only the key and the value. A server-data cache needs fetchedAt for freshness and observers so it never throws away data a mounted component is showing.

Four operations

  1. Get. Build the key and look it up. If the entry is there, it is a hit: return it and update lastUsedAt and hits. Otherwise it is a miss.
  2. Fill. On a miss, fetch or compute the value and store it with the time and an estimated size. If a fetch for the same key is already in flight, join it.
  3. Evict. If the cache is now over its limit (entries or bytes), remove entries chosen by the policy until it fits, skipping entries with observers.
  4. Invalidate or clear. When the app learns the data changed (a mutation, a push message, logout), mark entries stale or delete them. Events drive this, not the size limit.
A flowchart of one cache read. Start with get key. Is there an entry? If no, it is a miss: fetch, then store the entry and evict over the limit, then return the new data. If yes, is it still fresh? If yes, it is a hit: return it at once. If no, return the stale data now and refetch in the background.get(key)entry exists?nomiss: fetchstore, evictif over the limitreturn new datayesstill fresh?yeshit: return itnoreturn stale nowrefetch in background
Two separate questions on every read: is the entry there, and is it fresh? A stale entry is still shown; only a missing one makes the user wait.

A hit raises a second question: is the entry fresh? Libraries keep the two apart. TanStack Query's staleTime decides when cached data is refetched in the background, and its gcTime decides how long an entry with no observers stays in memory. Stale data is still shown; deleted data is not.

The number that tells you if a cache works

The hit rate is hits divided by total reads. A cache with a 90% hit rate saves nine reads out of ten. At 5%, it is mostly cost: memory and a lookup on every read for almost no benefit.

Hit rate depends on three things: how the app reads (the access pattern), how much the cache can hold (the capacity), and which entries it keeps (the policy). The lab in the next chapter lets you change all three. Its surprise is that a cache holding more entries with the wrong policy can lose to a smaller one with the right policy.

Think first

Why should eviction skip entries that a mounted component is currently showing, even if they are the least recently used?

Eviction policies: who leaves when the cache is full

When a full cache gets a new entry, its eviction policy picks a victim. The ideal policy evicts the entry needed furthest in the future, but nobody knows the future, so every real policy guesses from the past.

The five policies you will meet

  • LRU (least recently used): evict the entry read longest ago. It is the default almost everywhere because people go back to what they just saw.
  • LFU (least frequently used): evict the entry with the fewest reads. It resists bursts of one-time reads, but remembers old popularity too long unless the counts decay.
  • FIFO (first in, first out): evict the oldest insert, however often it is read. Cheap, but usually worse than LRU.
  • TTL (time to live): evict an entry once it is older than a fixed age, say 5 minutes. It is a freshness rule more than a size rule: a burst of new keys can still grow the cache without bound.
  • Size-aware: count bytes instead of entries, so a 2 MB report costs more than a 200-byte record. Usually combined with LRU: evict the least recent entries until the bytes fit.

Real caches combine them. The lru-cache npm package takes a max count, an optional maxSize with a sizeCalculation function, and an optional ttl. TanStack Query gives each inactive entry a timer (gcTime, 5 minutes by default): a TTL that starts only when no component is watching.

Access patterns decide the winner

No policy wins everywhere. What matters is the access pattern, the shape of the reads. Three shapes cover most frontend apps:

  • Hot set: a few keys get most of the reads (the current user, the open project), and a long tail gets the rest. LFU beats LRU when the cache is small, because the tail cannot push out the hot keys.
  • Scan: the reader walks through more keys than fit, in order, then starts again, like paging through 30 search results with room for 20. LRU, FIFO, and plain LFU all score zero: each new key evicts exactly the one needed next.
  • Shifting focus: the hot set moves, say from project A to project B. LRU follows at once. LFU keeps project A's high counts and evicts project B's new entries until the counts decay.
Pattern:
Policy:
Capacity:

70% of reads go to 5 popular keys, the rest to 35 others. At most 8 entries. LRU: 67% hits, peak 8 entries and 336 KB. Same stream with the other policies: LFU 74%, TTL 57%.

120 reads of the hot set pattern through a LRU cache of 8 entries. 80 of 120 reads were hits, a hit rate of 67%. Peak memory was 336 KB in 8 entries. The other policies on the same stream: LFU 74%, TTL 57%.120 reads, in orderhitmisshit rate: 67%peak memory: 336 KB8 entries at the peakLFU 74%, TTL 57%other policies, same reads
Each small square is one read, in order, left to right and top to bottom: filled squares are hits. Memory is the total size of the cached responses at their peak; each response is 8 to 72 KB.

The lab runs the same 120 reads through each policy at 4, 8, or 16 entries. On the loop, LRU and LFU score 0% until the cache holds all 12 keys, then 90%: once the cache covers the working set (the keys in use), the policy barely matters.

On the hot set, LFU beats LRU at 4 and 8 entries; on shifting focus, LRU wins clearly because LFU clings to the first half's counts. TTL ignores the capacity, so its memory depends only on how many keys arrive within its time window.

Decision Which eviction policy should a frontend data cache use?

LRU with an entry limit

A Map in recency order; overflow deletes the first key.

Pros

  • O(1) operations with a plain Map
  • Matches back-and-forward browsing

Cons

  • A scan bigger than the cache flushes the hot set
  • A 2 MB entry counts the same as a 200-byte one

LRU with a byte budget

Same order, plus a running total of estimated sizes; evict from the front until the total fits.

Pros

  • Bounds memory, the thing you actually care about

Cons

  • Needs a size estimate per entry, which is approximate in JavaScript

LFU with decay

Count reads per key, halve the counts now and then, and evict the lowest count.

Pros

  • Keeps hot keys through a burst of one-off reads

Cons

  • Slow to follow a shifting focus
  • More bookkeeping and a decay rate to tune

TTL or idle timer only

Delete each entry a fixed time after it was written, or after its last observer left.

Pros

  • Simple, and old data leaves on its own

Cons

  • No hard bound on memory during bursts

What we pick: Use LRU, bounded by bytes when entry sizes vary a lot (lists, documents, images) and by count when they don't, plus an idle timer so unwatched data leaves on its own. Consider LFU only for a stable reference set that gets scanned past, after measuring.

Check your understanding · Exercise 2

A photo viewer keeps the last 20 decoded images in an LRU cache. Users often flick through an album of 25 photos from start to end, then start again. Which statements are true? Select all that apply.

Building an LRU cache with a Map

The classic LRU cache in textbooks is a hash map plus a doubly linked list: the map finds an entry in O(1), the list keeps the entries in recency order so moving one to the front and dropping one from the back are also O(1). It is a popular interview exercise, and it is about 60 lines of careful pointer code.

In JavaScript you get the list for free. A Map remembers the order in which keys were inserted, and iteration always walks them oldest first. Deleting a key and setting it again moves it to the end. So the first key is always the least recently used one, and map.keys().next().value reads it in O(1).

Read:
Holds at most:

Read some keys. Try A, B, C, D, then A again, then E: which one gets evicted?

A Map used as an LRU cache, holding at most 4 entries. In Map order, from least to most recently used: empty. 0 hits and 0 misses so far. Last Map calls: new Map().firstnewestnew Map()hits 0, misses 0
The Map's own insertion order is the recency order. Left is the first key, the next to go; right is the newest. map.keys().next().value always reads the leftmost key.

The whole cache in about 25 lines

ts
export class LRUCache<K, V> {
  #map = new Map<K, V>();
  readonly max: number;
  constructor(max: number) { this.max = max; }

  get(key: K): V | undefined {
    if (!this.#map.has(key)) return undefined;
    const value = this.#map.get(key) as V;
    this.#map.delete(key);      // move to the newest end
    this.#map.set(key, value);
    return value;
  }

  set(key: K, value: V): this {
    this.#map.delete(key);      // an overwrite also counts as a use
    this.#map.set(key, value);
    if (this.#map.size > this.max) {
      const oldest = this.#map.keys().next().value as K;
      this.#map.delete(oldest);
    }
    return this;
  }

  has(key: K) { return this.#map.has(key); }
  get size() { return this.#map.size; }
}
A Map in recency order: read or write moves a key to the end, overflow deletes the first key.

Three details are easy to get wrong. get checks has rather than testing the value, so a cached 0, false, or empty string still counts as a hit. set deletes first, so overwriting a key refreshes its position instead of leaving it where it was. And has does not move the key: peeking should not count as a use, or a debug panel that lists the cache would change what gets evicted.

Testing it with node:test

Node ships a test runner, so a cache like this can be tested with no dependencies. Save the class as lru.ts and the tests as lru.test.js, then run node --test. The tests pin down the three rules above, which are exactly the ones a refactor tends to break.

js
import test from "node:test";
import assert from "node:assert/strict";
import { LRUCache } from "./lru.ts";

test("evicts the least recently used key", () => {
  const lru = new LRUCache(2);
  lru.set("a", 1).set("b", 2);
  lru.get("a");                 // a is now the newest
  lru.set("c", 3);              // so b is evicted
  assert.equal(lru.has("b"), false);
  assert.deepEqual([lru.get("a"), lru.get("c")], [1, 3]);
});

test("an overwrite refreshes the key", () => {
  const lru = new LRUCache(2);
  lru.set("a", 1).set("b", 2).set("a", 10).set("c", 3);
  assert.equal(lru.size, 2);
  assert.equal(lru.get("a"), 10);
  assert.equal(lru.has("b"), false);
});

test("falsy values are hits", () => {
  const lru = new LRUCache(2).set("zero", 0);
  assert.equal(lru.get("zero"), 0);
});
Three small tests that cover recency, overwrite, and falsy values. All three pass on Node 22.18 and later, which runs a .ts file by stripping its types.

From an entry count to a byte budget

Turning the count into a byte budget takes one more field per entry and a running total. Each entry stores its estimated size; set adds it, eviction subtracts it, and the loop walks from the oldest key until the total fits again. Deleting the entry you are currently visiting inside a for...of over a Map is safe in JavaScript.

ts
set(key: K, value: V): this {
  const size = this.sizeOf(value);
  if (size > this.maxBytes) return this;  // never cache something bigger than the budget
  this.delete(key);                        // subtracts the old size if present
  this.#map.set(key, { value, size });
  this.#bytes += size;
  for (const [k, e] of this.#map) {        // oldest first
    if (this.#bytes <= this.maxBytes) break;
    this.#map.delete(k);
    this.#bytes -= e.size;
  }
  return this;
}
The byte-budget version of set. sizeOf is any estimate you trust; the next chapters cover how rough it can be.

For production you will usually reach for a tested package such as lru-cache or for the cache built into your data library. Writing your own is still worth it for small, hot paths such as memoizing a parser, and it is the version you will be asked to write on a whiteboard.

Check your understanding · Exercise 3

A teammate's LRU get looks like this: const v = map.get(key); if (v) { map.delete(key); map.set(key, v); } return v;. Which problems does it have? Select all that apply.

Cache by query key

A cache is only as good as its keys. The key has to answer one question: "would the server give the same answer to this request?". If two requests would get the same answer, they should share a key. If they would get different answers, they must not.

Get it wrong in one direction and you have a collision: two different requests share a key, so one of them shows the other's data. Get it wrong in the other direction and you have fragmentation: the same request gets a different key each time, so every read is a miss and the cache fills with copies.

What goes into a key

Put in everything that changes the response, and nothing that doesn't. For a list of tasks that usually means the resource, the parent it belongs to, and the filters, sort, and page.

One query key, read from general to specific: "projects", the resource; 42, the parent id; "tasks", the child list; and an object of params, page 2 and status open, with its keys sorted and defaults filled. The first two parts form a prefix that invalidates the whole project. Below, three things that must never be in a key: Date.now(), a render count, and a Set or Map.one key, general to specificresource"projects"parent id42child list"tasks"params: sorted, defaults filled{ page: 2, status: "open" }prefix: invalidates the whole projectnever in a key: it makes a new key on every call or collidesDate.now()a render counta Set or Map
Every part changes the server's answer, and nothing else is in it. The prefix ["projects", 42] reaches every list and filter under the project in one call.
  • Resource and parent first: ["projects", 42, "tasks", ...]. Going from general to specific lets you invalidate a whole branch by prefix.
  • Every parameter the server reads: filters, sort, page or cursor, page size, search text, and locale if the server translates.
  • Nothing the server ignores: a render counter, a callback, a Date.now() timestamp, or a whole props object. Each makes a new key on every call.
  • Defaults filled in: { page: 1 } and {} usually mean the same request, so fill defaults before building the key.
  • Inputs normalized like the server does: if search ignores case and outer spaces, trim and lowercase first, so "Shoes " and "shoes" share one entry.

Stable keys: the JSON.stringify trap

The quick way to turn parameters into a string key is JSON.stringify(params). It has one trap: it keeps object properties in insertion order. { status: "open", page: 2 } and { page: 2, status: "open" } become two different strings, so two components that build the same filter in a different order get two cache entries and two requests.

ts
// Same output for objects with the same content, whatever the key order.
export function stableKey(value: unknown): string {
  return JSON.stringify(value, (_k, v) =>
    v && typeof v === "object" && !Array.isArray(v)
      ? Object.fromEntries(Object.entries(v).sort(([a], [b]) => (a < b ? -1 : 1)))
      : v,
  );
}

stableKey(["tasks", { status: "open", page: 2 }]);
stableKey(["tasks", { page: 2, status: "open" }]);
// both: '["tasks",{"page":2,"status":"open"}]'
Sort object keys at every level, keep array order (array order is meaningful). TanStack Query hashes its query keys the same way.

A few more JSON.stringify quirks: properties whose value is undefined are dropped, so { q: undefined } and {} share a key (usually what you want). Date objects become ISO strings, and NaN and Infinity become null. A Map or a Set becomes {}, a silent collision, so keep keys to plain arrays, objects, strings, numbers, and booleans.

Hierarchical keys make invalidation cheap

Libraries such as TanStack Query and SWR accept arrays as keys. TanStack Query adds prefix matching: invalidateQueries({ queryKey: ["projects", 42] }) marks every entry whose key starts with those two items as stale, which covers the project and its task lists for every filter. The cache invalidation lesson goes deeper; you only get that power if you design the keys as a tree from the start.

Keys designed as a tree, from general to specific. One prefix call reaches every list and filter under a parent.
KeyWhat it cachesInvalidated by prefix
["projects", 42]Project 42's header data["projects"], ["projects", 42]
["projects", 42, "tasks", { status: "open", page: 1 }]First page of open tasksthe two above, plus ["projects", 42, "tasks"]
["projects", 42, "tasks", { status: "done", page: 1 }]First page of done taskssame as the row above
["me"]The signed-in useronly ["me"]

Who is asking is part of the answer

The server's answer to GET /api/me/tasks depends on who is signed in, but the URL doesn't say so. In a single-account app, clear the cache when the user changes (chapter 9). In an app with several accounts open at once, like a mail client with a switcher, put the account id first in every key, or one account's data will show in another's view.

Check your understanding · Exercise 4

Which of these query keys is a collision or a fragmentation bug? Select all that apply.

Per-query caches and normalized entity caches

There are two ways to lay out what you store under those keys. The difference decides how much memory you use, and whether an edit on one screen shows up on the others.

Per-query (document) caches

A per-query cache stores each response whole, under its query key. ["tasks", { status: "open" }] holds the array the server sent; ["task", 17] holds the detail object. TanStack Query, SWR, and RTK Query work this way by default. It is simple, it needs no knowledge of your data's shape, and evicting an entry frees exactly that response.

The catch is duplication. Task 17 appears in the open list, in the "assigned to me" list, in search results, and in its own detail entry: four copies of the same object. When the user renames it in the detail view, the three list copies still show the old name until each one is refetched or patched.

Normalized entity caches

A normalized cache splits every response into entities and stores each entity once, under a type and id such as Task:17. Lists and query results hold references ({ __ref: "Task:17" }) instead of copies. Apollo Client's InMemoryCache does this using __typename and id; Relay keeps a record store keyed by global ids. The normalization lesson covers the data side.

Two halves. Left, a per-query cache: the open list, my list, and the task 17 detail each hold their own copy of task 17. After a rename in the detail view, only the detail copy says Ship v3; the two list copies still say Ship v2. Right, a normalized cache: the open list, my list, and the detail all point at one record, Task:17, which says Ship v3, so every view shows the new name.per query: copiesopen listTask 17: "Ship v2"my listTask 17: "Ship v2"detail 17Task 17: "Ship v3"3 copies, 1 updatednormalized: one recordopenminedetailTask:17"Ship v3"1 record, every view updates
Copies drift apart after an edit; references can't. The price is that every object needs a type and an id, and memory can only be freed once nothing points at a record.

Now the rename writes one record, and every list that points at Task:17 renders the new name on the next read. With heavy overlap the cache is also smaller: a feed of 200 posts by 30 authors stores 30 author records rather than 200 copies.

What normalization costs

  • Every entity needs a stable id and a type. Objects without one, such as a stats summary, are stored inside their parent and duplicated again.
  • Lists are still per-query. The cache cannot know that a new task matches tasks(status: open), so you add it by hand or refetch the list.
  • Reads cost more. Each read rebuilds the response by following references, so libraries memoize the results.
  • Eviction is harder. You cannot free Task:17 while another list still points at it. Freeing memory becomes a reachability problem, like a small garbage collector: Apollo has cache.evict and cache.gc, and Relay frees records that no retained query needs.

Decision Should the app's server-data cache be per-query or normalized?

Per-query cache

Store each response under its key (TanStack Query, SWR, RTK Query); after a mutation, invalidate or patch the affected keys

Pros

  • Simple to reason about and debug
  • Works with any API shape

Cons

  • Duplicates entities that appear in many queries
  • An edit must invalidate every list it touches

Normalized entity cache

Store each entity once by type and id; queries hold references (Apollo, Relay)

Pros

  • An edit shows everywhere at once
  • Less memory when entities overlap a lot

Cons

  • Needs ids and types on every object
  • Lists still need manual updates
  • Garbage collection is harder

Per-query with a small normalized slice

Per-query, plus a small normalized store for the few entities edited in many places, such as the current user

Pros

  • Most of the consistency benefit for little complexity

Cons

  • Two sources of truth to keep in sync

What we pick: For a REST app with modest overlap, a per-query cache with good keys and prefix invalidation is simpler and usually better. When the same entities show up in many views and change often (feeds, chat, project tools) and every object has an id, a normalized cache pays for its complexity. In an interview, the reasoning matters more than the choice.

Think first

A user scrolls an infinite feed for an hour and loads 3,000 posts. With a normalized cache, the feed query is then removed. Is the memory freed?

Memory limits and measuring size

A heap cache shares the tab's memory with your code, the DOM, decoded images, and the framework. How much a tab may use depends on the device, the browser, and what else is running, but a phone with 3 GB of RAM and twenty open tabs will not let your page grow forever.

What happens when memory runs short

  • Everything slows down first. A bigger heap means more garbage-collection work: janky scrolling and slow input on low-end phones, long before any crash.
  • Background tabs get discarded. Chrome's Memory Saver and mobile browsers drop inactive tabs under memory pressure, so the user comes back to a full reload and an empty cache.
  • The tab crashes. A page that hits the engine's heap limit shows an out-of-memory error page, and mobile systems may end it even sooner.

So the budget is your choice. A reasonable starting point is the low tens of megabytes for server data; above that, move data to IndexedDB or stop caching it.

Where the bytes actually go

Most JSON data is smaller than people fear. As JSON text, a user record is about 1 KB, a page of 50 tasks about 25 KB, and a long rich-text document a few hundred KB. The parsed objects can take a few times more, but thousands of entries still fit in a modest budget.

A bar chart of typical cache entries, drawn to scale. A user record as JSON text: about 1 KB. A page of 50 tasks as JSON: about 25 KB. A long rich-text document: about 300 KB. These three are too small to see at this scale. One decoded 1920 by 1080 frame: 8.3 MB. One decoded 12-megapixel photo: 48 MB.size per entry, to scalea user record (JSON)about 1 KBa page of 50 tasks (JSON)about 25 KBa long rich-text documentabout 300 KBone decoded 1920 x 1080 frame8.3 MBone decoded 12-megapixel photo48 MB
Data is cheap; pixels are not. The JSON sizes are the text (parsed objects take a few times more); a decoded image costs width x height x 4 bytes whatever its file size, so image caches need a byte budget of their own.

A decoded image costs width times height times 4 bytes, whatever its file size: about 8.3 MB for a 1920 by 1080 frame and 48 MB for a 12-megapixel photo. Caching ImageBitmap objects or canvases by count is the fastest way to crash a mobile tab. Blob URLs are similar: URL.createObjectURL keeps the blob alive until URL.revokeObjectURL, even after you drop every other reference.

Estimating an entry's size

JavaScript has no sizeof: the engine decides how objects are laid out, and stores a string in one or two bytes per character depending on its content. So any size you compute is an estimate, which is fine: a budget needs a number that grows with the real cost, not an exact one.

  • Use the text you already have. text.length before JSON.parse is a free estimate, usually within a small factor of the real heap cost.
  • For binary data, use exact sizes: blob.size, buffer.byteLength, or width * height * 4 for decoded pixels.
  • Don't call JSON.stringify on every write just to measure a large value: it costs about as much time as the parse you saved.
  • Calibrate once. With the cache full, take a heap snapshot in the Chrome DevTools Memory panel, compare the retained size of the cache's Map with your estimate, and scale the estimate to match.

For field data, performance.measureUserAgentSpecificMemory() estimates the page's memory, but only in Chromium-based browsers and on cross-origin isolated pages (served with the Cross-Origin-Opener-Policy and Cross-Origin-Embedder-Policy headers). The older performance.memory is non-standard and too coarse for one cache.

WeakMap and WeakRef caches, and their limits

JavaScript has two tools that let the garbage collector help. A WeakMap keeps a value only as long as its key object is reachable from somewhere else. That suits caching facts about an object you don't own: when the object goes away, so does the cached value, with no eviction code.

ts
// Derived data that lives exactly as long as its source object.
const sortedCache = new WeakMap<Task[], Task[]>();

export function sortByDue(tasks: Task[]): Task[] {
  let sorted = sortedCache.get(tasks);
  if (!sorted) {
    sorted = [...tasks].sort((a, b) => a.due - b.due);
    sortedCache.set(tasks, sorted);
  }
  return sorted;
}
A memo keyed by the array itself. When the server-data cache drops that array, the sorted copy can be collected too.

Its keys must be objects (or symbols not created with Symbol.for), so you cannot key a WeakMap by a URL or an id string, which is what a data cache needs. It has no size and cannot be iterated, so you cannot measure, list, or clear it except by replacing it.

A WeakRef goes the other way: it holds a value weakly, and deref() returns undefined once the value was collected. A Map of string keys to WeakRefs looks like a cache that frees itself, but the engine collects whenever it likes, so values may vanish at once or never. MDN advises avoiding WeakRef where you can; use it, if at all, behind a real size limit.

Check your understanding · Exercise 5

A gallery caches decoded thumbnails as ImageBitmaps in a Map keyed by photo id, with a limit of 500 entries. Each bitmap is 400 by 300 pixels. Which statements are true? Select all that apply.

Sharing a cache across components and tabs

A cache only saves work if the readers share it. Inside one tab that is a question of where the cache object lives and how components hear about changes. Across tabs the answer is harder: they do not share memory at all.

One instance per app, not per module

The simplest shared cache is a module-level const cache = new Map(). Every component that imports the module sees the same map. It works in a client-only app, and it has two traps.

  • On the server it is shared by every request. With server-side rendering, a module is loaded once per server process, so a cache filled while rendering Alice's page is still there when the server renders Bob's: a data leak between users. Create the cache per request on the server; TanStack Query's docs create the QueryClient inside the app, not at module level, for this reason.
  • In tests it is shared by every test. One test's cached data makes the next pass or fail by accident; a cache created by a provider can be fresh for each test.

So the usual shape is a cache object created at the app root and passed down through context (QueryClientProvider, ApolloProvider, SWR's SWRConfig with a provider). Components read through a hook, and the hook subscribes the component to the keys it reads.

Telling components that an entry changed

A Map doesn't announce writes. A cache that drives UI adds a small subscription layer: each key keeps a set of listeners, a write calls them, and the hook re-renders. In React, useSyncExternalStore(subscribe, getSnapshot) is the built-in way to connect such a store without tearing (two components showing different versions in the same render).

One rule makes or breaks this: getSnapshot must return the same object when nothing changed, or React sees a change on every render and loops. Libraries go further with structural sharing: after a refetch, TanStack Query keeps the old object for every part of the response that is deeply equal, so an unchanged row keeps its reference and its memoized component skips the render.

A sequence diagram with four lifelines: Tab A, a BroadcastChannel, Tab B, and the API. Tab A sends PATCH /tasks/17 to the API and gets 200 OK, then updates its own cache. Tab A posts a message on the channel saying tasks changed. The channel delivers a message event to Tab B. Tab B invalidates its tasks entries and sends a GET for the open tasks to the API, which replies with the fresh list. Tab B now shows the new name.Tab ABroadcastChannelTab BAPIPATCH /tasks/17200 OKtasks changedmessage eventGET open tasksfresh listupdated locallyshows the new name
The tabs share an event, not memory. Each tab still holds and fetches its own copy; the message only tells it which keys to drop.

Each tab has its own heap

Open the app in two tabs and you have two caches, two sets of requests, and two copies of everything. Rename a task in tab A, and tab B keeps showing the old name until it refetches. Each tab runs its own JavaScript heap, even on the same origin.

The cheap fix is to share events, not data. A BroadcastChannel delivers messages to every same-origin tab, window, and worker that opened a channel with the same name. After a mutation, tab A posts which keys changed; every other tab invalidates those keys and refetches what it is showing.

ts
const channel = new BroadcastChannel("query-cache");

// After a successful mutation in this tab:
export function announceChange(prefix: unknown[]) {
  channel.postMessage({ type: "invalidate", prefix });
}

// In every tab (the sender does not receive its own message):
channel.onmessage = (event) => {
  if (event.data?.type === "invalidate") {
    queryClient.invalidateQueries({ queryKey: event.data.prefix });
  }
};
Broadcast the key prefix that changed, not the data. Each tab refetches only what it is showing.
Four ways to deal with one heap per tab, from cheapest to most involved.
ApproachWhat is sharedGood forWatch out for
BroadcastChannel invalidationEventsKeeping tabs consistent after editsEach tab still fetches its own copy
Broadcast the new dataSmall valuesTiny, hot values such as unread countsLarge payloads are copied to every tab
Persist to IndexedDBA snapshot on diskWarm start for new tabs and reloadsStale on read; clear it on logout too
SharedWorker owns the cacheOne cache for all tabsHeavy data shared by many tabsEvery read is an async message; Chrome on Android only from version 148

Check your understanding · Exercise 6

A Next.js app keeps export const cache = new Map() in a module and fills it while rendering pages on the server. What can go wrong? Select all that apply.

When to clear, and what goes wrong

Eviction is about space. Clearing is about correctness: there are moments when cached data is no longer safe to show, however much room is left. Most of the serious bugs with heap caches come from missing one of these moments.

The life of an entry

An entry is a small state machine. Created by a fetch, it is fresh for a while, turns stale when its freshness time runs out, becomes inactive when the last component using it unmounts, and is removed by eviction, by its idle timer, or by an explicit clear.

A state machine for one cache entry. Fetching goes to fresh when the data is loaded. Fresh goes to stale when staleTime ends. Stale goes back to fetching for a background refetch. Stale goes to inactive when the last component using it unmounts. Inactive goes to removed when its idle timer ends. Fresh goes straight to removed on evict or clear, and fetching goes straight to removed on logout, when clear runs while a request is in flight.fetchingfreshstaleinactiveremovedloadedstaleTimebackground refetchlast user unmountstimer endsevict or clear()on logout
Stale data is still shown; only removed data is gone. The arrow people forget is the one from fetching: a clear must also cancel the request, or its response brings the entry back.

Stale entries are still shown while a refetch runs; inactive entries wait in case someone comes back. Clearing jumps straight to removed from any state, including from fetching, which is the case people forget.

The moments that must clear the cache

  • Logout. Everything cached belongs to the user who just left. On a shared computer, the next person must not press Back and see the previous inbox.
  • User or account switch. Same as logout. In multi-account apps, clear the old account's keys, or keep the account id in every key.
  • Tenant, workspace, or role change. Entries fetched under old permissions may show things the user can no longer see.
  • App version change. New code may expect a different shape. Heap caches die with the old page, but persisted caches do not: stamp them with a version (TanStack Query's persister calls it buster) and discard on mismatch.
  • Locale or currency change. If the server formats or translates, cached strings are now wrong. Put the locale in the key or clear on change.

Logout, in the right order

Clearing is not one call. A request started before logout can resolve after the clear and write the old user's data back into the fresh cache. Do the steps in this order:

  1. Cancel in-flight requests (queryClient.cancelQueries(), or abort your own AbortControllers), so no late response refills the cache.
  2. Clear the heap cache: queryClient.clear() in TanStack Query, client.clearStore() in Apollo (not resetStore(), which also refetches every active query), or new Map() for your own.
  3. Delete persisted copies: IndexedDB stores, service worker caches with user data, and any snapshot used for warm starts.
  4. Tell the other tabs: post a logout message on a BroadcastChannel so they clear too.
  5. Navigate to the sign-in page. A full page load (location.assign) is the bluntest guarantee: it throws away the whole heap.

On the server side, a logout response can also send Clear-Site-Data: "cache", "storage" to ask the browser to wipe the HTTP cache and site storage for the origin. Browser support varies, so treat it as a second line, not the only one.

Failure modes in production

Six bugs that show up in real apps, and where to look first.
SymptomLikely causeFix
Memory climbs on every route changeNo limit and no idle timerAdd an LRU limit or gcTime; confirm with two heap snapshots
Tab crashes on phones onlyImages or buffers cached by countByte budget; close() bitmaps; revoke blob URLs
Previous user's data after loginNo clear on logout, or a late response refilled itCancel, then clear, then reload
Same request fires twice on one screenUnstable keys (key order, timestamps)Stable key function; fill defaults
Edit shows on one screen but not anotherPer-query copies not invalidatedPrefix invalidation, or normalize that entity
Other tab shows old dataOne heap per tabBroadcastChannel invalidation

Check your understanding · Exercise 7

A user logs out, and the app calls queryClient.clear() and then shows the sign-in form without reloading. A tester then signs in as a second user and briefly sees the first user's notifications. What could explain it? Select all that apply.

In real products and in the interview

You will rarely hand-write the main data cache of a production app. You will pick a library, and its defaults decide your memory use and your stale-data bugs.

What the common libraries do by default

A comparison of five data libraries. TanStack Query: per-query layout; unused data leaves after gcTime, 5 minutes by default; edits reach other views when you invalidate keys. SWR: per query; unused data stays by default; edits reach other views when you mutate keys. RTK Query: per query; unused data leaves after 60 seconds; edits spread through tags. Apollo Client: normalized; unused data leaves when you call evict and gc; edits reach other views by id. Relay: normalized; unused records are garbage collected when queries are released; edits reach other views by id.librarylayoutunused data leaveshow edits spreadTanStack Queryper querygcTime, 5 mininvalidate keysSWRper querystays by defaultmutate keysRTK Queryper queryafter 60 svia tagsApollo Clientnormalizedevict + gc()by idRelaynormalizedGC on releaseby id
The per-query libraries spread edits by key or tag; the normalized ones spread them by id. Know your library's default for unused data: it is your memory policy unless you set one.
  • TanStack Query keeps one entry per query key. Data is stale at once (staleTime: 0), so it refetches in the background on mount and window focus. An entry with no observers is removed after gcTime, 5 minutes by default.
  • SWR keeps entries in a Map for the life of the page by default, with no eviction, and revalidates on mount and focus. The provider option swaps in your own cache, such as an LRU.
  • RTK Query keeps one entry per endpoint and argument, and removes unused data after keepUnusedDataFor, 60 seconds by default. Tags connect mutations to the queries they invalidate.
  • Apollo Client normalizes into InMemoryCache by __typename and id. The entity store has no size limit of its own: you remove data with cache.evict and cache.gc.
  • Relay normalizes into a record store and garbage-collects records that no retained query needs, keeping the last few released queries (10 by default) for reuse.

Outside data libraries, the pattern is everywhere. Client routers keep the data of recently visited routes so Back is instant, Markdown and syntax-highlighting tools memoize parsed results by input, and image-heavy apps keep a small window of decoded images around the viewport. In every case the three questions from chapter 1 have clear answers: the key, the limit, and what clears it.

When to bring it up in an interview

Mention the in-memory cache whenever the design has navigation between a list and a detail view, the same data in several places, or a requirement such as "Back should be instant". Name it when you draw the client architecture, as a box between your components and the network layer.

  • Key: "Keyed by resource and parameters with a stable key function, as a tree so one prefix invalidates a project."
  • Layout: "Per-query by default; I'd normalize users and messages because they appear in many views and change often."
  • Limit: "Inactive entries leave after five minutes; images have their own byte budget of about 30 MB with LRU eviction."
  • Freshness: "Show cached data at once and revalidate in the background; mutations invalidate the affected prefixes."
  • Clearing: "On logout I cancel in-flight requests, clear the cache and persisted copies, broadcast to other tabs, and reload."

How deep to go by level

LevelWhat a strong answer shows
JuniorKnows a library caches by key; explains why Back is instant; knows to clear on logout
MidDesigns stable, hierarchical keys; picks per-query or normalized with a reason; sets limits and stale times
SeniorBudgets memory by bytes for heavy data; handles SSR per-request caches, multi-tab consistency, the logout race, and version changes; knows how to prove there is no leak

Check your understanding · Exercise 8

You are designing a team chat app. Which cache decisions are sound? Select all that apply.

Summary

  • A heap cache holds parsed, ready-to-render objects: it is the only cache layer that gives synchronous reads, which is what makes Back instant.
  • Every cache needs three answers: the key, the limit, and what clears it. Without a limit, a Map cache is a memory leak.
  • LRU is the right default for UI data; a Map gives you an O(1) LRU in about 25 lines. Bound by bytes when entry sizes vary, especially for images.
  • Keys must include every input that changes the answer and nothing else. Use a stable serializer and design keys as a tree for prefix invalidation.
  • Per-query caches are simpler; normalized caches keep shared, often-edited entities consistent at the cost of ids, manual list updates, and harder garbage collection.
  • Each tab has its own heap: broadcast invalidations across tabs, and create caches per request on the server.
  • Clear on logout, user switch, permission change, and version change, cancelling in-flight requests first so nothing refills the cache.

In the interview

Questions interviewers ask about this. Answer each out loud first, then compare.

Question 1

Why keep server data in a JavaScript cache when the browser already has an HTTP cache?

Question 2

Implement an LRU cache in JavaScript. What are the edge cases?

Question 3

How do you choose query keys, and what goes wrong with bad ones?

Question 4

Per-query cache or normalized cache: how do you decide?

Question 5

How do you keep an in-memory cache from becoming a memory leak?

Question 6

What happens to the cache when the user logs out, and when they have two tabs open?

Practice it

These design problems lean on in-memory caches.

References

Read next