In-memory caches
Data your app keeps as live JavaScript objects in the tab's memory, keyed and bounded, so that going back, switching tabs, or opening the same thing twice is instant: the fastest cache you have, and the easiest one to let grow out of control.
Why apps keep data in memory
Picture a mail app. You open your inbox, which loads 50 threads, then click a thread, read it, and press Back. Without an in-memory cache, the inbox component was unmounted, its state went with it, and the app fetches the 50 threads again. The user sees a spinner, the list jumps back into place, and their scroll position is lost.
The browser's HTTP cache does not save you here. Even on a fresh hit, the app still waits for an asynchronous read, then parses the JSON and builds its objects again. And many API responses are sent with Cache-Control: no-store or private, no-cache, so there is often nothing to hit at all.
An in-memory cache keeps the parsed result as ordinary JavaScript objects in the page's heap (the memory where the JavaScript engine stores objects). When the inbox mounts again, the app reads the 50 threads from a Map in well under a millisecond, paints them at once, and then quietly checks the server for anything new.
Where the heap cache sits among the other caches
A frontend has several caches stacked on top of each other. The heap cache is the top layer, the one closest to your components.
- JS heap cache: live objects in a
Mapor a library store. Read in microseconds, synchronously. Lives as long as the tab and is visible only to it. - IndexedDB and other client storage: structured data on disk. Read in milliseconds, asynchronously, then copied back into objects. Survives reloads.
- Service worker cache: whole
Responseobjects stored by your worker. Survives reloads and works offline, but still costs an asynchronous read and a parse. - HTTP cache and CDN: responses stored by the browser and the edge, steered by headers. No code needed, but every read still pays for parsing.
Each layer down is slower and lasts longer. The heap cache is the only one that hands you objects that are ready to render, with no await and no parse. That is why the main data libraries (TanStack Query, SWR, Apollo Client, Relay, RTK Query) are each built around one.
What a heap cache buys you
- Instant back and forward. Going back to a list shows the old data on the first frame, with no spinner or layout jump, so scroll restoration works.
- Shared reads. Ten components that need the current user read one entry instead of firing ten requests (request deduplication covers the in-flight half).
- Stale-while-revalidate in the UI. Show what you have now, fetch a fresh copy in the background, and swap it in when it arrives.
- Cheaper derived work. Memoized results, such as a sorted view of 5,000 rows or a parsed Markdown document, are computed once per input.
What it costs
Memory is not free. Every entry you keep stays in the heap until you delete it, and the garbage collector cannot help you: a Map holds strong references, so anything in it is reachable and stays alive. A cache with no limit is a memory leak with a nice name.
The other cost is correctness. A heap cache can show data that is out of date, data from a user who just logged out, or data shaped for an older app version. Most of this lesson is about keeping the cache small and keeping it right.
Check your understanding · Exercise 1
A product page's API response is a fresh hit in the HTTP cache. Which of these still cost time on every visit if the app has no in-memory cache? Select all that apply.
The mental model: a map with rules
Every in-memory cache, from a ten-line helper to Apollo Client, is the same idea: a map from keys to entries, plus a few rules about what goes in, what comes out, and when things leave. Once you see that shape, each library is a different set of rules.
Think of a small desk next to a big filing room. The filing room (the server) has everything, but each trip takes a while, so the desk holds the folders you used recently. When the desk is full, you put one back, and the rule for which one is the eviction policy. How long a folder can sit there before you double-check it is the freshness rule.
What an entry holds
A useful entry carries a little metadata besides the value, so the rules can decide without guessing.
type Entry<T> = {
data: T; // the parsed, ready-to-render value
fetchedAt: number; // when it came from the server (ms)
lastUsedAt: number; // for LRU: when a reader last asked for it
hits: number; // for LFU: how often it was read
size: number; // rough bytes, for a size budget
observers: number; // components using it right now
};
const cache = new Map<string, Entry<unknown>>();
Not every cache needs every field. A memo cache for a pure function needs only the key and the value. A server-data cache needs fetchedAt for freshness and observers so it never throws away data a mounted component is showing.
Four operations
- Get. Build the key and look it up. If the entry is there, it is a hit: return it and update
lastUsedAtandhits. Otherwise it is a miss. - Fill. On a miss, fetch or compute the value and store it with the time and an estimated size. If a fetch for the same key is already in flight, join it.
- Evict. If the cache is now over its limit (entries or bytes), remove entries chosen by the policy until it fits, skipping entries with observers.
- Invalidate or clear. When the app learns the data changed (a mutation, a push message, logout), mark entries stale or delete them. Events drive this, not the size limit.
A hit raises a second question: is the entry fresh? Libraries keep the two apart. TanStack Query's staleTime decides when cached data is refetched in the background, and its gcTime decides how long an entry with no observers stays in memory. Stale data is still shown; deleted data is not.
The number that tells you if a cache works
The hit rate is hits divided by total reads. A cache with a 90% hit rate saves nine reads out of ten. At 5%, it is mostly cost: memory and a lookup on every read for almost no benefit.
Hit rate depends on three things: how the app reads (the access pattern), how much the cache can hold (the capacity), and which entries it keeps (the policy). The lab in the next chapter lets you change all three. Its surprise is that a cache holding more entries with the wrong policy can lose to a smaller one with the right policy.
Think first
Why should eviction skip entries that a mounted component is currently showing, even if they are the least recently used?
Because evicting them saves no memory. The component still holds a reference to the same objects, so the garbage collector cannot free them; you only lose the ability to share them.
Worse, the next component that asks for the key misses and fetches a second copy: two copies in memory and an extra request. That is why libraries count observers and start the garbage-collection timer only when the count drops to zero.
Eviction policies: who leaves when the cache is full
When a full cache gets a new entry, its eviction policy picks a victim. The ideal policy evicts the entry needed furthest in the future, but nobody knows the future, so every real policy guesses from the past.
The five policies you will meet
- LRU (least recently used): evict the entry read longest ago. It is the default almost everywhere because people go back to what they just saw.
- LFU (least frequently used): evict the entry with the fewest reads. It resists bursts of one-time reads, but remembers old popularity too long unless the counts decay.
- FIFO (first in, first out): evict the oldest insert, however often it is read. Cheap, but usually worse than LRU.
- TTL (time to live): evict an entry once it is older than a fixed age, say 5 minutes. It is a freshness rule more than a size rule: a burst of new keys can still grow the cache without bound.
- Size-aware: count bytes instead of entries, so a 2 MB report costs more than a 200-byte record. Usually combined with LRU: evict the least recent entries until the bytes fit.
Real caches combine them. The lru-cache npm package takes a max count, an optional maxSize with a sizeCalculation function, and an optional ttl. TanStack Query gives each inactive entry a timer (gcTime, 5 minutes by default): a TTL that starts only when no component is watching.
Access patterns decide the winner
No policy wins everywhere. What matters is the access pattern, the shape of the reads. Three shapes cover most frontend apps:
- Hot set: a few keys get most of the reads (the current user, the open project), and a long tail gets the rest. LFU beats LRU when the cache is small, because the tail cannot push out the hot keys.
- Scan: the reader walks through more keys than fit, in order, then starts again, like paging through 30 search results with room for 20. LRU, FIFO, and plain LFU all score zero: each new key evicts exactly the one needed next.
- Shifting focus: the hot set moves, say from project A to project B. LRU follows at once. LFU keeps project A's high counts and evicts project B's new entries until the counts decay.
70% of reads go to 5 popular keys, the rest to 35 others. At most 8 entries. LRU: 67% hits, peak 8 entries and 336 KB. Same stream with the other policies: LFU 74%, TTL 57%.
The lab runs the same 120 reads through each policy at 4, 8, or 16 entries. On the loop, LRU and LFU score 0% until the cache holds all 12 keys, then 90%: once the cache covers the working set (the keys in use), the policy barely matters.
On the hot set, LFU beats LRU at 4 and 8 entries; on shifting focus, LRU wins clearly because LFU clings to the first half's counts. TTL ignores the capacity, so its memory depends only on how many keys arrive within its time window.
Decision Which eviction policy should a frontend data cache use?
What we pick: Use LRU, bounded by bytes when entry sizes vary a lot (lists, documents, images) and by count when they don't, plus an idle timer so unwatched data leaves on its own. Consider LFU only for a stable reference set that gets scanned past, after measuring.
Check your understanding · Exercise 2
A photo viewer keeps the last 20 decoded images in an LRU cache. Users often flick through an album of 25 photos from start to end, then start again. Which statements are true? Select all that apply.
Building an LRU cache with a Map
The classic LRU cache in textbooks is a hash map plus a doubly linked list: the map finds an entry in O(1), the list keeps the entries in recency order so moving one to the front and dropping one from the back are also O(1). It is a popular interview exercise, and it is about 60 lines of careful pointer code.
In JavaScript you get the list for free. A Map remembers the order in which keys were inserted, and iteration always walks them oldest first. Deleting a key and setting it again moves it to the end. So the first key is always the least recently used one, and map.keys().next().value reads it in O(1).
Read some keys. Try A, B, C, D, then A again, then E: which one gets evicted?
map.keys().next().value always reads the leftmost key.The whole cache in about 25 lines
export class LRUCache<K, V> {
#map = new Map<K, V>();
readonly max: number;
constructor(max: number) { this.max = max; }
get(key: K): V | undefined {
if (!this.#map.has(key)) return undefined;
const value = this.#map.get(key) as V;
this.#map.delete(key); // move to the newest end
this.#map.set(key, value);
return value;
}
set(key: K, value: V): this {
this.#map.delete(key); // an overwrite also counts as a use
this.#map.set(key, value);
if (this.#map.size > this.max) {
const oldest = this.#map.keys().next().value as K;
this.#map.delete(oldest);
}
return this;
}
has(key: K) { return this.#map.has(key); }
get size() { return this.#map.size; }
}
Three details are easy to get wrong. get checks has rather than testing the value, so a cached 0, false, or empty string still counts as a hit. set deletes first, so overwriting a key refreshes its position instead of leaving it where it was. And has does not move the key: peeking should not count as a use, or a debug panel that lists the cache would change what gets evicted.
Testing it with node:test
Node ships a test runner, so a cache like this can be tested with no dependencies. Save the class as lru.ts and the tests as lru.test.js, then run node --test. The tests pin down the three rules above, which are exactly the ones a refactor tends to break.
import test from "node:test";
import assert from "node:assert/strict";
import { LRUCache } from "./lru.ts";
test("evicts the least recently used key", () => {
const lru = new LRUCache(2);
lru.set("a", 1).set("b", 2);
lru.get("a"); // a is now the newest
lru.set("c", 3); // so b is evicted
assert.equal(lru.has("b"), false);
assert.deepEqual([lru.get("a"), lru.get("c")], [1, 3]);
});
test("an overwrite refreshes the key", () => {
const lru = new LRUCache(2);
lru.set("a", 1).set("b", 2).set("a", 10).set("c", 3);
assert.equal(lru.size, 2);
assert.equal(lru.get("a"), 10);
assert.equal(lru.has("b"), false);
});
test("falsy values are hits", () => {
const lru = new LRUCache(2).set("zero", 0);
assert.equal(lru.get("zero"), 0);
});
From an entry count to a byte budget
Turning the count into a byte budget takes one more field per entry and a running total. Each entry stores its estimated size; set adds it, eviction subtracts it, and the loop walks from the oldest key until the total fits again. Deleting the entry you are currently visiting inside a for...of over a Map is safe in JavaScript.
set(key: K, value: V): this {
const size = this.sizeOf(value);
if (size > this.maxBytes) return this; // never cache something bigger than the budget
this.delete(key); // subtracts the old size if present
this.#map.set(key, { value, size });
this.#bytes += size;
for (const [k, e] of this.#map) { // oldest first
if (this.#bytes <= this.maxBytes) break;
this.#map.delete(k);
this.#bytes -= e.size;
}
return this;
}
For production you will usually reach for a tested package such as lru-cache or for the cache built into your data library. Writing your own is still worth it for small, hot paths such as memoizing a parser, and it is the version you will be asked to write on a whiteboard.
Check your understanding · Exercise 3
A teammate's LRU get looks like this: const v = map.get(key); if (v) { map.delete(key); map.set(key, v); } return v;. Which problems does it have? Select all that apply.
Cache by query key
A cache is only as good as its keys. The key has to answer one question: "would the server give the same answer to this request?". If two requests would get the same answer, they should share a key. If they would get different answers, they must not.
Get it wrong in one direction and you have a collision: two different requests share a key, so one of them shows the other's data. Get it wrong in the other direction and you have fragmentation: the same request gets a different key each time, so every read is a miss and the cache fills with copies.
What goes into a key
Put in everything that changes the response, and nothing that doesn't. For a list of tasks that usually means the resource, the parent it belongs to, and the filters, sort, and page.
["projects", 42] reaches every list and filter under the project in one call.- Resource and parent first:
["projects", 42, "tasks", ...]. Going from general to specific lets you invalidate a whole branch by prefix. - Every parameter the server reads: filters, sort, page or cursor, page size, search text, and locale if the server translates.
- Nothing the server ignores: a render counter, a callback, a
Date.now()timestamp, or a whole props object. Each makes a new key on every call. - Defaults filled in:
{ page: 1 }and{}usually mean the same request, so fill defaults before building the key. - Inputs normalized like the server does: if search ignores case and outer spaces, trim and lowercase first, so
"Shoes "and"shoes"share one entry.
Stable keys: the JSON.stringify trap
The quick way to turn parameters into a string key is JSON.stringify(params). It has one trap: it keeps object properties in insertion order. { status: "open", page: 2 } and { page: 2, status: "open" } become two different strings, so two components that build the same filter in a different order get two cache entries and two requests.
// Same output for objects with the same content, whatever the key order.
export function stableKey(value: unknown): string {
return JSON.stringify(value, (_k, v) =>
v && typeof v === "object" && !Array.isArray(v)
? Object.fromEntries(Object.entries(v).sort(([a], [b]) => (a < b ? -1 : 1)))
: v,
);
}
stableKey(["tasks", { status: "open", page: 2 }]);
stableKey(["tasks", { page: 2, status: "open" }]);
// both: '["tasks",{"page":2,"status":"open"}]'
A few more JSON.stringify quirks: properties whose value is undefined are dropped, so { q: undefined } and {} share a key (usually what you want). Date objects become ISO strings, and NaN and Infinity become null. A Map or a Set becomes {}, a silent collision, so keep keys to plain arrays, objects, strings, numbers, and booleans.
Hierarchical keys make invalidation cheap
Libraries such as TanStack Query and SWR accept arrays as keys. TanStack Query adds prefix matching: invalidateQueries({ queryKey: ["projects", 42] }) marks every entry whose key starts with those two items as stale, which covers the project and its task lists for every filter. The cache invalidation lesson goes deeper; you only get that power if you design the keys as a tree from the start.
| Key | What it caches | Invalidated by prefix |
|---|---|---|
["projects", 42] | Project 42's header data | ["projects"], ["projects", 42] |
["projects", 42, "tasks", { status: "open", page: 1 }] | First page of open tasks | the two above, plus ["projects", 42, "tasks"] |
["projects", 42, "tasks", { status: "done", page: 1 }] | First page of done tasks | same as the row above |
["me"] | The signed-in user | only ["me"] |
Who is asking is part of the answer
The server's answer to GET /api/me/tasks depends on who is signed in, but the URL doesn't say so. In a single-account app, clear the cache when the user changes (chapter 9). In an app with several accounts open at once, like a mail client with a switcher, put the account id first in every key, or one account's data will show in another's view.
Check your understanding · Exercise 4
Which of these query keys is a collision or a fragmentation bug? Select all that apply.
Per-query caches and normalized entity caches
There are two ways to lay out what you store under those keys. The difference decides how much memory you use, and whether an edit on one screen shows up on the others.
Per-query (document) caches
A per-query cache stores each response whole, under its query key. ["tasks", { status: "open" }] holds the array the server sent; ["task", 17] holds the detail object. TanStack Query, SWR, and RTK Query work this way by default. It is simple, it needs no knowledge of your data's shape, and evicting an entry frees exactly that response.
The catch is duplication. Task 17 appears in the open list, in the "assigned to me" list, in search results, and in its own detail entry: four copies of the same object. When the user renames it in the detail view, the three list copies still show the old name until each one is refetched or patched.
Normalized entity caches
A normalized cache splits every response into entities and stores each entity once, under a type and id such as Task:17. Lists and query results hold references ({ __ref: "Task:17" }) instead of copies. Apollo Client's InMemoryCache does this using __typename and id; Relay keeps a record store keyed by global ids. The normalization lesson covers the data side.
Now the rename writes one record, and every list that points at Task:17 renders the new name on the next read. With heavy overlap the cache is also smaller: a feed of 200 posts by 30 authors stores 30 author records rather than 200 copies.
What normalization costs
- Every entity needs a stable id and a type. Objects without one, such as a stats summary, are stored inside their parent and duplicated again.
- Lists are still per-query. The cache cannot know that a new task matches
tasks(status: open), so you add it by hand or refetch the list. - Reads cost more. Each read rebuilds the response by following references, so libraries memoize the results.
- Eviction is harder. You cannot free
Task:17while another list still points at it. Freeing memory becomes a reachability problem, like a small garbage collector: Apollo hascache.evictandcache.gc, and Relay frees records that no retained query needs.
Decision Should the app's server-data cache be per-query or normalized?
What we pick: For a REST app with modest overlap, a per-query cache with good keys and prefix invalidation is simpler and usually better. When the same entities show up in many views and change often (feeds, chat, project tools) and every object has an id, a normalized cache pays for its complexity. In an interview, the reasoning matters more than the choice.
Think first
A user scrolls an infinite feed for an hour and loads 3,000 posts. With a normalized cache, the feed query is then removed. Is the memory freed?
Not by itself. Removing the feed query drops only its list of references; the 3,000 post records and their authors stay in the entity store.
In Apollo you would evict the feed field and then call cache.gc(), which deletes every record no longer reachable from a root; records another query still uses stay. In a per-query cache, evicting the feed entry frees the posts, because they lived only inside it.
Memory limits and measuring size
A heap cache shares the tab's memory with your code, the DOM, decoded images, and the framework. How much a tab may use depends on the device, the browser, and what else is running, but a phone with 3 GB of RAM and twenty open tabs will not let your page grow forever.
What happens when memory runs short
- Everything slows down first. A bigger heap means more garbage-collection work: janky scrolling and slow input on low-end phones, long before any crash.
- Background tabs get discarded. Chrome's Memory Saver and mobile browsers drop inactive tabs under memory pressure, so the user comes back to a full reload and an empty cache.
- The tab crashes. A page that hits the engine's heap limit shows an out-of-memory error page, and mobile systems may end it even sooner.
So the budget is your choice. A reasonable starting point is the low tens of megabytes for server data; above that, move data to IndexedDB or stop caching it.
Where the bytes actually go
Most JSON data is smaller than people fear. As JSON text, a user record is about 1 KB, a page of 50 tasks about 25 KB, and a long rich-text document a few hundred KB. The parsed objects can take a few times more, but thousands of entries still fit in a modest budget.
A decoded image costs width times height times 4 bytes, whatever its file size: about 8.3 MB for a 1920 by 1080 frame and 48 MB for a 12-megapixel photo. Caching ImageBitmap objects or canvases by count is the fastest way to crash a mobile tab. Blob URLs are similar: URL.createObjectURL keeps the blob alive until URL.revokeObjectURL, even after you drop every other reference.
Estimating an entry's size
JavaScript has no sizeof: the engine decides how objects are laid out, and stores a string in one or two bytes per character depending on its content. So any size you compute is an estimate, which is fine: a budget needs a number that grows with the real cost, not an exact one.
- Use the text you already have.
text.lengthbeforeJSON.parseis a free estimate, usually within a small factor of the real heap cost. - For binary data, use exact sizes:
blob.size,buffer.byteLength, orwidth * height * 4for decoded pixels. - Don't call
JSON.stringifyon every write just to measure a large value: it costs about as much time as the parse you saved. - Calibrate once. With the cache full, take a heap snapshot in the Chrome DevTools Memory panel, compare the retained size of the cache's
Mapwith your estimate, and scale the estimate to match.
For field data, performance.measureUserAgentSpecificMemory() estimates the page's memory, but only in Chromium-based browsers and on cross-origin isolated pages (served with the Cross-Origin-Opener-Policy and Cross-Origin-Embedder-Policy headers). The older performance.memory is non-standard and too coarse for one cache.
WeakMap and WeakRef caches, and their limits
JavaScript has two tools that let the garbage collector help. A WeakMap keeps a value only as long as its key object is reachable from somewhere else. That suits caching facts about an object you don't own: when the object goes away, so does the cached value, with no eviction code.
// Derived data that lives exactly as long as its source object.
const sortedCache = new WeakMap<Task[], Task[]>();
export function sortByDue(tasks: Task[]): Task[] {
let sorted = sortedCache.get(tasks);
if (!sorted) {
sorted = [...tasks].sort((a, b) => a.due - b.due);
sortedCache.set(tasks, sorted);
}
return sorted;
}
Its keys must be objects (or symbols not created with Symbol.for), so you cannot key a WeakMap by a URL or an id string, which is what a data cache needs. It has no size and cannot be iterated, so you cannot measure, list, or clear it except by replacing it.
A WeakRef goes the other way: it holds a value weakly, and deref() returns undefined once the value was collected. A Map of string keys to WeakRefs looks like a cache that frees itself, but the engine collects whenever it likes, so values may vanish at once or never. MDN advises avoiding WeakRef where you can; use it, if at all, behind a real size limit.
Check your understanding · Exercise 5
A gallery caches decoded thumbnails as ImageBitmaps in a Map keyed by photo id, with a limit of 500 entries. Each bitmap is 400 by 300 pixels. Which statements are true? Select all that apply.
When to clear, and what goes wrong
Eviction is about space. Clearing is about correctness: there are moments when cached data is no longer safe to show, however much room is left. Most of the serious bugs with heap caches come from missing one of these moments.
The life of an entry
An entry is a small state machine. Created by a fetch, it is fresh for a while, turns stale when its freshness time runs out, becomes inactive when the last component using it unmounts, and is removed by eviction, by its idle timer, or by an explicit clear.
Stale entries are still shown while a refetch runs; inactive entries wait in case someone comes back. Clearing jumps straight to removed from any state, including from fetching, which is the case people forget.
The moments that must clear the cache
- Logout. Everything cached belongs to the user who just left. On a shared computer, the next person must not press Back and see the previous inbox.
- User or account switch. Same as logout. In multi-account apps, clear the old account's keys, or keep the account id in every key.
- Tenant, workspace, or role change. Entries fetched under old permissions may show things the user can no longer see.
- App version change. New code may expect a different shape. Heap caches die with the old page, but persisted caches do not: stamp them with a version (TanStack Query's persister calls it
buster) and discard on mismatch. - Locale or currency change. If the server formats or translates, cached strings are now wrong. Put the locale in the key or clear on change.
Logout, in the right order
Clearing is not one call. A request started before logout can resolve after the clear and write the old user's data back into the fresh cache. Do the steps in this order:
- Cancel in-flight requests (
queryClient.cancelQueries(), or abort your ownAbortControllers), so no late response refills the cache. - Clear the heap cache:
queryClient.clear()in TanStack Query,client.clearStore()in Apollo (notresetStore(), which also refetches every active query), ornew Map()for your own. - Delete persisted copies: IndexedDB stores, service worker caches with user data, and any snapshot used for warm starts.
- Tell the other tabs: post a
logoutmessage on aBroadcastChannelso they clear too. - Navigate to the sign-in page. A full page load (
location.assign) is the bluntest guarantee: it throws away the whole heap.
On the server side, a logout response can also send Clear-Site-Data: "cache", "storage" to ask the browser to wipe the HTTP cache and site storage for the origin. Browser support varies, so treat it as a second line, not the only one.
Failure modes in production
| Symptom | Likely cause | Fix |
|---|---|---|
| Memory climbs on every route change | No limit and no idle timer | Add an LRU limit or gcTime; confirm with two heap snapshots |
| Tab crashes on phones only | Images or buffers cached by count | Byte budget; close() bitmaps; revoke blob URLs |
| Previous user's data after login | No clear on logout, or a late response refilled it | Cancel, then clear, then reload |
| Same request fires twice on one screen | Unstable keys (key order, timestamps) | Stable key function; fill defaults |
| Edit shows on one screen but not another | Per-query copies not invalidated | Prefix invalidation, or normalize that entity |
| Other tab shows old data | One heap per tab | BroadcastChannel invalidation |
Check your understanding · Exercise 7
A user logs out, and the app calls queryClient.clear() and then shows the sign-in form without reloading. A tester then signs in as a second user and briefly sees the first user's notifications. What could explain it? Select all that apply.
In real products and in the interview
You will rarely hand-write the main data cache of a production app. You will pick a library, and its defaults decide your memory use and your stale-data bugs.
What the common libraries do by default
- TanStack Query keeps one entry per query key. Data is stale at once (
staleTime: 0), so it refetches in the background on mount and window focus. An entry with no observers is removed aftergcTime, 5 minutes by default. - SWR keeps entries in a
Mapfor the life of the page by default, with no eviction, and revalidates on mount and focus. Theprovideroption swaps in your own cache, such as an LRU. - RTK Query keeps one entry per endpoint and argument, and removes unused data after
keepUnusedDataFor, 60 seconds by default. Tags connect mutations to the queries they invalidate. - Apollo Client normalizes into
InMemoryCacheby__typenameandid. The entity store has no size limit of its own: you remove data withcache.evictandcache.gc. - Relay normalizes into a record store and garbage-collects records that no retained query needs, keeping the last few released queries (10 by default) for reuse.
Outside data libraries, the pattern is everywhere. Client routers keep the data of recently visited routes so Back is instant, Markdown and syntax-highlighting tools memoize parsed results by input, and image-heavy apps keep a small window of decoded images around the viewport. In every case the three questions from chapter 1 have clear answers: the key, the limit, and what clears it.
When to bring it up in an interview
Mention the in-memory cache whenever the design has navigation between a list and a detail view, the same data in several places, or a requirement such as "Back should be instant". Name it when you draw the client architecture, as a box between your components and the network layer.
- Key: "Keyed by resource and parameters with a stable key function, as a tree so one prefix invalidates a project."
- Layout: "Per-query by default; I'd normalize users and messages because they appear in many views and change often."
- Limit: "Inactive entries leave after five minutes; images have their own byte budget of about 30 MB with LRU eviction."
- Freshness: "Show cached data at once and revalidate in the background; mutations invalidate the affected prefixes."
- Clearing: "On logout I cancel in-flight requests, clear the cache and persisted copies, broadcast to other tabs, and reload."
How deep to go by level
| Level | What a strong answer shows |
|---|---|
| Junior | Knows a library caches by key; explains why Back is instant; knows to clear on logout |
| Mid | Designs stable, hierarchical keys; picks per-query or normalized with a reason; sets limits and stale times |
| Senior | Budgets memory by bytes for heavy data; handles SSR per-request caches, multi-tab consistency, the logout race, and version changes; knows how to prove there is no leak |
Check your understanding · Exercise 8
You are designing a team chat app. Which cache decisions are sound? Select all that apply.
Summary
- A heap cache holds parsed, ready-to-render objects: it is the only cache layer that gives synchronous reads, which is what makes Back instant.
- Every cache needs three answers: the key, the limit, and what clears it. Without a limit, a
Mapcache is a memory leak. - LRU is the right default for UI data; a
Mapgives you an O(1) LRU in about 25 lines. Bound by bytes when entry sizes vary, especially for images. - Keys must include every input that changes the answer and nothing else. Use a stable serializer and design keys as a tree for prefix invalidation.
- Per-query caches are simpler; normalized caches keep shared, often-edited entities consistent at the cost of ids, manual list updates, and harder garbage collection.
- Each tab has its own heap: broadcast invalidations across tabs, and create caches per request on the server.
- Clear on logout, user switch, permission change, and version change, cancelling in-flight requests first so nothing refills the cache.
In the interview
Questions interviewers ask about this. Answer each out loud first, then compare.
Question 1
Why keep server data in a JavaScript cache when the browser already has an HTTP cache?
The HTTP cache stores bytes, and many API responses are not cacheable there at all. Even on a hit, the app has to read asynchronously, parse the JSON, and rebuild derived data, so the first render after mount has no data.
A heap cache holds parsed, ready-to-render objects that a component can read synchronously. That gives instant Back navigation, one shared copy for many components, and stale-while-revalidate in the UI. The cost is memory and the duty to clear it at the right moments.
Question 2
Implement an LRU cache in JavaScript. What are the edge cases?
Use a Map: its iteration order is insertion order. On get, if has(key), delete and re-set the key to move it to the newest end. On set, delete then set, and if the size is over the limit, delete map.keys().next().value, the oldest key. Every operation is O(1).
Edge cases: falsy values must still count as hits, so test with has rather than truthiness; overwriting a key should refresh it; peeking should not. For memory, switch the count to a byte budget with a running total, and refuse values larger than the whole budget.
Question 3
How do you choose query keys, and what goes wrong with bad ones?
A key must contain every input that changes the server's answer and nothing else: resource, parent ids, filters, sort, page, and locale if relevant. Fill defaults and normalize inputs the way the server does. Build it as a tree from general to specific so a prefix can invalidate a whole branch.
Missing inputs cause collisions, where one request shows another's data. Unstable parts (timestamps, object key order with plain JSON.stringify, new objects each render) cause fragmentation, where every read misses. Use a stable serializer that sorts object keys.
Question 4
Per-query cache or normalized cache: how do you decide?
Per-query stores each response whole. It is simple and works with any API, but entities are duplicated and an edit has to invalidate every list it appears in. Normalized stores each entity once by type and id, so an edit shows everywhere and overlap costs less memory, but it needs ids everywhere, lists still need manual updates, and freeing memory becomes a reachability problem.
I default to per-query with good keys for REST apps with modest overlap, and normalize when the same entities appear in many views and change often, like chat or feeds.
Question 5
How do you keep an in-memory cache from becoming a memory leak?
Give it a limit: an idle timer for entries nobody is watching, plus an LRU bound by count or, for heavy data such as images, by bytes. Estimate sizes cheaply from the response text or binary sizes, and calibrate once with a heap snapshot.
Then check it: take heap snapshots before and after navigating around, and make sure the cache's retained size falls back. Watch for retainers outside the cache, such as listeners or blob URLs, and don't rely on WeakRef for the limit because collection timing is not guaranteed.
Question 6
What happens to the cache when the user logs out, and when they have two tabs open?
On logout: cancel in-flight requests so a late response can't refill the cache, clear the heap cache and any persisted copies, tell other tabs over a BroadcastChannel, and do a full navigation to the sign-in page.
With two tabs, each has its own heap and its own cache. To keep them consistent, broadcast invalidation events after mutations so other tabs refetch what they show; share data itself only for tiny hot values, or move the cache into a SharedWorker if the data is heavy.
Practice it
These design problems lean on in-memory caches.
References
- Map Insertion-ordered iteration, which is what makes the Map-based LRU work.
- WeakRef Why collection timing is not guaranteed, and the advice to avoid WeakRef where possible.
- BroadcastChannel Messaging between same-origin tabs, windows, and workers.
- Performance: measureUserAgentSpecificMemory() Field memory measurement and its cross-origin isolation requirement.
- TanStack Query: Caching examples staleTime, gcTime, and the life of a cache entry.
- SWR: Cache The default global Map cache and custom cache providers.
- Relay: Presence of data Query retention, garbage collection, and the release buffer.
- Apollo Client: Caching overview How InMemoryCache normalizes responses by type and id.
- Chrome DevTools: Fix memory problems Heap snapshots and finding what keeps a cache alive.
Read next
- Server-state librariesTanStack Query, SWR, RTK Query, and Apollo cache server data by key, refetch stale entries in the background, and refresh them after mutations, so components stop hand-writing loading flags, races, and invalidation.
- Normalizing stateStoring each entity once, in a table keyed by its id, and having lists and relationships point at ids instead of holding copies, so one change updates every view that shows it.
- Cache invalidationHow every copy of a piece of data, from the server cache to the CDN to the open tab, learns that it is out of date: by time, by event, or by a new name, in the right order and without a stale read sneaking back in.
- Deduplication and batchingWhen many parts of a page ask for the same data at once, send one request and share the answer; when they ask for many small pieces at once, collect them into one request. Two small ideas that remove most of the wasted traffic in a component-based app.
- Client storage optionsThe six places a web app can keep data on the user's device (cookies, localStorage, sessionStorage, IndexedDB, the Cache API, and OPFS), how big, how fast, and how long-lived each one is, and how to pick the right one for each kind of data.
- HTTP cachingThe browser's built-in cache, steered by headers on each response, which lets a repeat visit skip the network entirely or ask only "has this changed?": the cheapest speed-up on the web.