Inline caches

Remember where a property is

When you write obj.x, the JavaScript engine has to find x. The engine is the program inside the browser or Node.js that runs your code. By the rules of the language, finding x is a search:

• First the engine looks at the object's own properties.
• If x is not there, it looks at the object's prototype. A prototype is another object that this object borrows properties from.
• It keeps going up this line of prototypes, called the prototype chain, until it finds x or runs out of objects.

This search is why obj.x works even when x lives on a prototype. But a search takes time, and a busy program may read the same property millions of times.

The word "inline" comes from history. Early engines wrote a tiny shortcut right inside the code, next to the property read. Modern engines keep this memory in other places, but the idea is the same: each place in your code learns from the objects it sees.

1const record = { user: { name: "Ada" } };2const proto = { role: "author" };3Object.setPrototypeOf(record.user, proto);4 5function naiveLookup(object, name) {6  let current = object;7  while (current) {8    if (Object.prototype.hasOwnProperty.call(current, name)) {9      return current[name];10    }11    current = Object.getPrototypeOf(current);12  }13  return undefined;14}15 16console.log(naiveLookup(record.user, "name"));17console.log(naiveLookup(record.user, "role"));

This function copies the language's search rule into plain JavaScript:

• Line 6 starts at the object itself.
• Line 7 keeps going while there is an object to look at.
• Line 8 asks: does this object have the property as its own property?
• Line 9 returns the value when it does.
• Line 11 moves one step up, to the prototype.

So naiveLookup(record.user, "name") finds "Ada" on the object itself and prints Ada. The "role" search finds nothing on the object, moves to the prototype, and prints author. Doing this full search on every busy read would be too slow. So after the first search, the engine remembers where the property was.

This lesson builds on type feedback. It helps to know object basics, the prototype chain, classes, and property descriptors. The next section explains layouts in just enough detail for this lesson. The later Hidden classes & object shapes lesson goes deeper.

Shapes: a layout shared by similar objects

Engines keep a hidden record for each object layout. The record lists the property names in order and says where each value is kept. Each place is a numbered slot. The slot number is also called the offset.

Different engines give this record different names:

• V8 (Chrome and Node.js) calls it a map.
• SpiderMonkey (Firefox) calls it a shape.
• Other texts call it a hidden class.

This lesson says shape. Objects with the same property names, added in the same order, share one shape. If the shape says x is in slot 0, the engine can check the shape and then read slot 0. No search is needed.

1function makeUser(name) {2  return { name, score: 0, active: true };3}4 5const ada = makeUser("Ada");6const lin = makeUser("Lin");7const reordered = { score: 0, name: "Grace", active: true };8 9console.log(Object.keys(ada).join(","));10console.log(Object.keys(lin).join(","));11console.log(Object.keys(reordered).join(","));

Line 2 always adds name, then score, then active. So ada and lin have the same layout. Line 7 writes the same three names in a different order: score first. Object.keys lists an object's property names in order, so the last line prints score,name,active. That different order is a clue that reordered has a different shape.

Normal JavaScript can't see shapes. The code above only prints the order of the names. To see real shapes, the next example uses Node.js 22, which runs V8 version 12.4. It turns on a test-only feature with the flag --allow-natives-syntax. That flag adds helpers that start with %, such as %HaveSameMap(a, b). It answers: do a and b have the same map? This code can't run in the browser editor, so its Run button is turned off.

1function Point(x) {2  this.x = x;3}4 5const a = { x: 1, y: 2 };6const b = { x: 3, y: 4 };7const c = { y: 2, x: 1 };8const p = new Point(1);9const q = new Point(2);10 11console.log(`Node ${process.versions.node} (V8 ${process.versions.v8.split(".").slice(0, 2).join(".")})`);12console.log("same property order", %HaveSameMap(a, b));13console.log("different property order", %HaveSameMap(a, c));14console.log("constructor start", %HaveSameMap(p, q));15p.y = 2;16console.log("after one transition", %HaveSameMap(p, q));17q.y = 3;18console.log("after both transition", %HaveSameMap(p, q));

Want to read more? The V8 team explains how it stores properties in Fast properties in V8. The V8 maps docs show that the map is the first thing stored in every object. Mathias Bynens and Benedikt Meurer's Shapes and Inline Caches article gives a friendly overview.

One shape: one check and one read

An inline cache belongs to one place in your code, not to a property name. If your program has two different obj.x reads, each one has its own cache. Engines call such a place a site.

At a site, the first object finds the cache empty. That is a miss. The engine does the slow search, then saves a small, ready-made shortcut for that shape and slot. Engines call this shortcut a stub. A later object that matches the saved shape is a hit.

1function createGetXSite() {2  let cachedShape = null;3  let cachedOffset = "x";4  return function getX(record) {5    if (record.shape === cachedShape) {6      return record.slots[cachedOffset];7    }8    cachedShape = record.shape;9    return record.slots.x;10  };11}12 13const getX = createGetXSite();Run14const a = { shape: "x,y", slots: { x: 1, y: 10 } };15const b = { shape: "x,y", slots: { x: 2, y: 20 } };16console.log(getX(a));17console.log(getX(b));

The replay is a simple model written in JavaScript. Each object carries a shape string, so you can watch the check happen. Step through it:

• The first getX(a) call is a miss. It saves the shape x,y and returns 1.
• The second call, getX(b), gets a different object with the same shape. The check on line 5 passes.
• Line 6 reads the value straight away and returns 2.

A cache that has seen only one shape is called monomorphic ("mono" means one, "morph" means form). This is the best case. The engine needs only two steps: check the shape, then read the slot.

A few shapes, then too many

Real programs often send two or three different layouts through one helper function. Then the site has a short list of saved shapes, and it checks them one by one. A site like this is polymorphic ("poly" means many).

Each site keeps only a few shapes. When too many different shapes arrive, the site stops saving them. It is now megamorphic ("mega" means huge). From then on it asks a slower table shared by the whole engine. Engines call that table the stub cache.

This lesson's model lets a site keep 4 shapes, and the 5th shape makes it megamorphic. Real engines choose their own number. It can change between engines, kinds of cache, and versions. No number is part of the JavaScript language.

1const limit = 4;Run2const site = { state: "uninitialized", entries: [], stubCache: new Map() };3 4function readX(record) {5  const key = record.shape + ":x";6  if (site.state === "megamorphic" && site.stubCache.has(key)) {7    return site.stubCache.get(key)(record);8  }9  if (site.entries.includes(record.shape)) {10    return record.slots.x;11  }12  if (site.entries.length === limit) {13    site.state = "megamorphic";14    site.stubCache.set(key, (object) => object.slots.x);15    return record.slots.x;16  }17  site.entries.push(record.shape);18  site.state = site.entries.length === 1 ? "monomorphic" : "polymorphic";19  return record.slots.x;20}21 22const records = [23  { shape: "S1", slots: { x: 1 } },24  { shape: "S2", slots: { x: 2 } },25  { shape: "S3", slots: { x: 3 } },26  { shape: "S4", slots: { x: 4 } },27  { shape: "S5", slots: { x: 5 } },28];29 30for (const record of records) console.log(readX(record));31console.log(readX(records[4]));

1const site = createInlineCacheSite("getX");2const objects = [3  makeObject([["x", 1], ["a", 10]], "x-a"),4  makeObject([["a", 20], ["x", 2]], "a-x"),5  makeObject([["x", 3], ["b", 30]], "x-b"),6  makeObject([["b", 40], ["x", 4]], "b-x"),7  makeObject([["c", 50], ["x", 5]], "c-x"),8];9 10for (const object of objects) {11  console.log(site.read(object, "x"));12}

The table below lists the four states a site can be in. Read each row as: the site is in this state, it has seen this, and a read costs this much.

In V8, a megamorphic property read looks in the shared stub cache. It finds the entry using the object's map and the property name. This stops one busy site from saving more and more shortcuts forever. But it is slower than one quick shape check.

Caches for keys, writes, calls, and global names

Reading obj.x is the easiest case to picture. But engines keep caches for many other operations too.

A keyed cache handles reads with square brackets, like obj[key]. The key can be a property name or an array index, so the cache has to tell them apart. Arrays add more cases:

• A packed array has a value at every index.
• A holey array has gaps, called holes, where no value was ever set.
• An array can hold only small whole numbers, only decimal numbers, or any values. The engine tracks which kind it is.

1function readKey(target, key) {2  return target[key];3}4 5const user = { name: "Ada" };6const packed = ["zero", "one", "two"];7const holey = [];8holey[2] = "two";9 10console.log(readKey(user, "name"));11console.log(readKey(packed, 1));12console.log(0 in holey);13console.log(readKey(holey, 2));

Line 1 reads with target[key]. Line 10 passes it a property name, and line 11 passes an index into a packed array. Line 8 sets only index 2 of holey, so indexes 0 and 1 are holes. The in operator asks whether a property exists, so line 12 prints false.

A store cache handles writes, like obj.x = value. It learns one of two things:

• The write changes a slot that already exists.
• The write adds a new property, so the object moves to a new shape. Engines call this move a transition.

1function writeX(object, value) {2  object.x = value;3  return object.x;4}5 6const a = { x: 0, y: 0 };7const b = { x: 1, y: 1 };8const c = { y: 0 };9 10console.log(writeX(a, 7));11console.log(writeX(b, 8));12console.log(Object.keys(c).join(","));13console.log(writeX(c, 9));14console.log(Object.keys(c).join(","));

Line 2 is the write. For a and b, x already exists, so the write only changes a slot. Line 12 shows that c has only y. Line 13 adds x to c, which is a transition. Line 14 then prints y,x.

A call cache remembers which function a call ran. If one call site always runs the same function, a faster compiler can later copy that function's body right into the caller. This is called inlining.

1function render(formatter, value) {2  return formatter(value);3}4 5function money(amount) {6  return "$" + amount.toFixed(2);7}8 9function label(value) {10  return "item:" + value;11}12 13console.log(render(money, 3));14console.log(render(money, 4));15console.log(render(label, "A"));

Line 2 calls whatever function arrives as formatter. Lines 13 and 14 send money twice. Line 15 sends a different function, label. Now the call cache on line 2 has seen two functions.

Global caches do the same job for global names such as Math or document. Reads that go through a prototype need one more thing. The cache must remember the object's shape and check that the prototypes have not changed since. V8 keeps a small flag for this, called a validity cell. The later hidden classes and prototypes inside the engine lessons go deeper.

1const proto = { x: 1 };2const child = Object.create(proto);3 4console.log(child.x);5proto.x = 2;6console.log(child.x);7child.x = 3;8console.log(child.x);9console.log(proto.x);

Line 2 makes child with proto as its prototype. Line 4 prints 1, found on the prototype. Line 5 changes the prototype, so the same child.x on line 6 now prints 2. Line 7 gives child its own x, so line 8 prints 3. A saved shortcut must notice each change. That is why the engine checks before it trusts the cache.

The table below sums up the kinds of cache. Read each row as: this kind of cache, for code like this, remembers this.

How Firefox writes a cache: CacheIR

Firefox's engine, SpiderMonkey, writes each cache shortcut as a short list of simple steps. This list format is called CacheIR ("IR" means a simple in-between language that the engine turns into machine code). The list has two kinds of steps:

• Guards check that something is still true. If a check fails, the shortcut is skipped.
• One final result step produces the value.

The official docs show a simple obj.x read. It checks that the input is an object, checks its shape, and then reads a fixed slot.

1GetProp "x"2  GuardToObject input3  GuardShape shape=S14  LoadFixedSlotResult slot=0

In plain words, line by line:

1. Line 1 names the operation: read the property "x".
2. Line 2 is a guard: the input must be an object.
3. Line 3 is a guard: the object must have shape S1.
4. Line 4 is the result: read slot 0 of the object.

Jan de Mooij's CacheIR post explains why Firefox moved to this format. The SpiderMonkey docs say two of Firefox's compilers, Baseline and Warp, both build their code from these steps. Safari's engine, JavaScriptCore, uses inline caches too. WebKit's Speculation in JavaScriptCore explains how it guesses from what the code did so far, and checks each guess.

How faster compilers use what the cache learned

Engines run code in stages, called tiers. Early tiers start quickly and fill the caches as the code runs. When a function runs many times, an optimizing compiler turns it into fast machine code. It reads what the caches learned to decide what to build.

For a monomorphic read, the machine code can do just this:

1. Check that the object has shape S.
2. If not, leave the fast code and go back to the slower code. Engines call this deoptimizing.
3. If yes, read the value at slot k in memory.

That is how one property read can become one shape check and one memory read.

A polymorphic site can still get fast code, with a short list of checks. A megamorphic site is harder. It says "this read sees many different shapes," so the compiler has less to plan around. It often keeps a call to the slower shared lookup. Call caches work the same way. One known function can be inlined. Many different functions make inlining less worth it.

Write objects with steady shapes

The advice for real code is simple:

• Create every property you expect, right away.
• Create them in the same order each time.
• Avoid delete on objects that busy code uses again and again.
• Don't send many unrelated kinds of objects through one small, busy helper.

Don't twist clear code out of shape just to please the cache.

1function User(name, admin = false) {2  this.name = name;3  this.admin = admin;4  this.score = 0;5}6 7const ada = new User("Ada");8const lin = new User("Lin", true);9lin.score = 10;10 11console.log(Object.keys(ada).join(","));12console.log(Object.keys(lin).join(","));13delete ada.admin;14console.log(Object.keys(ada).join(","));

Lines 2 to 4 create the same three properties, in the same order, for every User. So ada and lin share one layout. Line 13 shows the habit to avoid in busy code: delete removes a property, which changes the layout. V8 may even switch the object to a slower way of storing properties. Sometimes delete is still the clearest choice. Just don't expect it to be free inside a busy loop.

• Build objects with the same properties in the same order.
• To clear a property on a busy object, set it to null or undefined instead of using delete.
• If one busy helper gets many unrelated kinds of objects, and a profile says it matters, split it into smaller helpers.
• Keep the code easy to read first. Measure before and after any change.

Common misconceptions

• "The cache saves values." It saves where to find the value, behind a shape check. The value can change.
• "A polymorphic site is always bad." A few steady shapes are normal, and engines often handle them well.
• "Megamorphic means broken." It means the engine switched to a slower shared lookup for a site with many shapes. It only matters if a profile shows that site is busy and slow.
• "Shapes are part of JavaScript." They are a trick inside engines. The language only says a property is found on the object or its prototypes.
• "Engine tricks should decide how I write all my code." Clear data, tests, and real measurements matter more than rumors about speed.

Some of these words sound alike. The table below sets them side by side. Read each row as: this idea means this, and it is easy to mix up with that.

Practice exercises

function getX(object) {
  return object.x;
}
const a = { x: 1, y: 10 };
const b = { x: 2, y: 20 };
console.log(getX(a));
console.log(getX(b));

const holey = [];
holey[2] = "two";
console.log(0 in holey);

Check your understanding

Key takeaways

• An inline cache belongs to one place in your code. One obj.x can be monomorphic while another is megamorphic.
• A shape lists property names and their slots. So a cached read becomes one shape check and one slot read.
• This lesson's model keeps 4 shapes per site. More than that sends the site to a slower shared table.
• Engines also cache keyed reads, writes, calls, global names, and reads through prototypes. Faster compilers use what they learn.
• Give similar objects the same properties in the same order, avoid delete in busy code, and measure before you optimize.

In one line: the engine remembers where a property was, and checks the shape before it trusts that memory.

Next in this module, Baseline compilers shows simple compilers that start fast and skip some interpreter work, before the strongest optimizing compilers take over.