The heap & pointer compression

The heap is organized space

In the lessons on stack and heap and memory management, you learned a simple picture: objects live in a big area of memory called the heap. This lesson looks inside the heap of V8, the JavaScript engine in Chrome and Node.js. An engine is the program that runs your JavaScript.

V8 does not keep the heap as one big pile. It splits it into areas and pieces. This lesson shows how, and why.

Two topics are left to other lessons. How V8 stores small numbers inside values is in the value-representation lesson. How V8 finds and frees unused objects (garbage collection) is in the garbage-collection lessons, starting with gc-basics. This lesson asks only two questions:

• Where does the engine put each object?
• Why does V8 sometimes store an address in half the usual space?

This lesson comes after Prototypes inside the engine. The next lesson, Stack frames & calling conventions, moves from the heap to the stack that function calls use.

New, old, code, and large-object spaces

A space is one area of V8's heap with one job. These are the main ones:

• New space holds new, small objects. Most objects start here.
• Old space holds objects that stayed in use for a while. V8 moves them here from new space.
• Code space holds machine code: instructions the computer can run directly. V8 makes this code from your JavaScript to run it faster. It needs different safety settings, so it is kept apart.
• Large-object space holds objects that are too big for the normal pieces of a space.

You can list the spaces yourself in Node.js. The built-in node:v8 module reports facts about V8. Its getHeapSpaceStatistics() function returns one entry per space, and each entry has a space_name. This code only runs in Node.js, not in a browser page.

1const v8 = require("node:v8");2const names = v8.getHeapSpaceStatistics().map((space) => space.space_name).sort();3console.log(names.join(", "));

Each row of this table is one space. Read across to see what it holds and why V8 keeps it apart.

The list changes as V8 changes. This Node.js version also lists read-only, trusted, and shared spaces. These names belong to V8, not to the JavaScript language. The language rules never mention spaces.

Pages and bump allocation

A page is a fixed-size piece of a space. In current V8, a regular page is 256 KB (262,144 bytes). The biggest object that fits the normal way is half a page: 128 KB.

These numbers come from V8's own source code, which is written in C++. Each line below sets one number:

• Line 2 sets kPageSizeBits to 18.
• Line 3 makes the page size 1 << 18. The << sign shifts the bits of 1 to the left 18 times, which is 2 multiplied by itself 18 times: 262,144 bytes.
• Line 5 makes the biggest regular object 1 << 17 bytes, which is half a page.

1// V8 source, src/base/build_config.h:2// constexpr int kPageSizeBits = 18;3// constexpr int kRegularPageSize = 1 << kPageSizeBits;4// V8 source, src/common/globals.h:5// constexpr int kMaxRegularHeapObjectSize = (1 << (kPageSizeBits - 1));

To place a new object on a page quickly, V8 often uses bump allocation. To allocate means to give an object a place in memory. Bump allocation works like this:

1. Keep a number, here called nextFree, that marks the first free byte on the page.
2. Check that the new object fits between nextFree and the end of the page.
3. If it fits, place the object at nextFree, then move nextFree forward by the object's size.
4. If it does not fit, move to a new page, or take a slower path.

There is no searching for a gap. That is why it is fast.

The step-through below runs a small model of a bump allocator. It uses tiny 1024-byte pages so you can see a page fill up. It also rounds each size up to a multiple of 16 bytes (line 6), because allocators keep objects lined up on even boundaries. This is called alignment.

1const pageSize = 1024;Run2let page = 0;3let nextFree = 0;4 5function align16(bytes) {6  return Math.ceil(bytes / 16) * 16;7}8 9function allocate(label, requested) {10  const size = align16(requested);11  if (nextFree + size > pageSize) {12    page += 1;13    nextFree = 0;14  }15  const address = { page, start: nextFree, end: nextFree + size };16  nextFree += size;17  return label + ": page " + address.page + ", bytes " + address.start + "-" + address.end;18}19 20console.log(allocate("header", 120));21console.log(allocate("items", 380));22console.log(allocate("names", 460));23console.log(allocate("avatar", 180));

Line 16 is the fast path: nextFree += size moves the marker forward. Line 12 runs only when the next object would go past the end of the page. This is a model to learn from. It does not look at your browser's real heap.

Each row of this table is one word from this section. Read across to see what it means and why it matters.

Why very large objects get their own space

Some objects are bigger than 128 KB. For example, a big array needs one long stretch of memory to hold its items. That stretch is too big for a regular page. So V8 gives it a place of its own in large-object space.

Being in large-object space says nothing bad about the object. It is not a leak (memory your app forgot to let go). It is just too big for the normal path.

This Node.js code checks that a big array really lands in large-object space. It uses two new tools:

• global.gc() asks V8 to clean up unused objects right now. It exists only when Node.js starts with the --expose-gc option.
• space_used_size is how many bytes a space is using.

In plain words, line by line:

1. Line 7 saves how much large-object space is in use before.
2. Line 8 makes an array of 20,000 items and keeps it, so it is not cleaned up.
3. Line 10 reads the size again after.
4. Line 11 prints whether the space grew. Line 12 prints whether it grew by at least 131,072 bytes (128 KB).

1const v8 = require("node:v8");2if (typeof global.gc !== "function") throw new Error("run with --expose-gc");3function used(spaceName) {4  return v8.getHeapSpaceStatistics().find((space) => space.space_name === spaceName)?.space_used_size ?? 0;5}6global.gc();7const before = used("large_object_space");8global.big = new Array(20000).fill(1);9global.gc();10const after = used("large_object_space");11console.log(after > before);12console.log(after - before >= 131072);

Pointer compression and the 4 GB cage

Objects point at each other. For example, an array holds the addresses of its items. An address (also called a pointer) is a number that says where something is in memory. On a 64-bit computer, a full address takes 8 bytes.

V8 in Chrome saves memory with pointer compression. It works like this:

1. V8 reserves one block of memory that is 4 GB big, and puts all its heap objects inside it. This block is called the cage. Its first address is the cage base.
2. For each address inside the cage, V8 stores only the distance from the cage base. This distance is called the offset.
3. The offset is never more than 4 GB, so it fits in 4 bytes (32 bits) instead of 8.
4. When V8 needs the full address, it adds the cage base back to the offset.

The model code below works with addresses as numbers. A few tools appear in it:

• Numbers that start with 0x are written in hexadecimal (base 16), the usual way to write addresses.
• The n at the end, as in 0x0000000400000000n, makes a BigInt: a number type that holds whole numbers of any size exactly.
• toString(16) writes a number in hexadecimal, and padStart(8, "0") adds zeros in front until it has 8 digits.
• offset >>> 0 turns the offset into a whole number from 0 up to 4 GB minus 1.

1const cageBase = 0x0000000400000000n;Run2const cageSize = 0x0000000100000000n;3 4function compressPointer(address) {5  if (address < cageBase || address >= cageBase + cageSize) {6    throw new RangeError("address is outside the 4 GB cage");7  }8  return Number(address - cageBase);9}10 11function decompressPointer(offset) {12  return cageBase + BigInt(offset >>> 0);13}14 15const address = cageBase + 0x12345678n;16const compressed = compressPointer(address);17console.log("0x" + compressed.toString(16).padStart(8, "0"));18console.log("0x" + decompressPointer(compressed).toString(16));

1const cageBase = 0x0000000400000000n;2const cageSize = 0x0000000100000000n;3 4function compressPointer(address) {5  if (address < cageBase || address >= cageBase + cageSize) {6    throw new RangeError("outside the cage");7  }8  return Number(address - cageBase);9}

1const cageBase = 0x0000000400000000n;2const cageSize = 0x0000000100000000n;3 4function compressPointer(address) {5  if (address < cageBase || address >= cageBase + cageSize) {6    throw new RangeError("address is outside the 4 GB cage");7  }8  return Number(address - cageBase);9}10 11function decompressPointer(offset) {12  return cageBase + BigInt(offset >>> 0);13}14 15const address = cageBase + 0x12345678n;16const compressed = compressPointer(address);17console.log("0x" + compressed.toString(16).padStart(8, "0"));18console.log("0x" + decompressPointer(compressed).toString(16));

In plain words, the key lines:

1. Lines 1 and 2 set the cage base and the cage size, 4 GB.
2. Line 5 is the safety check: an address outside the cage cannot be stored as an offset, so the code throws an error.
3. Line 8 returns the offset: the address minus the cage base.
4. Line 12 turns an offset back into a full address by adding the cage base.
5. Lines 17 and 18 print 0x12345678 (the offset) and 0x412345678 (the full address again).

Each row of this table is one idea from this section. Read across to see what it stores and what to watch for.

Pointer compression also changes how small whole numbers are stored. V8 keeps a small whole number right inside the value, with no separate object on the heap. V8 calls this a Smi ("small integer"). The value-representation lesson explains Smis fully.

The code below asks V8 whether a number is a Smi. %IsSmi is a special V8 function. It works only when Node.js starts with the --allow-natives-syntax option, so this code does not run in a browser.

1console.log(%IsSmi(2 ** 31 - 1));2console.log(%IsSmi(2 ** 31));

The official Node.js build does not use pointer compression. In it, 2 ** 31 - 1 is a Smi (line 1 prints true), and 2 ** 31 is not (line 2 prints false). Chrome on 64-bit computers uses pointer compression, and there Smis have only 31 bits for the number, as the V8 team's article on pointer compression describes.

The V8 sandbox

Pointer compression saves memory. The V8 sandbox is about safety, and it is a different thing. A sandbox is a closed area that code cannot reach out of.

Programs have bugs. A bug in V8 could let an attacker change bytes in the heap. The V8 sandbox keeps the heap closed off, so such a bug cannot reach the rest of the browser's memory. Here is how, as the V8 team's article on the sandbox describes:

• Addresses inside the sandbox are stored as offsets, like in pointer compression.
• Addresses of memory outside the sandbox are not stored in the heap at all. The heap stores a number that points into a separate list, such as the external pointer table. V8 checks that list before it uses the address.

process.config.variables lists the options Node.js was built with. Line 1 prints the pointer compression option. Line 2 prints the sandbox option.

1console.log(process.config.variables.v8_enable_pointer_compression);2console.log(process.config.variables.v8_enable_sandbox);

The V8 sandbox does not replace the browser's own process sandbox, which keeps each site in a separate locked-down program. It also does not make every bug harmless. It is one more layer: an attacker who changes the heap still needs a second bug to break out and reach other memory.

Heap size limits

V8 has a top size for its heap. In Node.js, v8.getHeapStatistics().heap_size_limit reports it in bytes. You can raise or lower it with the --max-old-space-size option, which sets the biggest size of old space in megabytes (MB).

This limit covers only V8's heap. It is not the total memory your operating system shows for the program. It also does not tell you exactly when an app will run out of memory.

The two commands below start Node.js twice, once with a 64 MB old space and once with 256 MB. The -e option runs the code in quotes, which prints the heap limit.

1node --max-old-space-size=64 -e "console.log(require('node:v8').getHeapStatistics().heap_size_limit)"2node --max-old-space-size=256 -e "console.log(require('node:v8').getHeapStatistics().heap_size_limit)"

If a Node.js program runs out of heap, a bigger limit can give it more time. But it does not tell you why memory keeps growing. To find out, use a heap snapshot: a list of every object in the heap at one moment, which browser and Node.js developer tools can take. Also use an allocation profile, which shows which code makes the most objects.

What developers should do

Most web apps never need to think about V8 page sizes or build options. This knowledge helps you read memory measurements:

• A giant array, image data, or a big parsed JSON response may show up in large-object space.
• Many short-lived small objects show up in new space, and are cleaned up quickly.
• A cache that keeps growing shows up in old space.

Each of these is a clue about what to measure next. It is not a rule to follow without measuring.

• Take a heap snapshot to see which objects are still kept, then fix the code that keeps them.
• Measure before you rewrite your data to fit a guess about pages or pointer size.
• Let go of huge arrays or strings once the page no longer needs them.
• In Node.js, treat --max-old-space-size as a way to give the program more room, not as a fix for a leak.

If your question is about how V8 frees memory, read the garbage-collection lessons. If it is about the memory a function call uses, read the stack frames lesson next.

Common misconceptions

• “The JavaScript rules say objects live in new space or old space.” No. Spaces are V8's own way of organizing memory. The language rules never mention them.
• “Pointer compression means Node.js always has a 4 GB heap.” No. The official Node.js builds used here do not turn pointer compression on.
• “An object in large-object space is a leak.” No. It is only too big for a regular page. A leak is memory your app keeps by mistake, and it grows over time.
• “The sandbox and the garbage collector do the same job.” No. The sandbox limits what an attacker can reach. The garbage collector frees objects that nothing uses anymore.
• “The heap limit is the program's total memory.” No. It covers only V8's heap. The program also uses memory outside it.

Each row of this table is one idea. Read across to see what it really means and what it is not.

Practice exercises

// Page 0 has only 48 bytes left.
// The next object needs 192 bytes after rounding.

const base = 0x400000000n;
const address = base + 0x20n;
// Print the offset as 8 hexadecimal digits.

const base = 0x400000000n;
const address = base + 0x20n;
const offset = Number(address - base);
console.log("0x" + offset.toString(16).padStart(8, "0"));

Check your understanding

Key takeaways

• V8 splits its heap into spaces, because new objects, long-lived objects, machine code, and very big objects need different care.
• In current V8, a regular page is 256 KB. The biggest regular object is half a page, 128 KB.
• Bump allocation is fast: place the object at nextFree, then move nextFree forward. If it does not fit, use a new page.
• Pointer compression stores an address as a 4-byte offset inside a 4 GB cage. Chrome uses it. The official Node.js builds used here do not.
• The V8 sandbox is a safety layer. It stops a broken heap from reaching the rest of the program's memory.

Up next: Stack frames & calling conventions, where you will see what a function call puts on the stack.