completefrontend
Open lab My progress
THE WEB PERFORMANCE FIELD GUIDE

More lab tools

sitespeed.io, the Firefox Profiler, Safari's timelines, and Perfetto: when to reach for each. The store run through browsertime, profiled in Firefox, queried with SQL in Perfetto, and loaded in Chrome, Firefox and WebKit.

By the end, you can
  • 01
    Pick the tool by the questionSay which tool answers a question: many runs from a script, Firefox's work, Safari's work, or a whole trace.
  • 02
    Read what each tool saysRead browsertime's runs and its TBT rule, a Firefox profile's markers and samples, and Perfetto's answers in SQL.
  • 03
    Test in every engineLoad one page in Chrome, Firefox and WebKit, and compare them fairly.

A tool for each question

You've tested the Leaf and Kettle store with two tools so far. In The Performance panel, you recorded a load and read what Chrome did. In How Lighthouse works, Lighthouse loaded the store for you and wrote a report. Both tools open the store in Chrome. Both load it once, while you wait.

Sooner or later, you'll ask a question those two can't answer well:

  • Can I run the same test five times, every night, with nobody clicking?
  • Some visitors use Firefox. What is Firefox busy doing while the store loads?
  • Others use Safari, on an iPhone or a Mac. What is Safari busy doing?
  • I have a trace with thousands of events. Can I ask it a question, the way I'd search a spreadsheet?
  • Does the same code even take the same time in every browser?

Each question has its own tool. The course ran its store through four of them. It also loaded the store in three browsers: Chrome, Firefox and WebKit. A browser engine is the part of a browser that loads and draws pages, and WebKit is the one Safari is built on. Some tools can run WebKit without the rest of Safari, which is how the course tested it on a laptop.

To keep the story simple, this lesson follows one small piece of the store through every tool: its start-up work. Here it is, shortened from the store's app.js:

The store's start-up work, shortened from app.jsJavaScript
document.addEventListener("DOMContentLoaded", () => {  buildIndex();  showTeas(24);}); // Start-up work, like building a search index: sort 200,000 past order prices.function buildIndex() {  const prices = Array.from({ length: 200000 }, (_, i) => (i * 7919) % 1000);  return prices.sort((a, b) => a - b);}
  • Line 1 waits for DOMContentLoaded. The browser fires that event once it has read the whole HTML.
  • Line 2 calls buildIndex. Line 3 draws 24 teas on the page.
  • Lines 7 to 10 are buildIndex. Line 8 makes a list of 200,000 past order prices. Line 9 sorts them, the way a shop might build a search index.

The real listener does a little more, such as setting up the Add to cart button. But lines 2 and 3 are where its time goes, as the Firefox Profiler will show. It's a small job. But it's the same job in every tool, so you can compare what each one shows.

Here is the whole lesson in one picture: each question, and the tool that answers it.

Five questions about a page, and the lab tool that answers each: The same test, many runs? browsertime, sitespeed.io; What is Firefox busy doing? Firefox Profiler; What is Safari busy doing? Web Inspector; Ask a trace in SQL? Perfetto; Same speed in every browser? Load it in each engine.Your questionThe tool that answers itThe same test, many runs?browsertime, sitespeed.ioWhat is Firefox busy doing?Firefox ProfilerWhat is Safari busy doing?Web InspectorAsk a trace in SQL?PerfettoSame speed in every browser?Load it in each engine
Start on the left, with what you want to know. The tool comes second. Each of the next five sections takes one row.
Real-life analogyA toolbox: pick the tool by the job

You want to put up a shelf. Your toolbox holds a tape measure, a spirit level and a stud finder. The tape measure tells you how wide the wall is. The level tells you whether the shelf is straight. The stud finder tells you where the wooden beam is behind the plaster, so the screws will hold. You don't reach for the shiniest tool. You reach for the one that answers your question.

In real life: The toolbox
In the lab: Your lab tools: DevTools, Lighthouse, and the four in this lesson
In real life: "How wide? Is it straight? Where's the beam?"
In the lab: "Over many runs? In Firefox? In Safari? Across a whole trace?"
In real life: Reaching for the tool that answers the question
In the lab: Picking browsertime, a profiler or Perfetto by what you need to know
In real life: The shiniest tool in the box
In the lab: The tool you happen to know best
The main idea

Start from the question you're asking, then pick the tool that answers it. And test in the browsers your visitors use. The store's start-up work took a median of 34 ms in Chrome, 10 ms in Firefox and 18 ms in WebKit.

sitespeed.io and browsertime

First question: can you run the same test many times, with nobody clicking? In Scripting Lighthouse, a script ran Lighthouse for you. browsertime does the same kind of job for any page, in a real browser.

One command, five runs

browsertime is a command-line tool. Its docs sum it up: "Browsertime drives a real browser, runs your URL (or your script), and collects performance data." It is the engine inside sitespeed.io, a bigger tool that adds reports, dashboards and alerts.

One run goes like this:

  1. browsertime starts the browser through WebDriver, a standard way for one program to steer a browser.
  2. The browser loads your page.
  3. When the page is ready, browsertime runs small scripts inside it. They read the timings the page keeps about itself.
  4. It saves a HAR, short for HTTP Archive. That's a file listing every request and response, like a delivery log that lists every parcel.
  5. If you ask, it also films the screen.

browsertime measures real loads. Lighthouse's default works its numbers out instead, like the satnav in Throttling and simulation. The course ran browsertime on the store, served from the same laptop, with one command:

Five runs, one commandShell
npx browsertime@29.0.0 http://localhost:8080/ -n 5 --headless \
  --video false --visualMetrics false --chrome.CPUThrottlingRate 4
  • Line 1 runs browsertime 29.0.0 through npx, on the store's address. -n 5 asks for five runs. --headless runs Chrome without a window. The \ at the end carries the command on to line 2.
  • Line 2 turns the video off. Then it makes the CPU four times slower, like DevTools' mid-tier preset.

One command sets every run up the same way: same browser, same CPU, same page. It's Ravi's stopwatch from Choosing test conditions, started at the same door every time.

What five runs say

browsertime wrote its results to a file called browsertime.json. Here are the five LCPs from it. LCP, Largest Contentful Paint, is the moment the biggest image or block of text appeared.

Five runs, one middlePop out in the code editor (opens in a new tab)JavaScript
// LCP in browsertime's five runs of the store, in ms.const lcps = [464, 440, 452, 524, 320];const sorted = [...lcps].sort((a, b) => a - b);const middle = sorted[Math.floor(sorted.length / 2)]; console.log(`The middle run's LCP was ${middle} ms.`);console.log(`The runs ranged from ${sorted[0]} to ${sorted.at(-1)} ms.`);
  • Line 2 holds the five recorded LCPs, in milliseconds (ms).
  • Line 3 sorts a copy of them, smallest first.
  • Line 4 takes the middle one. That's the median.
  • Lines 6 and 7 print The middle run's LCP was 452 ms. and The runs ranged from 320 to 524 ms.

Five runs of the same page, set up the same way, and the slowest LCP came 204 ms after the fastest. That's the spread you met in Noise and variability. One run alone could have told you 320 or 524. So run a page several times, and compare medians, never one run with another.

Long tasks, but no blocking time

browsertime also reports long tasks. A long task is a piece of work on the page's main thread that takes more than 50 ms. While one runs, the page can't answer a tap. Total Blocking Time (TBT) adds up the part of each long task over 50 ms, as you saw in The supporting metrics.

Long tasks and TBT, run by runPop out in the code editor (opens in a new tab)JavaScript
// browsertime's five runs: time spent in long tasks, and Total Blocking Time, in ms.const longTasks = [138, 139, 139, 409, 364];const blocking = [0, 0, 0, 0, 0]; console.log(`Time in long tasks: ${longTasks.join(", ")} ms.`);console.log(`Total Blocking Time: ${blocking.join(", ")} ms.`);
  • Line 2 holds the recorded time each run spent in long tasks.
  • Line 3 holds each run's recorded TBT.
  • Lines 5 and 6 print Time in long tasks: 138, 139, 139, 409, 364 ms. and Total Blocking Time: 0, 0, 0, 0, 0 ms.

Every run had long tasks. Yet every TBT was zero. How?

browsertime's source code answers it. It adds a long task to TBT only when this test is true: if (firstContentfulPaint && longTask.startTime > firstContentfulPaint). In plain words: only if the task started after FCP, First Contentful Paint, the moment the first text or image appeared. Here is that rule, used on the first run:

browsertime's rule, on the first runPop out in the code editor (opens in a new tab)JavaScript
// browsertime's first run, in ms from the start of the load.const fcp = 464;const startupBegan = 243; // browsertime adds a long task to TBT only if it starts after FCP.if (startupBegan > fcp) {  console.log("The start-up task counts toward TBT.");} else {  console.log("The start-up task began before FCP, so TBT skips it.");}
  • Line 2 is the first run's FCP: 464 ms.
  • Line 3 is when its start-up work began: 243 ms.
  • Line 6 is browsertime's test. 243 isn't greater than 464, so line 9 runs.
  • It prints The start-up task began before FCP, so TBT skips it.

The start-up work runs inside one long task. That task began no later than the work itself, so it began before FCP too. This picture shows the first run:

browsertime's first run of the store: the start-up work ran from 243 ms to 378 ms, inside a long task, and FCP came at 464 ms. browsertime adds a long task to TBT only if it starts after FCP, so this one added nothing.browsertime, run 1FCP, 464 msstart-up work243 to 378 ms0 msnot countedcounted
browsertime skips every long task that starts left of the FCP line. The store's start-up work, and the long task it ran in, are all on that side.

In The supporting metrics, Lighthouse cut each task at FCP and counted the part after it. browsertime doesn't cut. A task that started before FCP doesn't count at all. So for a task that starts before FCP and ends after it, the two tools give different answers. Before you compare a number from two tools, check how each one defines it.

A steady laptop, and every request

After each run, browsertime times a small loop in the page. That's its CPU benchmark. It doesn't measure your page. It measures the machine. The docs say why you want it: "If the benchmark is unstable, your other metrics will also be unstable." Here it stayed between 191 and 208 ms across the five runs.

The HAR held 25 requests from the 5 runs. Each run asked for the store's four files and /favicon.ico, the small icon a browser shows in the tab.

A film of the load

The course turned the video off. Turned on, it shows you something no single number can: what the screen looked like at each moment.

Real-life analogyA dashcam films the whole delivery

A delivery van has a camera on its dashboard. It films the whole trip, from the depot to your door. A parcel's tracking page gives you a few times. The film shows everything between them. You can step through it frame by frame: the moment your street came into view, the moment the van stopped at your gate.

In real life: The dashcam, filming the whole trip
In the lab: browsertime's video of the screen, from the start of the load
In real life: Stepping through it frame by frame
In the lab: The filmstrip: one still picture after another
In real life: The moment your street came into view
In the lab: First Visual Change
In real life: The last moment anything in view moved
In the lab: Last Visual Change

With video on, browsertime films the screen while the page loads. sitespeed.io's docs say it records "a video at 30 fps of the screen": 30 pictures a second, one every 33 ms or so. A program called Visual Metrics then compares the frames, and works out when the screen changed:

  • First Visual Change: "The time when something is first painted within the viewport." The viewport is the part of the page you can see.
  • Speed Index: "the average time at which visible parts of the page are displayed." The sooner most of the screen fills in, the lower it is.
  • Last Visual Change: "The time when something last changes within the viewport."

sitespeed.io also lays still frames side by side as a filmstrip, in its HTML report. A filmstrip shows what a visitor saw at each moment: which part appeared first, and how long the screen stayed empty. Use it when you need to see how a page appears, not just when.

Video needs extra programs, such as FFmpeg. The docs call Docker "the easiest path", because its image already has them.

Running it every hour

Five runs today tell you about today. To watch a site over weeks, sitespeed.io runs the same tests on a schedule.

Real-life analogyA weather station on the roof, not one inspection

In How Lighthouse works, Lighthouse was a home inspector: one visit, one report. A weather station on your roof works differently. It reads the air every few minutes, day and night, and keeps every reading. Its graph shows you the cold snap that began on Tuesday, not just today's weather. And it can warn you when frost is coming.

In real life: The inspector's one visit
In the lab: One test, like one Lighthouse report
In real life: The station reading the air every few minutes
In the lab: sitespeed.io running the same tests on a schedule
In real life: Every reading kept, with its time
In the lab: Each run's numbers sent to a time series database
In real life: The graph that shows Tuesday's cold snap
In the lab: A Grafana graph that shows the day the page got slower
In real life: The frost warning on your phone
In the lab: An alert when a metric gets worse

To monitor a site, sitespeed.io's docs say, you "run the same tests again and again and send the metrics to a time series database". Then you follow them over time in Grafana. Two new names:

  • A time series database stores each number with the time it was measured.
  • Grafana draws those numbers as graphs, over days or months.

The docs' example starts the tests every hour. sitespeed.io can also send an alert, by email or Slack, when a metric gets worse. This is called synthetic monitoring. It's synthetic because a robot browser makes the visits, not real people. So it's lab data, as Lab data vs field data put it. It's monitoring because it never stops. Its graph can show you the day a release made the store slower, even in a week when nobody tested by hand.

When a number moves, is the change real, or noise? sitespeed.io's compare plugin checks for you. It runs statistical tests on the runs before and after a change, and gives a score. The docs: "A score below 0.05 means the difference is statistically significant." In plain words, noise alone would rarely make a gap that big. It's the honest comparison from Noise and variability, done for you.

The Firefox Profiler

Second question: what is Firefox busy doing while the store loads? Chrome's Performance panel can't tell you. Firefox records its own profiles: recordings of what the browser did, and when. You read them in the Firefox Profiler, at profiler.firefox.com.

The everyday way to record starts on that site. Click its big button, and it adds a profiler button to Firefox's toolbar. Choose the Web Developer preset, which the docs suggest "if you want to profile a web site". Start recording, load your page, and stop.

To catch a page from the very first moment, start the profiler together with Firefox. You do that with environment variables: settings you hand a program as it starts. The course started Firefox like this:

Profile Firefox from start to finishShell
MOZ_PROFILER_STARTUP=1 \
MOZ_PROFILER_STARTUP_FEATURES=js,stackwalk,screenshots \
MOZ_PROFILER_SHUTDOWN=profile.json \
firefox http://localhost:8080/
  • Line 1, MOZ_PROFILER_STARTUP=1, starts the profiler with Firefox. The docs: "the profiler is started as early as possible during startup."
  • Line 2 turns on three features. js records the JavaScript functions, and stackwalk records the browser's own C++ code. screenshots takes a picture of the window each time it's drawn.
  • Line 3, MOZ_PROFILER_SHUTDOWN, names "the file where the recorded profile should be saved" when Firefox quits.
  • Line 4 starts Firefox on the store. The \ at the end of each line before it carries the command on.

Samples and markers

A profile holds two kinds of record. Picture the tea shop first:

Real-life analogyGlancing into the kitchen every minute

The tea shop's manager wants to know where the cook's time goes. Every minute, she glances into the kitchen and ticks what the cook is doing: chopping, boiling, washing up. After an hour, the ticks show where most of the time went. But a ten-second job, like fetching the milk, can fall between two glances and never get a tick. The till is different. It prints the exact time of every order, and misses none.

In real life: A glance into the kitchen every minute
In the lab: A sample: the profiler notes what's running, every millisecond
In real life: The ticks, added up by job
In the lab: Samples counted by function, like buildIndex
In real life: A ten-second job between two glances
In the lab: Code too quick to land on a sample
In real life: The till's exact time for every order
In the lab: A marker: an event written down every time it happens

The glances are samples. In the docs' words, "the profiler stops the execution of the profiled code at a fixed rate, for example, every 1ms and records relevant information like the current stack." The stack is the list of functions running at that moment: which one called which.

The till's receipts are markers. Firefox writes one each time certain events happen, such as a DOM event firing. Markers "provide a view into the executing code that does not miss anything."

The course's profile of the store came from Firefox 155.0. First, the marker for the start-up work:

The marker: exact timesPop out in the code editor (opens in a new tab)JavaScript
// The store's DOMContentLoaded marker in a Firefox profile, in ms.const marker = { start: 473.2, end: 482.6 }; console.log(`The start-up work took ${(marker.end - marker.start).toFixed(1)} ms.`);
  • Line 2 holds the recorded start and end of the store's DOMContentLoaded marker, in ms.
  • Line 4 subtracts the start from the end, and prints The start-up work took 9.4 ms.

Then the samples taken inside that marker:

The samples: who was runningPop out in the code editor (opens in a new tab)JavaScript
// The samples Firefox took during the start-up work, counted by function.const taken = 9;const byFunction = { buildIndex: 8, showTeas: 1 }; for (const [name, count] of Object.entries(byFunction)) {  console.log(`${name} was running in ${count} of ${taken} samples.`);}
  • Line 2 is how many samples fell inside the marker: nine, about one per millisecond.
  • Line 3 counts them by the function that was running.
  • Lines 5 to 7 print buildIndex was running in 8 of 9 samples. and showTeas was running in 1 of 9 samples.
The store's DOMContentLoaded marker in Firefox 155.0: 9.4 ms, from 473.2 ms to 482.6 ms. Inside it, 9 samples, one every 1 ms: 8 in buildIndex, 1 in showTeas.MarkerDOMContentLoaded, 9.4 msSamplesbuildIndex ×8showTeas ×1
Top: the marker, with the start-up work's exact start and end. Bottom: the nine samples taken inside it, one a millisecond, grouped by the function that was running.

Put the two together. The marker says exactly how long the start-up work took. The samples say where that time went: almost all of it in buildIndex, sorting. To make the store start faster in Firefox, that's the function to look at.

Nine samples is a rough count, though. The docs warn: "There is no guarantee that all of the code that is running will be sampled." A job shorter than a millisecond can fall between two samples, like the trip for milk. For a short job, the docs suggest doing it many times while you record, so its samples add up.

Safari Web Inspector timelines

Third question: what is Safari busy doing? For that, you need Safari itself. Safari's developer tools are called Web Inspector, and their recordings live in the Timelines tab. WebKit's docs say it "contains all of the performance profiling and introspection tools provided by Web Inspector".

Each timeline records one kind of work. The main ones:

  • Screenshots: a picture of the visible page after every composite, each time the browser puts the screen together. It's Safari's dashcam.
  • Network Requests: every request, like DevTools' Network panel.
  • Layout & Rendering: every style recalculation, layout and paint.
  • JavaScript & Events: samples of JavaScript, grouped into call trees. It's the same idea as Firefox's samples.
  • CPU: how busy each thread of the page was.
  • Memory: how much memory the page used. It's off at first, because it can slow the page down.

A Frames view shows the same work another way: grouped by rendering frame, with lines at 30 and 60 frames a second. Those lines sit at about 33 ms and 16 ms. A frame taller than the 60 line took longer than 16 ms.

Three more things are worth knowing:

  • With Stop recording once page loads checked, a page-load recording stops by itself. It stops after 2 seconds with nothing to record, 10 seconds after the load event, or 90 seconds after it started.
  • While it records, Web Inspector turns its own debugging features off, breakpoints included. The docs say this is "in order to maximize the performance of the inspected page and therefore measure realistic performance." So a breakpoint you forgot won't pause the page in the middle of a recording.
  • You can export a recording to a file, and import it later, to share it with a teammate.

Safari from a script

sitespeed.io can drive Safari too, on a Mac, or on an iPhone plugged into a Mac with a USB cable. On a Mac, it misses some data. The docs: "On macOS Safari we do not support HAR, cookies and request headers." On an iPhone over USB, it collects a HAR, video and visual metrics.

Playwright's WebKit isn't Safari

The course's WebKit numbers, in the last section of this lesson, came from Playwright, a tool for steering browsers from code. Playwright's docs: "Playwright's WebKit is derived from the latest WebKit main branch sources, often before these updates are incorporated into Apple Safari and other WebKit-based browsers." And: "Playwright doesn't work with the branded version of Safari since it relies on patches." A patch is a small change to a program's source code.

So the course's WebKit 26.6 is a close relative of Safari, not Safari. For Safari's own numbers, use Safari.

Perfetto for deep traces

Fourth question: can you ask a trace a question? A trace is the kind of recording the Performance panel makes: a long list of events, each with its start and its length. In Reading a performance trace, you read one by eye: zooming, clicking, hovering. That works for one trace. It gets slow when you want the same answer from twenty.

Perfetto is an open-source tool for recording and reading traces. Chromium's developers now record most of their traces in its format. Its web page, the Perfetto UI, opens other formats too. That includes the JSON files DevTools' Performance panel saves: "Chrome JSON files can be opened directly in the Perfetto UI."

Its trace processor goes further. It reads a trace into tables, like a spreadsheet with rows and columns. The docs: the events "are parsed into Perfetto's standard SQL tables (e.g., slice, track, process, thread, counter, args)." Two words from that:

  • SQL is a language for asking tables questions.
  • A slice is one piece of work with a start and a length, such as one task.

It's like asking a library's catalogue for every book over 500 pages, instead of walking along every shelf. From Python, you install it with pip install perfetto.

The course traced one load of the store in Chrome, with the CPU four times slower. Then it asked the trace three questions. Here is the first, in SQL: every task of 50 ms or more.

Every long task, in SQLSQL
SELECT ROUND(s.dur / 1e6, 1) AS msFROM slice sJOIN thread_track tt ON s.track_id = tt.idJOIN thread t USING (utid)WHERE t.name = 'CrRendererMain' AND s.depth = 0 AND s.dur >= 50e6ORDER BY s.ts
  • Line 1 picks each slice's length. Perfetto counts time in nanoseconds, billionths of a second. Dividing by 1e6, a million, turns it into milliseconds.
  • Line 2 reads from the slice table.
  • Lines 3 and 4 join each slice to the thread it ran on.
  • Line 5 does the filtering. CrRendererMain is Chrome's name for the page's main thread. depth = 0 keeps whole tasks, not the smaller pieces inside them. 50e6 nanoseconds is 50 ms.
  • Line 6 lists them in the order they ran.

Pick a question to see its SQL, and the answer Perfetto gave:

Playground: asking a trace in SQL
The query, and what Perfetto answeredSQL
SELECT ROUND(s.dur / 1e6, 1) AS ms
FROM slice s
JOIN thread_track tt ON s.track_id = tt.id
JOIN thread t USING (utid)
WHERE t.name = 'CrRendererMain' AND s.depth = 0 AND s.dur >= 50e6
ORDER BY s.ts
ms
238.7
99.3
150.2
Try it yourself
Long tasks

Every whole task of 50 ms or more on the page's main thread, in the order they ran. Perfetto answered with 3 rows.

Recorded on Apple M1 Pro (8 cores): one load of the store in Chrome, with the CPU 4 times slower, traced and then queried in Perfetto's trace processor (perfetto==0.59.1). The page's own timing for that load: LCP 396 ms, start-up work 143.3 ms.

The long-task question came back with three rows: 238.7, 99.3 and 150.2 ms. The start-up work question found the store's DOMContentLoaded event: 143.3 ms, the same as the page's own timing for that load. With the CPU four times slower, that's about four times the 34 ms it took in Chrome at full speed.

One query replaced many clicks in a panel. And it runs on a hundred traces as easily as on one.

Perfetto reads the Firefox Profiler's format too. Given the course's Firefox profile, it loaded all 12,420 samples, from every thread. But not one function on them was named buildIndex. For Firefox's JavaScript, read the profile in the Firefox Profiler.

Testing in every engine

Last question: does the same code take the same time in every browser? Each browser runs JavaScript with its own JavaScript engine:

  • Chrome uses V8.
  • Firefox uses SpiderMonkey.
  • Safari, and the WebKit engine under it, use JavaScriptCore.
Real-life analogyOne recipe, three cooks

Give the same recipe to three cooks in three kitchens. Each one follows it, but chops and stirs in their own style, with their own knives. One finishes in ten minutes. Another takes half an hour. To know how long your dish takes, you time it in the kitchens where it will really be cooked.

In real life: The recipe
In the lab: The store's app.js, the same in every browser
In real life: Three cooks, each with their own way of working
In the lab: Three JavaScript engines: V8, SpiderMonkey and JavaScriptCore
In real life: Ten minutes for one, half an hour for another
In the lab: 10 ms in Firefox, 34 ms in Chrome
In real life: Timing the dish in the kitchens where it's cooked
In the lab: Testing in the browsers your visitors use

The course loaded the store in three browsers on the same laptop, five times each, taking turns, with no throttling. Chrome was Google Chrome. Firefox and WebKit were Playwright's builds. Here is the start-up work in each:

The same code, three browsersPop out in the code editor (opens in a new tab)JavaScript
// The store's start-up work in ms: five loads in each browser, taking turns, with no throttling.const startup = {  Chrome: [36.5, 33.8, 34.2, 33.5, 34],  Firefox: [9, 9, 10, 10, 10],  WebKit: [32, 17, 18, 18, 18],};const median = (list) => [...list].sort((a, b) => a - b)[Math.floor(list.length / 2)]; for (const [browser, list] of Object.entries(startup)) {  console.log(`${browser} ran it in ${median(list)} ms.`);}
  • Lines 3 to 5 hold the recorded times, five loads per browser, from each page's own timing.
  • Line 7 finds the median, as before.
  • Lines 9 to 11 print Chrome ran it in 34 ms., Firefox ran it in 10 ms. and WebKit ran it in 18 ms.
The store's start-up work with no throttling, the median of 5 loads: Chrome, 34 ms; Firefox, 10 ms; WebKit, 18 ms.Chrome (V8)34 msFirefox (SpiderMonkey)10 msWebKit (JavaScriptCore)18 ms
The same start-up code in three browsers, on one laptop. Each bar is the median of five loads. Each thin line is one load.

The same JavaScript ran more than three times as fast in Firefox as in Chrome, on the same laptop. So a fix that saves a lot in one engine can save less in another. Measure in the browsers your visitors use, not only the one you build in.

Pick a browser to see its five loads, with FCP and LCP too:

Playground: one page, three engines
5 loads0 to 236 ms
19 ms, 190 ms, 190 ms
29 ms, 116 ms, 116 ms
310 ms, 111 ms, 111 ms
410 ms, 104 ms, 104 ms
510 ms, 101 ms, 101 ms

start-up work FCP LCP. The times on the right are in the same order.

Try it yourself
Firefox

Firefox 155.0 runs JavaScript with SpiderMonkey. The start-up work took a median of 10 ms. The first text or image (FCP) appeared at a median of 111 ms, and the biggest one (LCP) at 111 ms. It reports 8 kinds of performance entry. It doesn't report element, interaction-contentful-paint, layout-shift, long-animation-frame, longtask, soft-navigation, visibility-state.

Recorded on Apple M1 Pro (8 cores), from the page's own timing. 5 loads in each browser, taking turns, each in a fresh browser window, phone-sized, with no throttling. Chrome is Google Chrome. Firefox and WebKit are Playwright's builds.

Different engines report different things

Engines also differ in what they can tell you. Every browser keeps a list called PerformanceObserver.supportedEntryTypes. It names each kind of performance entry the browser can report, such as layout-shift or longtask.

What each browser can reportPop out in the code editor (opens in a new tab)JavaScript
// PerformanceObserver.supportedEntryTypes, as each browser reported it.const types = {  Chrome: ["element", "event", "first-input", "interaction-contentful-paint", "largest-contentful-paint", "layout-shift", "long-animation-frame", "longtask", "mark", "measure", "navigation", "paint", "resource", "soft-navigation", "visibility-state"],  Firefox: ["event", "first-input", "largest-contentful-paint", "mark", "measure", "navigation", "paint", "resource"],  WebKit: ["event", "first-input", "largest-contentful-paint", "mark", "measure", "navigation", "paint", "resource"],};const everywhere = types.Chrome.filter((type) => types.Firefox.includes(type) && types.WebKit.includes(type));const chromeOnly = types.Chrome.filter((type) => !everywhere.includes(type)); console.log(`All three report: ${everywhere.join(", ")}.`);console.log(`Only Chrome reports: ${chromeOnly.join(", ")}.`);
  • Lines 3 to 5 hold each browser's recorded list.
  • Line 7 keeps the types all three share. Line 8 keeps the rest of Chrome's.
  • Line 10 prints All three report: event, first-input, largest-contentful-paint, mark, measure, navigation, paint, resource.
  • Line 11 prints Only Chrome reports: element, interaction-contentful-paint, layout-shift, long-animation-frame, longtask, soft-navigation, visibility-state.

Firefox and WebKit report no layout shifts, so a script can't add up CLS there. They report no long tasks either. Even paints differed. In Chrome and Firefox, LCP came at the same moment as FCP in every load. In WebKit, LCP came 13 to 44 ms after FCP. Before you compare two browsers, check that both can measure the thing you're comparing.

Only Chrome's CPU was slowed

The course slowed the CPU in Chrome only. CPU throttling is a command in the Chrome DevTools Protocol (CDP), the protocol tools use to steer Chromium-based browsers. The command is Emulation.setCPUThrottlingRate, which "Enables CPU throttling to emulate slow CPUs." And Playwright's docs say: "CDP sessions are only supported on Chromium-based browsers."

So Firefox and WebKit ran at the laptop's full speed. Compare browsers with each other at the same speed. Don't put Firefox's full-speed numbers beside Chrome's slowed-down ones.

These were lab loads, on one laptop. To see how your real visitors' browsers compare, Field data from every browser comes later in the course.

Which tool?

Here are the four tools from this lesson in one table. Read each row across: the tool, the browsers it works with, and what it's best at.

Four lab tools, side by side
ToolWhich browsers it works withWhat it's best at
browsertime and sitespeed.ioChrome, Firefox, Edge and Safari on a computer. Chrome and Firefox on Android, and Safari on iOS.Running a page many times from a script, with statistics, a HAR and video. On a schedule, too.
The Firefox ProfilerFirefox.Markers with exact times, and samples that name the JavaScript function that was running.
Web Inspector's Timelines tabSafari.Safari's own timelines: screenshots, requests, rendering, JavaScript, CPU and memory.
PerfettoIt reads traces from Chrome, from the Firefox Profiler, and in other formats.Asking one trace, or a hundred, questions in SQL.

Now sort each card by the tool it describes. Ask yourself first which question the card answers.

Which tool?
  • Runs a page five times from one command, and writes down every request
  • Times a small loop after every run, to check the machine stayed steady
  • Films the screen and works out Speed Index from the frames
  • Runs the same tests every hour and graphs them over months
  • Notes which function is running every millisecond, in Firefox
  • Starts recording the moment the browser starts, with MOZ_PROFILER_STARTUP=1
  • Screenshots, Layout & Rendering, and JavaScript & Events timelines, for Safari
  • Stops a page-load recording 10 seconds after the load event
  • Answers questions about a Chrome trace in SQL
  • Opens Chrome's JSON traces and Firefox Profiler files
Try it yourself
0 of 10 correct

Sort each card by the tool it describes.

Choose a category for every card. You can change an answer at any time; Reset clears them all.

Common mix-ups

  • "A TBT of zero means no long tasks." No. browsertime recorded 138 to 409 ms of long tasks in each run, and a TBT of 0 every time. Its TBT skips any task that starts before FCP.
  • "Monitoring with sitespeed.io measures my real visitors." No. A robot browser makes every visit. It's lab data, taken on a schedule.
  • "Playwright's WebKit is Safari." No. It's built from WebKit's newest sources, with patches. For Safari, use Safari.
  • "Every browser reports the same metrics." No. Firefox and WebKit reported no layout shifts and no long tasks.
  • "CPU throttling works in every browser." No. It's a Chrome DevTools Protocol command, and Playwright opens CDP sessions in Chromium-based browsers only.
  • "Any tool reads any trace just as well." No. Perfetto loaded the Firefox profile's samples, but not its JavaScript function names.

Practice exercises

Answer first, then use the hints.

Exercise 1 · Warm-upA slow run

If browsertime's second run had an LCP of 900 ms, what would line 6 print?

Starter codePop out in the code editor (opens in a new tab)JavaScript
// LCP in browsertime's five runs of the store, in ms.
const lcps = [464, 900, 452, 524, 320];
const sorted = [...lcps].sort((a, b) => a - b);
const middle = sorted[Math.floor(sorted.length / 2)];

console.log(`The middle run's LCP was ${middle} ms.`);
console.log(`The runs ranged from ${sorted[0]} to ${sorted.at(-1)} ms.`);

Answer, then press Check. Spacing and letter case don’t matter.

    Exercise 2 · Warm-upLayout shifts everywhere

    If Firefox and WebKit both reported layout shifts, what would line 11 print?

    Starter codePop out in the code editor (opens in a new tab)JavaScript
    // PerformanceObserver.supportedEntryTypes, as each browser reported it.
    const types = {
      Chrome: ["element", "event", "first-input", "interaction-contentful-paint", "largest-contentful-paint", "layout-shift", "long-animation-frame", "longtask", "mark", "measure", "navigation", "paint", "resource", "soft-navigation", "visibility-state"],
      Firefox: ["event", "first-input", "largest-contentful-paint", "layout-shift", "mark", "measure", "navigation", "paint", "resource"],
      WebKit: ["event", "first-input", "largest-contentful-paint", "layout-shift", "mark", "measure", "navigation", "paint", "resource"],
    };
    const everywhere = types.Chrome.filter((type) => types.Firefox.includes(type) && types.WebKit.includes(type));
    const chromeOnly = types.Chrome.filter((type) => !everywhere.includes(type));
    
    console.log(`All three report: ${everywhere.join(", ")}.`);
    console.log(`Only Chrome reports: ${chromeOnly.join(", ")}.`);

    Answer, then press Check. Spacing and letter case don’t matter.

      Exercise 3 · PracticeWebKit's LCP

      In WebKit, what was the store's median LCP?

      Answer, then press Check. Spacing and letter case don’t matter.

        Exercise 4 · PracticeStyle recalculations

        In the traced load, how many times did UpdateLayoutTree, the style recalculation, run on the main thread?

        Answer, then press Check. Spacing and letter case don’t matter.

          Exercise 5 · ChallengeEvery profiler setting

          Which environment variable makes Firefox list every profiler setting it accepts, and then quit?

          Answer, then press Check. Spacing and letter case don’t matter.

            Exercise 6 · ChallengeComparing runs

            sitespeed.io's compare plugin uses one test for two independent sets of runs, and another for paired data. Which test does it use for independent runs?

            Answer, then press Check. Spacing and letter case don’t matter.

              Check your understanding

              For each question, ask what you want to know, which tool records it, and in which browser.

              More lab tools quiz · 7 questionsScore: first tries count
              1. Question 1 of 7What does the third line of the engines example print?

                Read the code, then predictPop out in the code editor (opens in a new tab)JavaScript
                // The store's start-up work in ms: five loads in each browser, taking turns, with no throttling.
                const startup = {
                  Chrome: [36.5, 33.8, 34.2, 33.5, 34],
                  Firefox: [9, 9, 10, 10, 10],
                  WebKit: [32, 17, 18, 18, 18],
                };
                const median = (list) => [...list].sort((a, b) => a - b)[Math.floor(list.length / 2)];
                
                for (const [browser, list] of Object.entries(startup)) {
                  console.log(`${browser} ran it in ${median(list)} ms.`);
                }

                Choose an answer to see the explanation.

              2. Question 2 of 7If the start-up work had begun at 500 ms, after FCP, what would browsertime's rule print?

                Read the code, then predictPop out in the code editor (opens in a new tab)JavaScript
                // browsertime's first run, in ms from the start of the load.
                const fcp = 464;
                const startupBegan = 500;
                
                // browsertime adds a long task to TBT only if it starts after FCP.
                if (startupBegan > fcp) {
                  console.log("The start-up task counts toward TBT.");
                } else {
                  console.log("The start-up task began before FCP, so TBT skips it.");
                }

                Choose an answer to see the explanation.

              3. Question 3 of 7browsertime recorded long tasks in every run of the store, yet a Total Blocking Time of zero. Why?

                Choose an answer to see the explanation.

              4. Question 4 of 7In Firefox, you want to know which JavaScript function used the time inside a slow event listener. Which tool?

                Choose an answer to see the explanation.

              5. Question 5 of 7You want to find the day a release made the store slower. What do you need?

                Choose an answer to see the explanation.

              6. Question 6 of 7Why did the course slow the CPU in Chrome only?

                Choose an answer to see the explanation.

              7. Question 7 of 7Is Playwright's WebKit the same as Safari?

                Choose an answer to see the explanation.

              Key takeaways

              • Pick a lab tool by the question you're asking, not by the tool you know best.
              • browsertime runs a page many times from one command. It writes statistics, a HAR, a CPU benchmark and, if you ask, a video.
              • sitespeed.io runs those tests on a schedule and graphs them, so you can see the day a page got slower.
              • Check how a tool defines a metric before you compare it with another tool's. browsertime's TBT skips tasks that start before FCP.
              • The Firefox Profiler shows Firefox's work: markers with exact times, and samples that name the function running.
              • Safari's Web Inspector records Safari's own timelines. Playwright's WebKit is close to Safari, but it isn't Safari.
              • Perfetto turns a trace into tables, so you can ask it questions in SQL.
              • Browsers run the same code at different speeds, and report different metrics. Test in the ones your visitors use.

              Remember the one-liner.
              Ask your question first, then pick the tool, and the browsers your visitors use.

              Sources

              The facts on this page come from these docs and source files:

              Coming next: The Chrome User Experience Report, the first lesson of Measuring real users. So far, every number in the course came from a test you ran. Next, you'll see what Chrome collects from real visitors, and what it can and can't tell you.

              CompleteFrontend Clear concepts. Working examples.