More lab tools
sitespeed.io, the Firefox Profiler, Safari's timelines, and Perfetto: when to reach for each. The store run through browsertime, profiled in Firefox, queried with SQL in Perfetto, and loaded in Chrome, Firefox and WebKit.
- 01Pick the tool by the questionSay which tool answers a question: many runs from a script, Firefox's work, Safari's work, or a whole trace.
- 02Read what each tool saysRead browsertime's runs and its TBT rule, a Firefox profile's markers and samples, and Perfetto's answers in SQL.
- 03Test in every engineLoad one page in Chrome, Firefox and WebKit, and compare them fairly.
A tool for each question
You've tested the Leaf and Kettle store with two tools so far. In The Performance panel, you recorded a load and read what Chrome did. In How Lighthouse works, Lighthouse loaded the store for you and wrote a report. Both tools open the store in Chrome. Both load it once, while you wait.
Sooner or later, you'll ask a question those two can't answer well:
- Can I run the same test five times, every night, with nobody clicking?
- Some visitors use Firefox. What is Firefox busy doing while the store loads?
- Others use Safari, on an iPhone or a Mac. What is Safari busy doing?
- I have a trace with thousands of events. Can I ask it a question, the way I'd search a spreadsheet?
- Does the same code even take the same time in every browser?
Each question has its own tool. The course ran its store through four of them. It also loaded the store in three browsers: Chrome, Firefox and WebKit. A browser engine is the part of a browser that loads and draws pages, and WebKit is the one Safari is built on. Some tools can run WebKit without the rest of Safari, which is how the course tested it on a laptop.
To keep the story simple, this lesson follows one small piece of the store through every tool: its start-up work. Here it is, shortened from the store's app.js:
document.addEventListener("DOMContentLoaded", () => { buildIndex(); showTeas(24);}); // Start-up work, like building a search index: sort 200,000 past order prices.function buildIndex() { const prices = Array.from({ length: 200000 }, (_, i) => (i * 7919) % 1000); return prices.sort((a, b) => a - b);}- Line 1 waits for
DOMContentLoaded. The browser fires that event once it has read the whole HTML. - Line 2 calls
buildIndex. Line 3 draws 24 teas on the page. - Lines 7 to 10 are
buildIndex. Line 8 makes a list of 200,000 past order prices. Line 9 sorts them, the way a shop might build a search index.
The real listener does a little more, such as setting up the Add to cart button. But lines 2 and 3 are where its time goes, as the Firefox Profiler will show. It's a small job. But it's the same job in every tool, so you can compare what each one shows.
Here is the whole lesson in one picture: each question, and the tool that answers it.
You want to put up a shelf. Your toolbox holds a tape measure, a spirit level and a stud finder. The tape measure tells you how wide the wall is. The level tells you whether the shelf is straight. The stud finder tells you where the wooden beam is behind the plaster, so the screws will hold. You don't reach for the shiniest tool. You reach for the one that answers your question.
- In real life: The toolbox
- In the lab: Your lab tools: DevTools, Lighthouse, and the four in this lesson
- In real life: "How wide? Is it straight? Where's the beam?"
- In the lab: "Over many runs? In Firefox? In Safari? Across a whole trace?"
- In real life: Reaching for the tool that answers the question
- In the lab: Picking browsertime, a profiler or Perfetto by what you need to know
- In real life: The shiniest tool in the box
- In the lab: The tool you happen to know best
Start from the question you're asking, then pick the tool that answers it. And test in the browsers your visitors use. The store's start-up work took a median of 34 ms in Chrome, 10 ms in Firefox and 18 ms in WebKit.
sitespeed.io and browsertime
First question: can you run the same test many times, with nobody clicking? In Scripting Lighthouse, a script ran Lighthouse for you. browsertime does the same kind of job for any page, in a real browser.
One command, five runs
browsertime is a command-line tool. Its docs sum it up: "Browsertime drives a real browser, runs your URL (or your script), and collects performance data." It is the engine inside sitespeed.io, a bigger tool that adds reports, dashboards and alerts.
One run goes like this:
- browsertime starts the browser through WebDriver, a standard way for one program to steer a browser.
- The browser loads your page.
- When the page is ready, browsertime runs small scripts inside it. They read the timings the page keeps about itself.
- It saves a HAR, short for HTTP Archive. That's a file listing every request and response, like a delivery log that lists every parcel.
- If you ask, it also films the screen.
browsertime measures real loads. Lighthouse's default works its numbers out instead, like the satnav in Throttling and simulation. The course ran browsertime on the store, served from the same laptop, with one command:
npx browsertime@29.0.0 http://localhost:8080/ -n 5 --headless \
--video false --visualMetrics false --chrome.CPUThrottlingRate 4- Line 1 runs browsertime 29.0.0 through
npx, on the store's address.-n 5asks for five runs.--headlessruns Chrome without a window. The\at the end carries the command on to line 2. - Line 2 turns the video off. Then it makes the CPU four times slower, like DevTools' mid-tier preset.
One command sets every run up the same way: same browser, same CPU, same page. It's Ravi's stopwatch from Choosing test conditions, started at the same door every time.
What five runs say
browsertime wrote its results to a file called browsertime.json. Here are the five LCPs from it. LCP, Largest Contentful Paint, is the moment the biggest image or block of text appeared.
// LCP in browsertime's five runs of the store, in ms.const lcps = [464, 440, 452, 524, 320];const sorted = [...lcps].sort((a, b) => a - b);const middle = sorted[Math.floor(sorted.length / 2)]; console.log(`The middle run's LCP was ${middle} ms.`);console.log(`The runs ranged from ${sorted[0]} to ${sorted.at(-1)} ms.`);- Line 2 holds the five recorded LCPs, in milliseconds (ms).
- Line 3 sorts a copy of them, smallest first.
- Line 4 takes the middle one. That's the median.
- Lines 6 and 7 print
The middle run's LCP was 452 ms.andThe runs ranged from 320 to 524 ms.
Five runs of the same page, set up the same way, and the slowest LCP came 204 ms after the fastest. That's the spread you met in Noise and variability. One run alone could have told you 320 or 524. So run a page several times, and compare medians, never one run with another.
Long tasks, but no blocking time
browsertime also reports long tasks. A long task is a piece of work on the page's main thread that takes more than 50 ms. While one runs, the page can't answer a tap. Total Blocking Time (TBT) adds up the part of each long task over 50 ms, as you saw in The supporting metrics.
// browsertime's five runs: time spent in long tasks, and Total Blocking Time, in ms.const longTasks = [138, 139, 139, 409, 364];const blocking = [0, 0, 0, 0, 0]; console.log(`Time in long tasks: ${longTasks.join(", ")} ms.`);console.log(`Total Blocking Time: ${blocking.join(", ")} ms.`);- Line 2 holds the recorded time each run spent in long tasks.
- Line 3 holds each run's recorded TBT.
- Lines 5 and 6 print
Time in long tasks: 138, 139, 139, 409, 364 ms.andTotal Blocking Time: 0, 0, 0, 0, 0 ms.
Every run had long tasks. Yet every TBT was zero. How?
browsertime's source code answers it. It adds a long task to TBT only when this test is true: if (firstContentfulPaint && longTask.startTime > firstContentfulPaint). In plain words: only if the task started after FCP, First Contentful Paint, the moment the first text or image appeared. Here is that rule, used on the first run:
// browsertime's first run, in ms from the start of the load.const fcp = 464;const startupBegan = 243; // browsertime adds a long task to TBT only if it starts after FCP.if (startupBegan > fcp) { console.log("The start-up task counts toward TBT.");} else { console.log("The start-up task began before FCP, so TBT skips it.");}- Line 2 is the first run's FCP: 464 ms.
- Line 3 is when its start-up work began: 243 ms.
- Line 6 is browsertime's test. 243 isn't greater than 464, so line 9 runs.
- It prints
The start-up task began before FCP, so TBT skips it.
The start-up work runs inside one long task. That task began no later than the work itself, so it began before FCP too. This picture shows the first run:
In The supporting metrics, Lighthouse cut each task at FCP and counted the part after it. browsertime doesn't cut. A task that started before FCP doesn't count at all. So for a task that starts before FCP and ends after it, the two tools give different answers. Before you compare a number from two tools, check how each one defines it.
A steady laptop, and every request
After each run, browsertime times a small loop in the page. That's its CPU benchmark. It doesn't measure your page. It measures the machine. The docs say why you want it: "If the benchmark is unstable, your other metrics will also be unstable." Here it stayed between 191 and 208 ms across the five runs.
The HAR held 25 requests from the 5 runs. Each run asked for the store's four files and /favicon.ico, the small icon a browser shows in the tab.
A film of the load
The course turned the video off. Turned on, it shows you something no single number can: what the screen looked like at each moment.
A delivery van has a camera on its dashboard. It films the whole trip, from the depot to your door. A parcel's tracking page gives you a few times. The film shows everything between them. You can step through it frame by frame: the moment your street came into view, the moment the van stopped at your gate.
- In real life: The dashcam, filming the whole trip
- In the lab: browsertime's video of the screen, from the start of the load
- In real life: Stepping through it frame by frame
- In the lab: The filmstrip: one still picture after another
- In real life: The moment your street came into view
- In the lab: First Visual Change
- In real life: The last moment anything in view moved
- In the lab: Last Visual Change
With video on, browsertime films the screen while the page loads. sitespeed.io's docs say it records "a video at 30 fps of the screen": 30 pictures a second, one every 33 ms or so. A program called Visual Metrics then compares the frames, and works out when the screen changed:
- First Visual Change: "The time when something is first painted within the viewport." The viewport is the part of the page you can see.
- Speed Index: "the average time at which visible parts of the page are displayed." The sooner most of the screen fills in, the lower it is.
- Last Visual Change: "The time when something last changes within the viewport."
sitespeed.io also lays still frames side by side as a filmstrip, in its HTML report. A filmstrip shows what a visitor saw at each moment: which part appeared first, and how long the screen stayed empty. Use it when you need to see how a page appears, not just when.
Video needs extra programs, such as FFmpeg. The docs call Docker "the easiest path", because its image already has them.
Running it every hour
Five runs today tell you about today. To watch a site over weeks, sitespeed.io runs the same tests on a schedule.
In How Lighthouse works, Lighthouse was a home inspector: one visit, one report. A weather station on your roof works differently. It reads the air every few minutes, day and night, and keeps every reading. Its graph shows you the cold snap that began on Tuesday, not just today's weather. And it can warn you when frost is coming.
- In real life: The inspector's one visit
- In the lab: One test, like one Lighthouse report
- In real life: The station reading the air every few minutes
- In the lab: sitespeed.io running the same tests on a schedule
- In real life: Every reading kept, with its time
- In the lab: Each run's numbers sent to a time series database
- In real life: The graph that shows Tuesday's cold snap
- In the lab: A Grafana graph that shows the day the page got slower
- In real life: The frost warning on your phone
- In the lab: An alert when a metric gets worse
To monitor a site, sitespeed.io's docs say, you "run the same tests again and again and send the metrics to a time series database". Then you follow them over time in Grafana. Two new names:
- A time series database stores each number with the time it was measured.
- Grafana draws those numbers as graphs, over days or months.
The docs' example starts the tests every hour. sitespeed.io can also send an alert, by email or Slack, when a metric gets worse. This is called synthetic monitoring. It's synthetic because a robot browser makes the visits, not real people. So it's lab data, as Lab data vs field data put it. It's monitoring because it never stops. Its graph can show you the day a release made the store slower, even in a week when nobody tested by hand.
When a number moves, is the change real, or noise? sitespeed.io's compare plugin checks for you. It runs statistical tests on the runs before and after a change, and gives a score. The docs: "A score below 0.05 means the difference is statistically significant." In plain words, noise alone would rarely make a gap that big. It's the honest comparison from Noise and variability, done for you.
The Firefox Profiler
Second question: what is Firefox busy doing while the store loads? Chrome's Performance panel can't tell you. Firefox records its own profiles: recordings of what the browser did, and when. You read them in the Firefox Profiler, at profiler.firefox.com.
The everyday way to record starts on that site. Click its big button, and it adds a profiler button to Firefox's toolbar. Choose the Web Developer preset, which the docs suggest "if you want to profile a web site". Start recording, load your page, and stop.
To catch a page from the very first moment, start the profiler together with Firefox. You do that with environment variables: settings you hand a program as it starts. The course started Firefox like this:
MOZ_PROFILER_STARTUP=1 \
MOZ_PROFILER_STARTUP_FEATURES=js,stackwalk,screenshots \
MOZ_PROFILER_SHUTDOWN=profile.json \
firefox http://localhost:8080/- Line 1,
MOZ_PROFILER_STARTUP=1, starts the profiler with Firefox. The docs: "the profiler is started as early as possible during startup." - Line 2 turns on three features.
jsrecords the JavaScript functions, andstackwalkrecords the browser's own C++ code.screenshotstakes a picture of the window each time it's drawn. - Line 3,
MOZ_PROFILER_SHUTDOWN, names "the file where the recorded profile should be saved" when Firefox quits. - Line 4 starts Firefox on the store. The
\at the end of each line before it carries the command on.
Samples and markers
A profile holds two kinds of record. Picture the tea shop first:
The tea shop's manager wants to know where the cook's time goes. Every minute, she glances into the kitchen and ticks what the cook is doing: chopping, boiling, washing up. After an hour, the ticks show where most of the time went. But a ten-second job, like fetching the milk, can fall between two glances and never get a tick. The till is different. It prints the exact time of every order, and misses none.
- In real life: A glance into the kitchen every minute
- In the lab: A sample: the profiler notes what's running, every millisecond
- In real life: The ticks, added up by job
- In the lab: Samples counted by function, like
buildIndex - In real life: A ten-second job between two glances
- In the lab: Code too quick to land on a sample
- In real life: The till's exact time for every order
- In the lab: A marker: an event written down every time it happens
The glances are samples. In the docs' words, "the profiler stops the execution of the profiled code at a fixed rate, for example, every 1ms and records relevant information like the current stack." The stack is the list of functions running at that moment: which one called which.
The till's receipts are markers. Firefox writes one each time certain events happen, such as a DOM event firing. Markers "provide a view into the executing code that does not miss anything."
The course's profile of the store came from Firefox 155.0. First, the marker for the start-up work:
// The store's DOMContentLoaded marker in a Firefox profile, in ms.const marker = { start: 473.2, end: 482.6 }; console.log(`The start-up work took ${(marker.end - marker.start).toFixed(1)} ms.`);- Line 2 holds the recorded start and end of the store's
DOMContentLoadedmarker, in ms. - Line 4 subtracts the start from the end, and prints
The start-up work took 9.4 ms.
Then the samples taken inside that marker:
// The samples Firefox took during the start-up work, counted by function.const taken = 9;const byFunction = { buildIndex: 8, showTeas: 1 }; for (const [name, count] of Object.entries(byFunction)) { console.log(`${name} was running in ${count} of ${taken} samples.`);}- Line 2 is how many samples fell inside the marker: nine, about one per millisecond.
- Line 3 counts them by the function that was running.
- Lines 5 to 7 print
buildIndex was running in 8 of 9 samples.andshowTeas was running in 1 of 9 samples.
Put the two together. The marker says exactly how long the start-up work took. The samples say where that time went: almost all of it in buildIndex, sorting. To make the store start faster in Firefox, that's the function to look at.
Nine samples is a rough count, though. The docs warn: "There is no guarantee that all of the code that is running will be sampled." A job shorter than a millisecond can fall between two samples, like the trip for milk. For a short job, the docs suggest doing it many times while you record, so its samples add up.
Safari Web Inspector timelines
Third question: what is Safari busy doing? For that, you need Safari itself. Safari's developer tools are called Web Inspector, and their recordings live in the Timelines tab. WebKit's docs say it "contains all of the performance profiling and introspection tools provided by Web Inspector".
Each timeline records one kind of work. The main ones:
- Screenshots: a picture of the visible page after every composite, each time the browser puts the screen together. It's Safari's dashcam.
- Network Requests: every request, like DevTools' Network panel.
- Layout & Rendering: every style recalculation, layout and paint.
- JavaScript & Events: samples of JavaScript, grouped into call trees. It's the same idea as Firefox's samples.
- CPU: how busy each thread of the page was.
- Memory: how much memory the page used. It's off at first, because it can slow the page down.
A Frames view shows the same work another way: grouped by rendering frame, with lines at 30 and 60 frames a second. Those lines sit at about 33 ms and 16 ms. A frame taller than the 60 line took longer than 16 ms.
Three more things are worth knowing:
- With Stop recording once page loads checked, a page-load recording stops by itself. It stops after 2 seconds with nothing to record, 10 seconds after the load event, or 90 seconds after it started.
- While it records, Web Inspector turns its own debugging features off, breakpoints included. The docs say this is "in order to maximize the performance of the inspected page and therefore measure realistic performance." So a breakpoint you forgot won't pause the page in the middle of a recording.
- You can export a recording to a file, and import it later, to share it with a teammate.
Safari from a script
sitespeed.io can drive Safari too, on a Mac, or on an iPhone plugged into a Mac with a USB cable. On a Mac, it misses some data. The docs: "On macOS Safari we do not support HAR, cookies and request headers." On an iPhone over USB, it collects a HAR, video and visual metrics.
The course's WebKit numbers, in the last section of this lesson, came from Playwright, a tool for steering browsers from code. Playwright's docs: "Playwright's WebKit is derived from the latest WebKit main branch sources, often before these updates are incorporated into Apple Safari and other WebKit-based browsers." And: "Playwright doesn't work with the branded version of Safari since it relies on patches." A patch is a small change to a program's source code.
So the course's WebKit 26.6 is a close relative of Safari, not Safari. For Safari's own numbers, use Safari.
Perfetto for deep traces
Fourth question: can you ask a trace a question? A trace is the kind of recording the Performance panel makes: a long list of events, each with its start and its length. In Reading a performance trace, you read one by eye: zooming, clicking, hovering. That works for one trace. It gets slow when you want the same answer from twenty.
Perfetto is an open-source tool for recording and reading traces. Chromium's developers now record most of their traces in its format. Its web page, the Perfetto UI, opens other formats too. That includes the JSON files DevTools' Performance panel saves: "Chrome JSON files can be opened directly in the Perfetto UI."
Its trace processor goes further. It reads a trace into tables, like a spreadsheet with rows and columns. The docs: the events "are parsed into Perfetto's standard SQL tables (e.g., slice, track, process, thread, counter, args)." Two words from that:
- SQL is a language for asking tables questions.
- A slice is one piece of work with a start and a length, such as one task.
It's like asking a library's catalogue for every book over 500 pages, instead of walking along every shelf. From Python, you install it with pip install perfetto.
The course traced one load of the store in Chrome, with the CPU four times slower. Then it asked the trace three questions. Here is the first, in SQL: every task of 50 ms or more.
SELECT ROUND(s.dur / 1e6, 1) AS msFROM slice sJOIN thread_track tt ON s.track_id = tt.idJOIN thread t USING (utid)WHERE t.name = 'CrRendererMain' AND s.depth = 0 AND s.dur >= 50e6ORDER BY s.ts- Line 1 picks each slice's length. Perfetto counts time in nanoseconds, billionths of a second. Dividing by
1e6, a million, turns it into milliseconds. - Line 2 reads from the
slicetable. - Lines 3 and 4 join each slice to the thread it ran on.
- Line 5 does the filtering.
CrRendererMainis Chrome's name for the page's main thread.depth = 0keeps whole tasks, not the smaller pieces inside them.50e6nanoseconds is 50 ms. - Line 6 lists them in the order they ran.
Pick a question to see its SQL, and the answer Perfetto gave:
SELECT ROUND(s.dur / 1e6, 1) AS ms FROM slice s JOIN thread_track tt ON s.track_id = tt.id JOIN thread t USING (utid) WHERE t.name = 'CrRendererMain' AND s.depth = 0 AND s.dur >= 50e6 ORDER BY s.ts
| ms |
|---|
| 238.7 |
| 99.3 |
| 150.2 |
Every whole task of 50 ms or more on the page's main thread, in the order they ran. Perfetto answered with 3 rows.
The long-task question came back with three rows: 238.7, 99.3 and 150.2 ms. The start-up work question found the store's DOMContentLoaded event: 143.3 ms, the same as the page's own timing for that load. With the CPU four times slower, that's about four times the 34 ms it took in Chrome at full speed.
One query replaced many clicks in a panel. And it runs on a hundred traces as easily as on one.
Perfetto reads the Firefox Profiler's format too. Given the course's Firefox profile, it loaded all 12,420 samples, from every thread. But not one function on them was named buildIndex. For Firefox's JavaScript, read the profile in the Firefox Profiler.
Testing in every engine
Last question: does the same code take the same time in every browser? Each browser runs JavaScript with its own JavaScript engine:
- Chrome uses V8.
- Firefox uses SpiderMonkey.
- Safari, and the WebKit engine under it, use JavaScriptCore.
Give the same recipe to three cooks in three kitchens. Each one follows it, but chops and stirs in their own style, with their own knives. One finishes in ten minutes. Another takes half an hour. To know how long your dish takes, you time it in the kitchens where it will really be cooked.
- In real life: The recipe
- In the lab: The store's app.js, the same in every browser
- In real life: Three cooks, each with their own way of working
- In the lab: Three JavaScript engines: V8, SpiderMonkey and JavaScriptCore
- In real life: Ten minutes for one, half an hour for another
- In the lab: 10 ms in Firefox, 34 ms in Chrome
- In real life: Timing the dish in the kitchens where it's cooked
- In the lab: Testing in the browsers your visitors use
The course loaded the store in three browsers on the same laptop, five times each, taking turns, with no throttling. Chrome was Google Chrome. Firefox and WebKit were Playwright's builds. Here is the start-up work in each:
// The store's start-up work in ms: five loads in each browser, taking turns, with no throttling.const startup = { Chrome: [36.5, 33.8, 34.2, 33.5, 34], Firefox: [9, 9, 10, 10, 10], WebKit: [32, 17, 18, 18, 18],};const median = (list) => [...list].sort((a, b) => a - b)[Math.floor(list.length / 2)]; for (const [browser, list] of Object.entries(startup)) { console.log(`${browser} ran it in ${median(list)} ms.`);}- Lines 3 to 5 hold the recorded times, five loads per browser, from each page's own timing.
- Line 7 finds the median, as before.
- Lines 9 to 11 print
Chrome ran it in 34 ms.,Firefox ran it in 10 ms.andWebKit ran it in 18 ms.
The same JavaScript ran more than three times as fast in Firefox as in Chrome, on the same laptop. So a fix that saves a lot in one engine can save less in another. Measure in the browsers your visitors use, not only the one you build in.
Pick a browser to see its five loads, with FCP and LCP too:
start-up work FCP LCP. The times on the right are in the same order.
Firefox 155.0 runs JavaScript with SpiderMonkey. The start-up work took a median of 10 ms. The first text or image (FCP) appeared at a median of 111 ms, and the biggest one (LCP) at 111 ms. It reports 8 kinds of performance entry. It doesn't report element, interaction-contentful-paint, layout-shift, long-animation-frame, longtask, soft-navigation, visibility-state.
Different engines report different things
Engines also differ in what they can tell you. Every browser keeps a list called PerformanceObserver.supportedEntryTypes. It names each kind of performance entry the browser can report, such as layout-shift or longtask.
// PerformanceObserver.supportedEntryTypes, as each browser reported it.const types = { Chrome: ["element", "event", "first-input", "interaction-contentful-paint", "largest-contentful-paint", "layout-shift", "long-animation-frame", "longtask", "mark", "measure", "navigation", "paint", "resource", "soft-navigation", "visibility-state"], Firefox: ["event", "first-input", "largest-contentful-paint", "mark", "measure", "navigation", "paint", "resource"], WebKit: ["event", "first-input", "largest-contentful-paint", "mark", "measure", "navigation", "paint", "resource"],};const everywhere = types.Chrome.filter((type) => types.Firefox.includes(type) && types.WebKit.includes(type));const chromeOnly = types.Chrome.filter((type) => !everywhere.includes(type)); console.log(`All three report: ${everywhere.join(", ")}.`);console.log(`Only Chrome reports: ${chromeOnly.join(", ")}.`);- Lines 3 to 5 hold each browser's recorded list.
- Line 7 keeps the types all three share. Line 8 keeps the rest of Chrome's.
- Line 10 prints
All three report: event, first-input, largest-contentful-paint, mark, measure, navigation, paint, resource. - Line 11 prints
Only Chrome reports: element, interaction-contentful-paint, layout-shift, long-animation-frame, longtask, soft-navigation, visibility-state.
Firefox and WebKit report no layout shifts, so a script can't add up CLS there. They report no long tasks either. Even paints differed. In Chrome and Firefox, LCP came at the same moment as FCP in every load. In WebKit, LCP came 13 to 44 ms after FCP. Before you compare two browsers, check that both can measure the thing you're comparing.
Only Chrome's CPU was slowed
The course slowed the CPU in Chrome only. CPU throttling is a command in the Chrome DevTools Protocol (CDP), the protocol tools use to steer Chromium-based browsers. The command is Emulation.setCPUThrottlingRate, which "Enables CPU throttling to emulate slow CPUs." And Playwright's docs say: "CDP sessions are only supported on Chromium-based browsers."
So Firefox and WebKit ran at the laptop's full speed. Compare browsers with each other at the same speed. Don't put Firefox's full-speed numbers beside Chrome's slowed-down ones.
These were lab loads, on one laptop. To see how your real visitors' browsers compare, Field data from every browser comes later in the course.
Which tool?
Here are the four tools from this lesson in one table. Read each row across: the tool, the browsers it works with, and what it's best at.
| Tool | Which browsers it works with | What it's best at |
|---|---|---|
| browsertime and sitespeed.io | Chrome, Firefox, Edge and Safari on a computer. Chrome and Firefox on Android, and Safari on iOS. | Running a page many times from a script, with statistics, a HAR and video. On a schedule, too. |
| The Firefox Profiler | Firefox. | Markers with exact times, and samples that name the JavaScript function that was running. |
| Web Inspector's Timelines tab | Safari. | Safari's own timelines: screenshots, requests, rendering, JavaScript, CPU and memory. |
| Perfetto | It reads traces from Chrome, from the Firefox Profiler, and in other formats. | Asking one trace, or a hundred, questions in SQL. |
Now sort each card by the tool it describes. Ask yourself first which question the card answers.
- Runs a page five times from one command, and writes down every request
- Times a small loop after every run, to check the machine stayed steady
- Films the screen and works out Speed Index from the frames
- Runs the same tests every hour and graphs them over months
- Notes which function is running every millisecond, in Firefox
- Starts recording the moment the browser starts, with MOZ_PROFILER_STARTUP=1
- Screenshots, Layout & Rendering, and JavaScript & Events timelines, for Safari
- Stops a page-load recording 10 seconds after the load event
- Answers questions about a Chrome trace in SQL
- Opens Chrome's JSON traces and Firefox Profiler files
Sort each card by the tool it describes.
Common mix-ups
- "A TBT of zero means no long tasks." No. browsertime recorded 138 to 409 ms of long tasks in each run, and a TBT of 0 every time. Its TBT skips any task that starts before FCP.
- "Monitoring with sitespeed.io measures my real visitors." No. A robot browser makes every visit. It's lab data, taken on a schedule.
- "Playwright's WebKit is Safari." No. It's built from WebKit's newest sources, with patches. For Safari, use Safari.
- "Every browser reports the same metrics." No. Firefox and WebKit reported no layout shifts and no long tasks.
- "CPU throttling works in every browser." No. It's a Chrome DevTools Protocol command, and Playwright opens CDP sessions in Chromium-based browsers only.
- "Any tool reads any trace just as well." No. Perfetto loaded the Firefox profile's samples, but not its JavaScript function names.
Practice exercises
Answer first, then use the hints.
If browsertime's second run had an LCP of 900 ms, what would line 6 print?
// LCP in browsertime's five runs of the store, in ms.
const lcps = [464, 900, 452, 524, 320];
const sorted = [...lcps].sort((a, b) => a - b);
const middle = sorted[Math.floor(sorted.length / 2)];
console.log(`The middle run's LCP was ${middle} ms.`);
console.log(`The runs ranged from ${sorted[0]} to ${sorted.at(-1)} ms.`);It prints The middle run's LCP was 464 ms. Sorted, the runs are 320, 452, 464, 524 and 900. The median moved one place, while the range grew a lot. That's why a median is steadier than any single run.
If Firefox and WebKit both reported layout shifts, what would line 11 print?
// PerformanceObserver.supportedEntryTypes, as each browser reported it.
const types = {
Chrome: ["element", "event", "first-input", "interaction-contentful-paint", "largest-contentful-paint", "layout-shift", "long-animation-frame", "longtask", "mark", "measure", "navigation", "paint", "resource", "soft-navigation", "visibility-state"],
Firefox: ["event", "first-input", "largest-contentful-paint", "layout-shift", "mark", "measure", "navigation", "paint", "resource"],
WebKit: ["event", "first-input", "largest-contentful-paint", "layout-shift", "mark", "measure", "navigation", "paint", "resource"],
};
const everywhere = types.Chrome.filter((type) => types.Firefox.includes(type) && types.WebKit.includes(type));
const chromeOnly = types.Chrome.filter((type) => !everywhere.includes(type));
console.log(`All three report: ${everywhere.join(", ")}.`);
console.log(`Only Chrome reports: ${chromeOnly.join(", ")}.`);It prints Only Chrome reports: element, interaction-contentful-paint, long-animation-frame, longtask, soft-navigation, visibility-state. layout-shift is now on all three lists, so line 7 keeps it and line 8 leaves it out.
In WebKit, what was the store's median LCP?
128 ms. Its FCP median was 114 ms: in WebKit, the biggest paint came a little after the first.
In the traced load, how many times did UpdateLayoutTree, the style recalculation, run on the main thread?
6 times, 98.8 ms in all. Of the five kinds of work the query adds up, only FunctionCall took more time.
Which environment variable makes Firefox list every profiler setting it accepts, and then quit?
MOZ_PROFILER_HELP. Set it to 1 and start Firefox from a terminal. Firefox quits at once, and lists every variable it accepts.
sitespeed.io's compare plugin uses one test for two independent sets of runs, and another for paired data. Which test does it use for independent runs?
Mann Whitney U. The docs: it is "used for comparing two independent samples", while the Wilcoxon test is "suitable for paired data".
Check your understanding
For each question, ask what you want to know, which tool records it, and in which browser.
Question 1 of 7What does the third line of the engines example print?
Read the code, then predict// The store's start-up work in ms: five loads in each browser, taking turns, with no throttling. const startup = { Chrome: [36.5, 33.8, 34.2, 33.5, 34], Firefox: [9, 9, 10, 10, 10], WebKit: [32, 17, 18, 18, 18], }; const median = (list) => [...list].sort((a, b) => a - b)[Math.floor(list.length / 2)]; for (const [browser, list] of Object.entries(startup)) { console.log(`${browser} ran it in ${median(list)} ms.`); }Choose an answer to see the explanation.
Question 2 of 7If the start-up work had begun at 500 ms, after FCP, what would browsertime's rule print?
Read the code, then predict// browsertime's first run, in ms from the start of the load. const fcp = 464; const startupBegan = 500; // browsertime adds a long task to TBT only if it starts after FCP. if (startupBegan > fcp) { console.log("The start-up task counts toward TBT."); } else { console.log("The start-up task began before FCP, so TBT skips it."); }Choose an answer to see the explanation.
Question 3 of 7browsertime recorded long tasks in every run of the store, yet a Total Blocking Time of zero. Why?
Choose an answer to see the explanation.
Question 4 of 7In Firefox, you want to know which JavaScript function used the time inside a slow event listener. Which tool?
Choose an answer to see the explanation.
Question 5 of 7You want to find the day a release made the store slower. What do you need?
Choose an answer to see the explanation.
Question 6 of 7Why did the course slow the CPU in Chrome only?
Choose an answer to see the explanation.
Question 7 of 7Is Playwright's WebKit the same as Safari?
Choose an answer to see the explanation.
Key takeaways
- Pick a lab tool by the question you're asking, not by the tool you know best.
- browsertime runs a page many times from one command. It writes statistics, a HAR, a CPU benchmark and, if you ask, a video.
- sitespeed.io runs those tests on a schedule and graphs them, so you can see the day a page got slower.
- Check how a tool defines a metric before you compare it with another tool's. browsertime's TBT skips tasks that start before FCP.
- The Firefox Profiler shows Firefox's work: markers with exact times, and samples that name the function running.
- Safari's Web Inspector records Safari's own timelines. Playwright's WebKit is close to Safari, but it isn't Safari.
- Perfetto turns a trace into tables, so you can ask it questions in SQL.
- Browsers run the same code at different speeds, and report different metrics. Test in the ones your visitors use.
Remember the one-liner.
Ask your question first, then pick the tool, and the browsers your visitors use.
Sources
The facts on this page come from these docs and source files:
- Introduction, browsertime docs.
- Documentation, sitespeed.io docs.
- longTaskMetrics.js, browsertime source.
- Metrics, sitespeed.io docs.
- CPU Benchmark, sitespeed.io docs.
- Video, sitespeed.io docs.
- Installation, browsertime docs.
- Continuously run your tests, sitespeed.io docs.
- Compare: statistical methods for regression analysis, sitespeed.io docs.
- Getting Started, Firefox Profiler docs.
- Profiling Firefox Startup and Shutdown, Firefox Profiler docs.
- ProfilerState.h: the profiler's features, Firefox source.
- Profiler Fundamentals, Firefox Profiler docs.
- Timelines Tab, WebKit's Web Inspector docs.
- Browsers: Safari, sitespeed.io docs.
- Browsers: WebKit, Playwright docs.
- What is Perfetto?, Perfetto docs.
- Visualizing external trace formats with Perfetto, Perfetto docs.
- Trace Processor (Python), Perfetto docs.
- SQL tables, Perfetto docs.
- What is V8?, V8.
- SpiderMonkey, Mozilla.
- JavaScriptCore, WebKit docs.
- Chrome DevTools Protocol, Chrome DevTools Protocol docs.
- Emulation.setCPUThrottlingRate, Chrome DevTools Protocol docs.
- BrowserContext.newCDPSession, Playwright docs.
Coming next: The Chrome User Experience Report, the first lesson of Measuring real users. So far, every number in the course came from a test you ran. Next, you'll see what Chrome collects from real visitors, and what it can and can't tell you.