completefrontend
Open lab My progress
THE WEB PERFORMANCE FIELD GUIDE

Monitoring and alerting

Watch real visits after every release: dashboards for engineers and for leaders, release markers, p75 alerts that ignore noise, and a synthetic check on a schedule.

By the end, you can
  • 01
    Build dashboards people readSplit field data by device and LCP element for engineers, and give leaders a weekly trend instead of a pass or fail.
  • 02
    Mark every releasePut code releases and tag manager versions on the charts, and in every beacon, so a change lines up with its cause.
  • 03
    Alert on what's realAlert on the p75, skip hours with too few visits, wait for a lasting change, and schedule a synthetic check that sees what RUM can't.

A slowdown no pull request showed

It's Friday morning at the Leaf and Kettle. Ravi opens the team's dashboard with his first cup of tea. One line on it has jumped. On phones, the Mint tea page got slower on Thursday, and it stayed slow.

That's strange. In lesson 39.2, Ravi's team set up checks that run on every pull request: each change someone asks to merge into the site's code. All week, every check passed. So what changed? And why did no check see it? Here are the two numbers that worried him:

Wednesday and Friday, on phonesPop out in the code editor (opens in a new tab)JavaScript
// p75 LCP of phone visits, in msconst wednesday = 640;const friday = 1360; const times = (friday / wednesday).toFixed(1);console.log(`Friday's p75 was ${times} times Wednesday's`);console.log(friday <= 2500 ? "Both are under 2,500 ms: good" : "Friday is over 2,500 ms");
  • Line 1 is a comment. LCP, Largest Contentful Paint, is when the biggest image or block of text appears. A millisecond (ms) is a thousandth of a second.
  • Lines 2 and 3 hold the p75 of each day's phone visits. That's the time three visits in four were at or under.
  • Line 5 divides Friday's number by Wednesday's. Line 6 prints Friday's p75 was 2.1 times Wednesday's.
  • Line 7 checks Google's line for a good LCP, 2,500 ms. It prints Both are under 2,500 ms: good.

So for phone visitors, the wait more than doubled. Yet by Google's line, the page is still good. Why it matters: a number that only says pass or fail would have said nothing was wrong.

The answer sits in the tag manager. A tag manager lets Meera add scripts to the page from a website, without changing Ravi's code. Each time she publishes, a new version goes live, as lesson 24.3 showed. On Thursday at 10:20, she published version 22. It added a newsletter pop-up. There was no pull request, so no check ran. Here is what a phone showed, on Wednesday and on Thursday:

Real frames of the Mint tea page on Asha's phone, from the hourly robot. Wednesday at 11:00: at 1,294 ms, the whole tea photo, the LCP. Thursday at 11:00: at 1,294 ms, the whole tea photo; at 2,454 ms, the pop-up over the bottom of the screen, its photo still loading; at 3,540 ms, the pop-up's whole photo, wider than the tea photo, the new LCP.Wednesday, 11:00Thursday, 11:00Wed 11:00, at 1,294 ms: the whole tea photo1,294 mstea photo: LCPThu 11:00, at 1,294 ms: the whole tea photo1,294 mstea photoThu 11:00, at 2,454 ms: the pop-up over the bottom of the screen, its photo still loading2,454 mspop-up, no photoThu 11:00, at 3,540 ms: the pop-up's whole photo, wider than the tea photo3,540 mspop-up photo: LCP
Real frames Chrome captured during two of the robot's checks. On Wednesday, the tea photo was the biggest thing on the screen. On Thursday, the tea photo came on time, then the pop-up, and its photo 2,246 ms after the tea photo. It's bigger, so it became the LCP.

On Thursday, the tea photo still came on time. Then the pop-up appeared, with a bigger photo. LCP counts the biggest thing on the screen, whoever put it there. So the pop-up's photo became the LCP, and the LCP came later. web.dev warns about the same thing with cookie notices: on phones, they cover more of the screen, and they can become the LCP.

Why it matters: a tag can change your LCP without touching your code. Only a check that watches the live page can see it. Here is the whole week:

Each day's p75 LCP, Monday to Sunday. Phones: Mon 636, Tue 644, Wed 640, Thu 1056, Fri 1360, Sat 1552, Sun 1536 ms. Computers: Mon 180, Tue 192, Wed 188, Thu 184, Fri 200, Sat 188, Sun 180 ms. Release markers: web-41, Tue 14:00 and tags v22, Thu 10:20.p75 LCP each day, in msphonescomputersweb-41, Tue 14:00tags v22, Thu 10:20636180Mon644192Tue640188Wed1,056184Thu1,360200Fri1,552188Sat1,536180Sun
Ravi's dashboard for the week, one pair of bars a day: phones shaded, computers plain. Each tick at the top marks a release, at its time. The phones' bars jump on the day of the tag manager's release. The computers' bars don't move.

Phones jumped on Thursday and stayed up. Computers didn't move. On a wide screen, the pop-up is a small card in the corner, smaller than the tea photo. And Ravi's own release on Tuesday, web-41, changed nothing.

Real-life analogyA car's gauges and its warning lights

You don't stare at the fuel gauge the whole drive. You glance at it now and then. When fuel runs low, a light comes on, and it stays on until you fill up. A light that flashed at every bump would soon be ignored. And each time the car comes back from the garage, the repair goes in the service book.

In real life: The gauges you glance at
In monitoring: A dashboard of real visits
In real life: The low fuel light
In monitoring: An alert
In real life: A light that stays on, not one that flickers at every bump
In monitoring: An alert that fires only for a lasting change
In real life: The service book
In monitoring: Release markers: what changed, and when
The main idea

Monitoring means watching real visits to the live page, all the time. Mark every release on the charts, so a change lines up with its cause. Send an alert only when a slowdown is real, lasts, and is measured on enough visits. Checks on pull requests see your code before it ships. Monitoring sees whatever reaches visitors: code, tags, servers and networks.

The rest of the lesson answers four questions:

  • Who reads a dashboard, and what should it show them?
  • How do you know which release did it?
  • When should an alert fire, and when should it stay quiet?
  • How often should a robot check the page, and what can't it see?

How the course recorded it

  • The week: one week of the live page, squeezed into one run in Chrome 157. The course made every visit, in order: 1,386 of them, few at night and most in the evening. Each phone and computer had its own speed and network, drawn the way lesson 13.1 drew them.
  • The visitors: about one in three was new. The rest were regulars who already get the newsletter, so their browser had the shop's newsletter cookie. LCP stops counting at a visitor's first tap or scroll, as web.dev explains. So each visitor waited until the page had settled, then followed a link.
  • The page: lesson 39.2's Mint tea page, with the tag manager's script. Its rum.js sent each visit's numbers home, as in lesson 12.4.
  • The sites: the shop, the tag manager and the pop-up each had a made-up address, served over HTTPS from the course's computer. The tag manager and the pop-up are stand-ins. The pop-up is the size of lesson 24.3's pop-up, and its photo is 92 kB.
  • The robot: every hour, a fresh Chrome loaded the page as Asha's phone: Slow 4G, the CPU 4 times slower, and no cookies.

Dashboards for engineers and for leaders

A dashboard is a page of charts built from the beacons: the small messages each visit sends home with its numbers. Lesson 12.4 built one to answer a question. web.dev's guide to measuring adds one more thought: think about who needs to see the data. At the Leaf and Kettle, two kinds of people do.

Real-life analogyThe kitchen's order screen and the owner's weekly sheet

In a café kitchen, the cooks watch a screen of live orders: what's waiting, for how long, and for which table. The owner doesn't. Once a week, she reads one sheet: sales, waiting times, and how they compare with last week. Same café, same orders, two very different views.

In real life: The cooks' live screen
In monitoring: The engineers' dashboard: daily and hourly, split many ways
In real life: Which table each order is for
In monitoring: Splits: device, LCP element, release
In real life: The owner's weekly sheet
In monitoring: The leaders' report: a few numbers, once a week
In real life: "Compared with last week"
In monitoring: The trend, not just pass or fail

What engineers need

Engineers need to find the cause. So their dashboard splits the numbers. Here are the phone visits after Meera's release, split by the element their LCP came from:

Which element was the LCP?Pop out in the code editor (opens in a new tab)JavaScript
// Phone visits from Thursday 10:20 on, by the LCP element each beacon namedconst elements = {  "html>body>main>img.hero": 387,  "html>body>aside.offer>img.offer-photo": 207,}; for (const [element, visits] of Object.entries(elements)) {  console.log(`${element}: ${visits} visits`);}
  • Lines 3 and 4 hold the counts. Each name is a selector: a CSS-style address of the element, from the page down to the image. Ravi's beacon sends it, as the next section shows.
  • Lines 7 to 9 print one line for each element: html>body>main>img.hero: 387 visits and html>body>aside.offer>img.offer-photo: 207 visits.

207 of 594 phone visits named the pop-up's photo: the new visitors. Regulars never saw the pop-up. Why it matters: the element tells Ravi where to look before he opens a single trace. Now split the whole week yourself. Each row shows the p75 before and after Meera's release:

Playground: one week, split three ways
p75 LCP, before and after the release
  1. Every visitbefore 564 ms (647 visits), after 1,208 ms (739 visits)
Try it yourself
Everyone

One number for every visit, phones and computers together. Each row shows the p75 LCP before Thu 10:20 and after it.

From the course's recording of one week in Chrome 157. The release split is Meera's tag manager version, Thu 10:20.

Every visit together still shows the jump, because most visits come from phones. By device, you see that only phones changed. By LCP element, the cause has a name. Next to each number, show how many visits it came from. As lesson 13.1 showed, a p75 of a few visits is little more than a guess.

What leaders need

Leaders don't need hourly charts. They need to know if the site is getting faster or slower, and why. web.dev's article on fixing site speed across a company says it plainly: "what's not reported will not be focused on." It suggests putting speed in the weekly report to the business. Read this table across: each row asks one question of both dashboards.

Two dashboards from the same beacons
QuestionEngineers' dashboardLeaders' report
Who reads it?Ravi, Ben, and anyone fixing the page.The shop's owners, Meera, and anyone deciding what to work on.
How often?Every day, and every hour after a change.Once a week.
Which numbers?p75 of LCP, INP and CLS, with the visits behind each one.A few: phones' p75 LCP this week, last week, and the change.
Split how?By device, LCP element and release.By device at most.
What is it for?Finding the cause of a change.Seeing the trend, and deciding where to spend time.

One trap for leaders is pass or fail. Ravi's phones went from 640 to 1,360 ms, and stayed good. A report that only said "good" would have shown nothing. So give leaders the trend: this week, last week, and the change.

Google's numbers move slowly

Many leaders look at Google's numbers, in Search Console and PageSpeed Insights. Both come from CrUX, the data Chrome collects from real visits, from lesson 10.1. And both cover the past 28 days. The CrUX History API keeps one point a week. It's updated each Monday, with data up to the Saturday before. Here is what those points would show for Ravi's phones:

CrUX's weekly points, worked out from the weekPop out in the code editor (opens in a new tab)JavaScript
// Phones' p75 LCP over 28 days, at each weekly point, in msconst points = [  { newDays: 0, p75: 640 },  { newDays: 2.6, p75: 664 },  { newDays: 9.6, p75: 760 },  { newDays: 16.6, p75: 1008 },  { newDays: 23.6, p75: 1276 },  { newDays: 28, p75: 1428 },]; for (const { newDays, p75 } of points) {  console.log(`${newDays} of 28 days after the release: ${p75} ms`);}
  • Lines 3 to 8 hold one point a week. newDays is how many of its 28 days came after the release. Each day before it counts like an average day of the week before 10:20 on Thursday, and each day after it like an average day after.
  • Line 3 is the Saturday before the release. Line 4 is the first Saturday after it: only 2.6 of its 28 days are new.
  • Lines 11 to 13 print one line a point, from 0 of 28 days after the release: 640 ms to 28 of 28 days after the release: 1428 ms.
Phones' p75 LCP over 28 days, at each weekly point: 0 of 28 days new, 640 ms; 2.6 of 28 days new, 664 ms; 9.6 of 28 days new, 760 ms; 16.6 of 28 days new, 1008 ms; 23.6 of 28 days new, 1276 ms; 28 of 28 days new, 1428 ms. Ravi's own dashboard showed 1360 ms for Friday alone.p75 LCP of phones, in ms64006642.67609.61,00816.61,27623.61,42828days after the release, out of 281,360Fridayown data
Each shaded bar is one weekly point, worked out from the week's visits: 28 days at a time, with more of them after the release each week. The plain bar is Ravi's own number for Friday. Every bar is under 2,500 ms, Google's line for a good LCP.

The first point after the release barely moves. It takes four more weeks to show the whole change. Why it matters: CrUX tells you how the month went. To catch a slowdown on the day, you need your own field data. web.dev recommends it for exactly this: "more detailed and immediate feedback".

Marking every release

On Friday, Ravi's first question was one the SRE book, Google's book on running services, lists: "what else happened around the same time?" A release marker answers it. It's a line or a tick on a chart, at the moment something changed.

Real-life analogyA gardener's diary

Ana keeps a garden diary. The day she switched to a new plant food, she wrote it down. Weeks later, the tomato leaves turn yellow. She flips back, finds the note, and knows where to start. Without it, she'd be guessing between the weather, the soil and the seeds.

In real life: A note in the diary
In monitoring: A release marker on the chart
In real life: Switching to a new plant food
In monitoring: A release: code, tags or settings
In real life: Yellow leaves, weeks later
In monitoring: A slower p75 on the dashboard
In real life: Flipping back to the note
In monitoring: Lining the change up with its marker

What counts as a release? The SRE book calls a change a push: "Any change to a service's running software or its configuration." That's more than Ravi's code:

  • Code: mark each deploy. SpeedCurve, a monitoring service, can test the site each time you add a deploy, from your CI workflow or by hand. Sentry, an error and performance tool, ties each event to the release name sent with it.
  • Tags: Google Tag Manager keeps a publish history, "so you can see when versions were live and who published them."
  • Everything else: a new setting on the CDN, the servers that deliver your files, or a new image format. DebugBear, another monitoring service, suggests a chart annotation for changes like these.

Ravi's week has two markers: web-41 on Tuesday at 14:00, and tags version 22 on Thursday at 10:20. Only the second lines up with the jump.

Put the release in every beacon

A marker says when. Ravi also wants each visit to say what it ran. So his rum.js now sends that with every beacon:

Ravi's rum.js, with the release and the tags versionJavaScript
import { onCLS, onINP, onLCP } from "web-vitals/attribution";import { RELEASE } from "./release.js"; const metrics = []; export function startRum() {  onLCP((metric) => metrics.push({ name: "LCP", value: metric.value, element: metric.attribution.target }));  onINP((metric) => metrics.push({ name: "INP", value: metric.value }));  onCLS((metric) => metrics.push({ name: "CLS", value: metric.value }));  addEventListener("visibilitychange", () => {    if (document.visibilityState !== "hidden" || metrics.length === 0) return;    const beacon = {      release: RELEASE,      tags: window.tagManager?.version ?? null,      device: matchMedia("(max-width: 600px)").matches ? "phone" : "desktop",      metrics: metrics.splice(0),    };    navigator.sendBeacon(`rum${location.search}`, JSON.stringify(beacon));  });}
  • Line 1 imports from web-vitals' attribution build: a slightly bigger version of the library whose metrics also say why, in an attribution object.
  • Line 2 imports the release's name. The build writes it into release.js: here, web-41.
  • Line 7 keeps LCP's value and its element, metric.attribution.target: the selector you saw above.
  • Lines 12 to 17 build the beacon. Line 13 adds the release, line 14 the tag manager's version, and line 15 the device. The stand-in tag manager puts its version in window.tagManager. Google Tag Manager has a built-in variable, Container Version, for the same job.
  • Line 18 sends it with navigator.sendBeacon() when the page is hidden, as in lesson 12.4.

Now the visits can be grouped by what was live:

p75 by what was livePop out in the code editor (opens in a new tab)JavaScript
// p75 LCP of phone visits, by what was live, in msconst live = [  { release: "web-40", tags: 21, p75: 664 },  { release: "web-41", tags: 21, p75: 612 },  { release: "web-41", tags: 22, p75: 1428 },]; for (const { release, tags, p75 } of live) {  console.log(`${release} with tags ${tags}: ${p75} ms`);}
  • Lines 3 to 5 hold the p75 for each mix of release and tags version the week had.
  • Lines 8 to 10 print web-40 with tags 21: 664 ms, web-41 with tags 21: 612 ms and web-41 with tags 22: 1428 ms.

Ravi's own release changed nothing. The tags version did. Why it matters: with the release in every beacon, "which change was it?" takes one split, not an afternoon of guessing.

Alert thresholds on percentiles

The dashboard showed the jump on Friday. Ravi wants to hear about it on Thursday. And nobody should sit and watch charts: the SRE book says its teams avoid any job that needs someone to "stare at a screen to watch for problems." That's what an alert is for. It's a rule that runs every hour on the new beacons, and sends someone a message when it fires.

Lesson 12.4 wrote a rule that compared two releases. This one runs on a clock. It needs three choices:

  • which number to watch;
  • how many visits an hour needs before the rule reads it;
  • how long a slowdown must last before anyone is told.
Real-life analogyTaking a child's temperature

A parent takes a child's temperature right after a game of tag. It's a little high, but running about warms anyone up. So they let the child rest, and take it again an hour later. Only two high readings in a row, taken at rest, make them call the doctor.

In real life: One reading
In monitoring: One hour's p75
In real life: Right after a game of tag
In monitoring: An hour with only two or three visits
In real life: Waiting for a reading at rest
In monitoring: Skipping hours with too few visits
In real life: Two high readings in a row
In monitoring: Two hours in a row over the line
In real life: Calling the doctor
In monitoring: Sending the alert

Which number?

Here is the same hour, 11:00 to 12:00, on Wednesday and on Thursday. Thursday's is the first whole hour after the release:

Mean, p50 and p75 of one hourPop out in the code editor (opens in a new tab)JavaScript
// The phone visits between 11:00 and 12:00, in msconst wednesday = [1232, 704, 244, 696, 296, 724, 1060, 276];const thursday = [608, 728, 260, 3172, 1340, 252, 212, 1448]; const mean = (lcps) => Math.round(lcps.reduce((sum, lcp) => sum + lcp, 0) / lcps.length);const rank = (lcps, share) => [...lcps].sort((a, b) => a - b)[Math.ceil(lcps.length * share) - 1];for (const [day, lcps] of Object.entries({ wednesday, thursday })) {  console.log(`${day}: mean ${mean(lcps)}, p50 ${rank(lcps, 0.5)}, p75 ${rank(lcps, 0.75)}`);}
  • Lines 2 and 3 hold the LCPs of that hour's phone visits, in the order they came.
  • Line 5 works out the mean, the average. Line 6 sorts a copy and picks a rank, as lesson 13.1 counted percentiles. The p50 is the middle one. The p75 is the one three quarters of the way up.
  • It prints wednesday: mean 654, p50 696, p75 724 and thursday: mean 1003, p50 608, p75 1340.

In this hour, the p50 even went down. With 8 visits, the middle ones were regulars, who never saw the pop-up. The p75 jumped: 3 of the 8 visits saw the pop-up, and the p75 sits among the slowest quarter. The mean rose too, mostly because one visit took 3,172 ms.

The whole week shows the same pattern. After the release, phones' p50 went from 432 to 668 ms. The p75 went from 640 to 1,428 ms.

The SRE book warns that "a simple average can obscure these tail latencies, as well as changes in them." It says the median, the p50, "emphasizes the typical case." web.dev adds that the median "can mask" slow visits. That's why Core Web Vitals, Google's main speed numbers, are judged at the 75th percentile. Why it matters: a slowdown that hits a third of your visitors shows clearly in the p75.

Where to draw the line

Ravi uses lesson 39.1's rule again: the slowest value he has, rounded up to the next 100 ms. He takes the hours from Monday to Wednesday with at least 8 phone visits. Their slowest p75 gives a line of 900 ms. The same rule gives 800 ms for the mean and 700 ms for the p50.

Too few visits

Now the quiet hours. At night, only two or three phones visit. Here is one of those hours:

A quiet hour, before the releasePop out in the code editor (opens in a new tab)JavaScript
// The 2 phone visits between 01:00 and 02:00 on Thursday, in msconst lcps = [1884, 412];const line = 900; const p75 = [...lcps].sort((a, b) => a - b)[Math.ceil(lcps.length * 0.75) - 1];console.log(`p75 of ${lcps.length} visits: ${p75} ms`);console.log(p75 > line ? "Over the line: alert!" : "Under the line");
  • Line 2 holds the visits. Line 3 is Ravi's p75 line.
  • Line 5 picks the p75. With 2 visits, that's rank 2 of 2: the slower one.
  • It prints p75 of 2 visits: 1884 ms and Over the line: alert!

One slow phone in the middle of the night, and the rule fires. Nothing had changed on the page. A p75 of a few visits is mostly luck, as lesson 13.1 showed. Google's SRE workbook says alerting gets "particularly sensitive to nonsignificant events during low-traffic periods." So Ravi's rule skips any hour with fewer than 8 phone visits.

Two hours in a row

One more guard. A single bad hour can be bad luck too, such as a few slow phones at once. So the rule waits for two hours in a row over the line. Prometheus, an open-source monitoring tool, does this with a for setting. Until the time is up, its alert is "pending". SpeedCurve can delay an alert "until a threshold is crossed multiple times."

Ravi's rule, on ThursdayPop out in the code editor (opens in a new tab)JavaScript
// Thursday, hour by hour: phone visits, and their p75 in msconst hours = [  { until: "09:00", visits: 8, p75: 532 },  { until: "10:00", visits: 8, p75: 400 },  { until: "11:00", visits: 8, p75: 816 },  { until: "12:00", visits: 8, p75: 1340 },  { until: "13:00", visits: 10, p75: 1036 },  { until: "14:00", visits: 8, p75: 1244 },];const line = 900; let inARow = 0;for (const hour of hours) {  if (hour.visits < 8) continue;  inARow = hour.p75 > line ? inARow + 1 : 0;  if (inARow === 2) console.log(`Alert at ${hour.until}: two hours in a row over ${line} ms`);}
  • Lines 3 to 8 hold Thursday's hours: when each one ended, its phone visits, and their p75.
  • Line 10 is the line. Line 12 counts how many hours in a row were over it.
  • Line 14 skips an hour with fewer than 8 visits. Line 15 adds one when the p75 is over the line, or starts again from 0.
  • Line 16 prints Alert at 13:00: two hours in a row over 900 ms.

Meera published at 10:20. The alert went out at 13:00, under three hours later. Why it matters: Ravi heard about it on Thursday, not on Friday morning.

Waiting has a price, and Google's SRE workbook spells it out. The alert comes later. And if the value dips under the line for one hour, the count starts again, so a slowdown that flickers might never alert. The workbook prefers a longer window instead, such as the last few hours read together. Both ideas do the same job: they put more visits behind each decision.

Now try five rules on the whole week. Each square is one hour of phone visits:

Playground: five alert rules, one week
Mean, every hour
Mon
Tue
Wed
Thu
Fri
Sat
Sun
  1. Linemean over 800 ms
  2. False alarms, before the release5: Mon 05:00, Tue 00:00, Tue 02:00, Wed 02:00, Thu 02:00
  3. First alert after the releaseThu 12:00
  4. Alerts sent after the release9
Try it yourself
Noise before the release

Each square is one hour of phone visits, Monday at the top. Filled squares were over the line and faint ones under it. An alert is sent when an hour is over, and stays on until an hour is under. A red ring marks each alert, and the dashed square is the hour of Meera's release.

Phone visits from the course's recorded week in Chrome 157. Meera's release went live Thu 10:20.

Watch two things: how many alerts each rule sends before the release, when nothing had changed, and when it first fires after it. One more setting matters. SpeedCurve sends an alert only when a budget's status changes: "Repeated overages do not trigger repeated alerts." Ravi's rule works the same way: one message when a slowdown starts, not one every hour. Why it matters: the SRE book warns that when alerts come too often, people "second-guess, skim, or even ignore" them.

Synthetic monitoring on a schedule

Ravi's alert needs real visits, and at 3 in the morning there are hardly any. So he also points a robot at the live page. In lesson 39.2, a synthetic test was a scripted visit in a lab, on every pull request. A synthetic monitor is the same kind of visit, on a timer, against the live site. Ravi's robot checks the page every hour, as Asha's phone, with no cookies.

Real-life analogyA mystery shopper

A chain of cafés sends a mystery shopper to each shop at set times. She orders the same tea every visit and times the queue. If the queue gets longer, the owner knows by the next visit. But she's one person ordering one drink. She never sees how the regulars are treated.

In real life: The mystery shopper
In monitoring: The robot: one scripted visit
In real life: The same order every time
In monitoring: The same phone and network, and no cookies
In real life: Her set times
In monitoring: The schedule
In real life: The regulars she never sees
In monitoring: Real visitors the robot can't stand in for

The robot's line comes from the same rule: its slowest check from Monday to Wednesday, rounded up, 1,400 ms. Those 72 checks ranged from 1,288 to 1,340 ms, much closer together than real visits. Lab loads still wobble, as lesson 39.2 found. On Thursday at 07:00, one check took 1,408 ms, just over the line, with nothing changed. The next one took 1,312 ms. So the robot gets Ravi's rule too: two checks in a row over the line.

On Wednesday at 11:00, the robot's LCP was 1,296 ms. Thursday at 11:00 was its first check after Meera's release: 3,540 ms, with the pop-up's photo as the LCP. Every check after it took 3,524 to 3,696 ms. So the robot's alert went out at 12:00, an hour before the alert on real visits.

Why it matters: the robot doesn't wait for visitors. It checks the same way every time, so its numbers stay close together, and two checks over the line mean something.

How often?

More checks cost more. DebugBear's docs say once a day is enough for most sites. They suggest more often if you deploy several times a day, so you're "alerted to regressions more quickly", or if your numbers wobble a lot. SpeedCurve's docs give an example: your own site 3 times a day, and a competitor's once, to keep your monthly checks down.

Checks a week, and the wait after Thursday 10:20Pop out in the code editor (opens in a new tab)JavaScript
const release = 10 + 20 / 60; // Thursday 10:20, in hours for (const every of [1, 6, 24]) {  const next = Math.ceil(release / every) * every;  console.log(`Every ${every} h: ${168 / every} checks a week, the first ${(next - release).toFixed(1)} h after the release`);}
  • Line 1 is Thursday 10:20, in hours since midnight.
  • Line 4 rounds up to the next check. With a check every 6 hours, that's 12:00.
  • Line 5 prints one line for each schedule. The first is Every 1 h: 168 checks a week, the first 0.7 h after the release. Every 6 hours, the first check comes 1.7 h after it. Once a day, 13.7 h after.

Pick a schedule, and see which of the robot's real checks it keeps:

Playground: how often the robot checks
Every hour
  1. Checks a week168
  2. Last check before the releaseThu 10:00: LCP 1,296 ms
  3. First check after itThu 11:00: LCP 3,540 ms, the pop-up's photo
  4. Time from release to that check0 h 40 min
  5. Checks over the line before the release1: Thu 07:00
  6. Alert, at the second check in a row over the lineThu 12:00
Try it yourself
168 checks a week

The robot's line is 1,400 ms. Fewer checks cost less, and the first one after Meera's release comes later.

The robot's real hourly checks in Chrome 157; a slower schedule keeps the ones on its hours, from Monday 00:00.

You can also check right after a change. SpeedCurve can run its tests each time you add a deploy, from your CI workflow or by hand. But Meera's publish wasn't a deploy, so only the schedule caught it.

What the robot can't see

The robot found the pop-up within the hour. The field data, from real visits, told Ravi the rest:

  • Who was hit. The robot is always a new visitor. In the field, 207 of 594 phone visits had the pop-up's photo as their LCP. The regulars had the cookie.
  • How badly. The robot has one phone on one network. In the field, one new visitor in ten waited over 2,716 ms.
  • Computers. The robot only checks a phone. Only the field could say computers didn't change.
  • Taps. The robot never taps, so it has no INP, the time the page takes to answer a tap. web.dev notes that some lab tools "won't report a page's INP because they only observe the loading of a page without any interactions."

Google's SRE workbook warns about the other side too. If a problem hits real users but not the robot, "the successful artificial requests hide the real user signal." A/B tests, which show different visitors different versions, add noise too. SpeedCurve warns that its robots "end up in random test buckets." Read this table across to compare the robot with real-user monitoring (RUM):

The hourly robot and real-user monitoring
QuestionThe hourly robotReal-user monitoring
Who visits?One made-up visitor: Asha's phone, on Slow 4G, every time.Every real visitor, on their own phone or computer and network.
Cookies?None. It's a new visitor on every check.Whatever each visitor has. Regulars carry the newsletter cookie.
When?On its schedule, even at 3 in the morning with nobody there.Only when people visit. Quiet hours have few visits.
Does it tap or scroll?No. It loads the page and waits, so it has no INP.Yes. Real taps give each visit its INP.
How soon does it notice?At its next check: here, 40 minutes after the release.Once enough visits pile up: here, under three hours.
What can't it tell you?How many real visitors were hit, and how badly.Why: it needs the release markers and the LCP element to say.

Why it matters: use both. The robot is quick, steady, and awake at night. The field says who was hit, and how badly.

Which check sees it first?

Ravi now has three ways to catch a slowdown: the checks on each pull request, the hourly robot, and alerts on real visits. Sort each problem by the one that would see it first:

Which check sees it first?
  • A pull request adds a big date library to app.js
  • A pull request adds loading="lazy" to the tea photo
  • Meera publishes a pop-up for new visitors from the tag manager
  • The shop's server gets slow at 3 in the morning
  • A tag slows the page only for people with the newsletter cookie
  • Taps on the brewing timer get slow on old phones
  • The page gets slow on one mobile network in one country
Try it yourself
0 of 7 correct

Sort each problem by the check that would notice it first.

Choose a category for every card. You can change an answer at any time; Reset clears them all.

Common mix-ups

  • "If every pull request passes its checks, the live site can't get slower." No. Meera's pop-up went live from the tag manager, with no pull request. Phones' p75 more than doubled.
  • "The average shows every slowdown." No. A mean moves for one very slow visit as easily as for many, so it fires on noise. And the SRE book warns it can hide changes in the slowest visits.
  • "The p50 is the safest number to alert on." No. It shows the typical visit. In the hour after the release, it went down while a third of the visits got slower.
  • "Every hour's p75 is worth an alert." No. With 2 visits, the p75 is the slower of the two. Skip hours with too few visits.
  • "If Search Console says good, nothing got slower." No. It covers 28 days, and its line for good is 2,500 ms. Ravi's phones stayed good while their p75 more than doubled.
  • "A synthetic monitor sees what visitors see." No. It's one visitor, on one phone, with no cookies. It can't say how many real visitors were hit.
  • "More alerts keep you safer." No. When alerts come too often, people learn to ignore them.

Practice exercises

Answer first, then use the hints.

Exercise 1 · Warm-upThree hours in a row

Ravi tries waiting for three hours in a row. What does it print?

Starter codePop out in the code editor (opens in a new tab)JavaScript
// Thursday, hour by hour: phone visits, and their p75 in ms
const hours = [
  { until: "09:00", visits: 8, p75: 532 },
  { until: "10:00", visits: 8, p75: 400 },
  { until: "11:00", visits: 8, p75: 816 },
  { until: "12:00", visits: 8, p75: 1340 },
  { until: "13:00", visits: 10, p75: 1036 },
  { until: "14:00", visits: 8, p75: 1244 },
];
const line = 900;

let inARow = 0;
for (const hour of hours) {
  if (hour.visits < 8) continue;
  inARow = hour.p75 > line ? inARow + 1 : 0;
  if (inARow === 3) console.log(`Alert at ${hour.until}: three hours in a row over ${line} ms`);
}

Answer, then press Check. Spacing and letter case don’t matter.

    Exercise 2 · Warm-upA quiet hour with three visits

    Another night hour had three phone visits. What does the first line print?

    Starter codePop out in the code editor (opens in a new tab)JavaScript
    // The 3 phone visits between 05:00 and 06:00 on Monday, in ms
    const lcps = [320, 348, 952];
    const line = 900;
    
    const p75 = [...lcps].sort((a, b) => a - b)[Math.ceil(lcps.length * 0.75) - 1];
    console.log(`p75 of ${lcps.length} visits: ${p75} ms`);
    console.log(p75 > line ? "Over the line: alert!" : "Under the line");

    Answer, then press Check. Spacing and letter case don’t matter.

      Exercise 3 · PracticeThe pop-up's own p75

      After the release, what was the p75 LCP of the visits whose LCP was the pop-up's photo, in ms?

      Answer, then press Check. Spacing and letter case don’t matter.

        Exercise 4 · PracticeEvery hour, no matter how quiet

        Read every hour's p75, however few visits it had. How many alerts go out before the release?

        Answer, then press Check. Spacing and letter case don’t matter.

          Exercise 5 · ChallengeThe LCP element in DevTools

          Open Thursday's Mint tea page in Chrome, on a phone-sized screen. Which element is the LCP?

          Answer, then press Check. Spacing and letter case don’t matter.

            Exercise 6 · ChallengeStop the flapping

            Prometheus has a second setting next to for. It keeps an alert firing for a while after the condition stops, so it doesn't switch on and off. What is it called?

            Answer, then press Check. Spacing and letter case don’t matter.

              Check your understanding

              For each question, ask: what reached the visitors, which number shows it, and how many visits stand behind that number?

              The monitoring and alerting quiz · 7 questionsScore: first tries count
              1. Question 1 of 7Meera's pop-up reached visitors on Thursday, and every check on every pull request had passed. Why did no check stop it?

                Choose an answer to see the explanation.

              2. Question 2 of 7After the release, a third of phone visits got much slower and the rest didn't change. Which hourly number showed it most clearly?

                Choose an answer to see the explanation.

              3. Question 3 of 7What does this code print, the p75 of four visits by nearest rank?

                Read the code, then predictPop out in the code editor (opens in a new tab)JavaScript
                const lcps = [380, 1460, 420, 510];
                const sorted = [...lcps].sort((a, b) => a - b);
                
                console.log(sorted[Math.ceil(lcps.length * 0.75) - 1]);

                Choose an answer to see the explanation.

              4. Question 4 of 7Why does Ravi's alert skip hours with fewer than 8 phone visits?

                Choose an answer to see the explanation.

              5. Question 5 of 7Ravi's rule waits for two hours in a row over the line. What does that cost?

                Choose an answer to see the explanation.

              6. Question 6 of 7Search Console showed Ravi's phone LCP as good all week. What does that say about Thursday's change?

                Choose an answer to see the explanation.

              7. Question 7 of 7The hourly robot caught the pop-up at its first check after the release. What could it not tell Ravi?

                Choose an answer to see the explanation.

              Key takeaways

              • A slowdown can reach visitors without a pull request. Meera's tag manager release did, so only checks on the live page saw it.
              • Engineers need splits, by device, LCP element and release, with the visits behind each number. Leaders need a weekly trend, not pass or fail.
              • Mark every push on the charts, code and tags alike, and send the release and the tags version in every beacon.
              • Alert on the p75, skip hours with too few visits, wait for a lasting change, and send one message per problem.
              • CrUX covers 28 days, so it shows a change weeks late.
              • A synthetic monitor is quick, steady and awake at night. Real-user data says who was hit, and how badly. Use both.

              That's Module 39. Lesson 39.1 drew the lines: budgets from real data. Lesson 39.2 checked every pull request against them, with room for the wobble. This lesson watches what really reaches visitors, whatever path it took.

              Remember the one-liner.
              Watch real visits, mark every release, and alert only on slowdowns that are real, lasting, and measured on enough visits.

              Sources

              The facts on this page come from these docs and books:

              Coming next: Running a performance audit. You'll start from field data like Ravi's, pick the pages and visitors that matter, and turn what you find into a ranked list of fixes.

              CompleteFrontend Clear concepts. Working examples.