Reading performance data

One day of real visits

Picture the Leaf and Kettle team on a Monday morning. They open the dashboard for the mint tea page. It shows one number for each hour of the day before: the p75 LCP.

That number has two parts, both from earlier lessons:

• LCP, Largest Contentful Paint, is when the biggest image or block of text appeared. On this page, it's the photo of mint tea.
• The p75, the 75th percentile, is a time that three visits in four matched or beat. Building a RUM pipeline found it by lining the visits up, quickest first.

Here are three of the hours:

1// p75 LCP for three hours of the day, in ms. The page never changed.2const p75 = { "11:00": 232, "22:00": 920, "23:00": 640 };3 4console.log(`From 11:00 to 22:00, p75 rose by ${p75["22:00"] - p75["11:00"]} ms.`);5console.log(`From 22:00 to 23:00, it fell by ${p75["22:00"] - p75["23:00"]} ms.`);

• Line 1 is a comment. A millisecond (ms) is a thousandth of a second.
• Line 2 holds the p75 for three hours: the hours that start at 11:00, at 22:00 and at 23:00.
• Line 4 prints From 11:00 to 22:00, p75 rose by 688 ms.
• Line 5 prints From 22:00 to 23:00, it fell by 280 ms.

So did the page break at ten at night, and mend itself by eleven? Nobody shipped anything all day. Here is every hour at once:

You can know the answer here, because the course built this day on purpose. It made 384 visits to a copy of the page in Chrome: 24 hours of 16 visits each. Each visit drew its own CPU speed and network at random, set through Chrome's DevTools protocol. Only one thing followed a plan: how many visits in each hour came from phones. It went from 4 in 16 around midday to 12 in 16 at night. The page itself never changed.

So nothing broke. Two other things moved the number:

• The mix. At 11:00, 4 of the 16 visits came from phones. At 22:00, 12 did. Phones took longer to show the photo: over the whole day, phones had a p75 of 932 ms and desktops 220 ms. So an hour with more phones had a slower p75.
• Chance. 22:00 and 23:00 both had 12 phones. Same page, same mix. But each hour drew different visitors, on different networks, and their p75s landed 280 ms apart.

The rest of this lesson takes the day apart in four steps:

• Look at the shape of all 384 visits, with a histogram and a cumulative chart.
• Measure how much one hour's p75 wobbles, and put an error bar on it.
• Work out how many visits make the error bar small enough.
• See how the mix of visitors moves p75 through the day.

Histograms and cumulative charts

Before you trust one number, look at what it sums up: all the visits.

A histogram does that. It splits the times into equal ranges, called bins, such as 0 to 249 ms and 250 to 499 ms. It counts the visits in each bin, and draws a bar for each count. NIST's handbook of statistics says a histogram is there "to graphically summarize the distribution of a univariate data set." In plain words: to show the shape of one list of numbers.

Here is the whole day as a histogram. Each row is one bin, and the last row holds everything from 2,000 ms up. The buttons switch to one hour or the evening, and to a second kind of chart you'll meet in a moment:

Most visits bunch up in the first rows: 148 of the 384 took under 250 ms. Then the bars shrink, row by row. A few visits trail far behind. The slowest took 4,256 ms, more than 50 times the quickest, which took 80 ms.

That trailing end is called a long tail. Draw time from left to right, and the tail is on the right. NIST calls this shape skewed right: "the tail is on the right side." It also says why times often look like this: "data that have a lower bound are often skewed right." A visit can't take less than nothing. But a slow network or a busy phone can make it take many times longer than usual.

The day's visits behave the same way:

• The mean, every LCP added up and divided by 384, was 513 ms.
• The median, the middle visit, was 364 ms.
• The p75 was 704 ms.

The mean was slower than 233 of the 384 visits. The tail pulled it up, like Asha's missed trains. NIST says that for skewed data, the mean and the median "are markedly different". Why it matters: one very slow visit can move the mean a lot, and the p75 by one place in line at most. That's why this course reads field data with percentiles.

Adding the bins up

A histogram answers "what does the day look like?". It's less handy for another question: "what share of visits were quicker than this?" For that, you add the bins up as you go.

Asha could ask: on how many days was I at work within 30 minutes? She'd count the days under 10 minutes, then add the days under 20, then the days under 30.

NIST calls this a cumulative histogram. Each bar holds the count "for that bin plus all bins for smaller values". Divide each count by the total, and you get a share. Here it is with the day's real bins:

1// The day's 384 LCPs, counted in 250 ms bins: under 250 ms, 250 to 499 ms, and so on.2const counts = [148, 80, 73, 39, 23, 12, 2, 3, 4];3 4let below = 0;5for (const [bin, count] of counts.entries()) {6  below += count;7  console.log(`Under ${(bin + 1) * 250} ms: ${Math.round((below / 384) * 100)}% of visits`);8  if (below / 384 >= 0.75) break;9}

• Line 2 is recorded: the day's 384 LCPs, counted into 250 ms bins. 148 visits took under 250 ms, 80 took 250 to 499 ms, and so on.
• Line 4 starts a running count at 0. Line 6 adds each bin to it.
• Line 7 prints the running count as a share of all 384 visits.
• Line 8 stops once the share reaches 75%.
• It prints Under 250 ms: 39% of visits, Under 500 ms: 59% of visits and Under 750 ms: 78% of visits.

The share passed 75% inside the bin from 500 to 749 ms, so that's where the p75 is. The bins can't say exactly where, because they only count.

A cumulative chart also works the other way round. Pick a time, and read the share of visits under it:

1// The same counts. The last bin holds every visit of 2,000 ms or more.2const counts = [148, 80, 73, 39, 23, 12, 2, 3, 4];3 4const slowest = counts.at(-1);5console.log(`${slowest} visits took 2,000 ms or more.`);6console.log(`${Math.round(((384 - slowest) / 384) * 100)}% were under 2,000 ms.`);

• Line 2 holds the same counts. The last one is the bin for 2,000 ms or more.
• Line 4 takes that last count with counts.at(-1): the item one from the end.
• Line 5 prints 4 visits took 2,000 ms or more. Line 6 prints 99% were under 2,000 ms.

Now drop the bins and use every visit. Sort all 384 LCPs. For each one, mark the share of visits at or below it. The marks join up into a curve that climbs from 0% to 100%:

Across from 75%, the curve sits at 704 ms: the day's exact p75. Press Cumulative in the playground above to see the same idea in bins. Each row shows the share of visits below its end, and the highlighted row holds the p75.

CrUX, Chrome's public field data, keeps its numbers this way too. Its methodology says: "Most metric values within the CrUX dataset are represented as histograms of values and bin sizes". Why it matters: from bins you can work out any share yourself, and you can see the tail that one number hides.

Percentiles and their error bars

Back to the team's question. The p75 fell from 920 ms at 22:00 to 640 ms at 23:00. Is either number exact? Two things make a p75 less exact than it looks. The first is small: how your tool picks it. The second is big: which visitors happened to come.

Three ways to pick the p75

That pointing is all a percentile is. NIST puts it this way: "Percentiles split a set of ordered data into hundredths." The 75th percentile is three quarters of the way up the ordered list.

Take the evening hour from 20:00 to 21:00. It had 16 visits. The first way to pick is nearest rank, the method from Building a RUM pipeline: work out 75% of the count, round up, and read that visit.

1// LCP of the 16 visits between 20:00 and 21:00, sorted, in ms.2const lcps = [128, 148, 156, 236, 320, 396, 436, 560, 652, 672, 680, 772, 864, 892, 916, 932];3 4const rank = Math.ceil(0.75 * lcps.length);5console.log(`Nearest rank: visit ${rank} in line, ${lcps[rank - 1]} ms`);

• Line 2 is recorded: the hour's 16 LCPs, sorted from quickest to slowest.
• Line 4 works out the rank: 0.75 × 16 is 12. Math.ceil() rounds up when the answer isn't a whole number.
• Line 5 prints Nearest rank: visit 12 in line, 772 ms. It reads lcps[rank - 1], because arrays count from 0.

Nearest rank always picks a real visit. Other tools pick a spot between two visits, and work out a value part of the way from one to the next. NIST's handbook gives its own way, and describes the one Excel uses:

• NIST multiplies 0.75 by one more than the count: 0.75 × 17 = 12.75. That's three quarters of the way from visit 12 to visit 13.
• The other way adds 1 to 0.75 times one less than the count: 1 + 0.75 × 15 = 12.25. NIST says it "is the method used by Excel and is the default method for R."

1// The same 16 visits, sorted, in ms.2const lcps = [128, 148, 156, 236, 320, 396, 436, 560, 652, 672, 680, 772, 864, 892, 916, 932];3 4// The value at a spot such as 12.75: three quarters of the way from visit 12 to visit 13.5const at = (spot) => {6  const k = Math.floor(spot);7  return lcps[k - 1] + (spot - k) * (lcps[k] - lcps[k - 1]);8};9console.log(`NIST, spot ${0.75 * (lcps.length + 1)}: ${at(0.75 * (lcps.length + 1))} ms`);10console.log(`Excel and R, spot ${1 + 0.75 * (lcps.length - 1)}: ${at(1 + 0.75 * (lcps.length - 1))} ms`);

• Line 2 holds the same 16 visits.
• Lines 5 to 8 make a function, at(), that takes a spot such as 12.75. Line 6 keeps its whole part, 12.
• Line 7 starts at visit 12, 772 ms, and adds the fraction of the gap to visit 13, 864 ms. For 12.75 that's 772 + 0.75 × 92.
• Line 9 prints NIST, spot 12.75: 841 ms. Line 10 prints Excel and R, spot 12.25: 795 ms.

Same 16 visits, three answers: 772, 841 and 795 ms. NIST says these methods give "fairly similar, but not exactly the same (particularly for small samples), results." With the whole day's 384 visits, the three gave 704, 707 and 705 ms. Why it matters: know which method your tool uses, and don't compare a p75 from one tool with a p75 from another.

The same mix, a different p75

The method moved the hour's p75 by up to 69 ms. Now the bigger question. If the same kind of visitors came again, would they give the same p75?

The day can answer that. Ten of its hours had exactly the same mix: 12 phones and 4 desktops. Same page, same mix. Only the visitors were different.

1// p75 LCP of the ten hours with 12 phones and 4 desktops, in ms.2// Hours 0 to 5, then 20 to 23. Same page, same mix.3const p75s = [896, 708, 1008, 768, 728, 872, 772, 688, 920, 640];4 5console.log(`The quickest of these hours: ${Math.min(...p75s)} ms`);6console.log(`The slowest of these hours: ${Math.max(...p75s)} ms`);

• Lines 1 and 2 are comments: they say which hours these are.
• Line 3 is recorded: each hour's p75, in the order of the day.
• Line 5 prints The quickest of these hours: 640 ms. Line 6 prints The slowest of these hours: 1008 ms.

That's the bathroom scale. Ten readings of the same thing, from 640 to 1,008 ms. With only 16 visits, the p75 is the 12th in line, so one or two slow phones decide it. This wobble is the p75's uncertainty: how much it would move if you drew another set of visitors. Why it matters: the drop from 920 to 640 ms at 23:00 wasn't news. Hours with this mix landed anywhere in that range.

An error bar from one set of visits

The day had ten hours with the same mix. Real data rarely gives you that. Usually you have one set of visits, and one p75. How can you tell how much it would wobble?

NIST's handbook offers the bootstrap. It "is used to estimate the uncertainty of a statistic": of a number worked out from data, like a p75. The idea is to treat your visits as the whole crowd, and draw new samples from them. It's like drawing names from a hat, and putting each name back before the next draw. Step by step:

1. Make a new list the same size as yours, by picking visits from yours at random. Put each one back after you pick it, so "any data point can be sampled multiple times or not sampled at all". This new list is a resample.
2. Work out the p75 of the resample.
3. Do it again, many times. NIST: "500 subsamples is usually sufficient."
4. Sort the 500 p75s, and keep the middle 90%: from the 25th to the 475th. NIST calls that range a 90% confidence interval. It's the error bar.

Here is one resample of the evening hour:

1// LCP of the 16 visits between 20:00 and 21:00, in the order they came, in ms.2const lcps = [436, 560, 892, 156, 396, 772, 916, 672, 932, 128, 680, 236, 652, 148, 320, 864];3 4// A small random number maker that gives the same numbers on every run.5let seed = 1;6const random = () => (seed = (seed * 16807) % 2147483647) / 2147483647;7 8const resample = lcps.map(() => lcps[Math.floor(random() * lcps.length)]);9console.log(`One resample: ${resample.join(", ")}`);10console.log(`It holds ${new Set(resample).size} different visits of the 16.`);

• Line 2 is recorded: the hour's 16 LCPs, in the order the visits came.
• Lines 5 and 6 make a small random number maker. Each call gives a number from 0 to 1. Math.random() would give new numbers on every run. This one gives the same numbers each time, so the page prints what this text says.
• Line 8 builds the resample. For each of the 16 places, Math.floor(random() * lcps.length) picks a whole number from 0 to 15, and takes that visit.
• Line 9 prints One resample: 436, 892, 652, 672, 932, 156, 436, 680, 680, 320, 916, 932, 148, 436, 436, 932.
• Line 10 prints It holds 10 different visits of the 16.

Look at the repeats. The 436 ms visit came up four times, and the 932 ms visit three times. Six visits didn't come up at all, including the 772 ms one. This resample is a slightly different evening, made from the same kind of visitors. Sort it, and its 12th visit, its p75, is 892 ms, not 772. Now do that 500 times:

1// The same 16 visits, the same random numbers, and nearest rank for p75.2const lcps = [436, 560, 892, 156, 396, 772, 916, 672, 932, 128, 680, 236, 652, 148, 320, 864];3let seed = 1;4const random = () => (seed = (seed * 16807) % 2147483647) / 2147483647;5const p75 = (values) => [...values].sort((a, b) => a - b)[Math.ceil(values.length * 0.75) - 1];6 7const p75s = [];8for (let i = 0; i < 500; i++) p75s.push(p75(lcps.map(() => lcps[Math.floor(random() * lcps.length)])));9p75s.sort((a, b) => a - b);10console.log(`The hour's p75: ${p75(lcps)} ms. The middle 90% of 500 resamples: ${p75s[24]} to ${p75s[474]} ms.`);

• Lines 2 to 4 are the same visits and the same random number maker.
• Line 5 is nearest rank, as a function.
• Line 8 does the last example's line 8, 500 times. Each time, it builds a resample and keeps its p75.
• Line 9 sorts the 500 p75s, quickest first.
• Line 10 reads the 25th and the 475th, p75s[24] and p75s[474], because arrays count from 0. It prints The hour's p75: 772 ms. The middle 90% of 500 resamples: 652 to 892 ms.

So the honest way to report that hour is "p75 772 ms, 90% interval 652 to 892 ms". Its error bar is 240 ms wide.

Back to the team's two hours. Bootstrap each of them, and the bars overlap: 22:00's runs from 508 to 1,032 ms, and 23:00's from 544 to 824 ms. 23:00's whole bar sits inside 22:00's. When two error bars overlap that much, the gap between the two numbers can easily be chance. Proving a change worked shows proper tests for this.

More visits make the bar shorter. The picture shows the hour, the evening's 96 visits, and the whole day's 384:

The whole day's p75 was 704 ms, with a 90% interval from 656 to 748 ms: 92 ms wide. Why it matters: a p75 without its error bar is a guess with no size. Show both on a dashboard.

How many samples you need

One hour gave an error bar 240 ms wide. The whole day gave one 92 ms wide. So how many visits do you need before you can trust a p75?

NIST's page on sample sizes starts with the same question: "How many measurements should be included in the sample?" Its answer: "Unfortunately, there is no correct answer without additional information (or assumptions)." For field data, the piece you have to choose yourself is how big a change you want to see.

For a mean, NIST gives a formula. In plain words, the error shrinks with the square root of the count. The square root of a number is the number that, multiplied by itself, gives it: the square root of 4 is 2. So four times the visits halve the error, and 16 times the visits divide it by 4.

1// NIST's formula for a mean: the error shrinks with the square root of the count.2// How much narrower than with 16 visits?3for (const visits of [64, 256, 1024]) {4  console.log(`${visits} visits: ${Math.sqrt(visits / 16)} times narrower`);5}

• Lines 1 and 2 are comments.
• Line 3 steps through three sizes. Line 4 divides each one by 16, and takes the square root with Math.sqrt().
• It prints 64 visits: 2 times narrower, 256 visits: 4 times narrower and 1024 visits: 8 times narrower.

A p75 isn't a mean, so the course measured it instead. It drew 500 samples of each size from the day's 384 visits, picking at random and putting each visit back, like new visitors from the same mix. Then it kept the middle 90% of each size's 500 p75s:

1// The middle 90% of p75 LCP from 500 samples of each size, drawn from the day's visits, in ms.2const spread = { 16: [384, 948], 64: [576, 880], 256: [640, 752], 1024: [680, 744] };3 4for (const [visits, [low, high]] of Object.entries(spread)) {5  console.log(`${visits} visits: ${low} to ${high} ms, ${high - low} ms wide`);6}

• Line 2 comes from the recording: for each size, the lowest and the highest p75 of the middle 90%.
• Line 4 steps through the sizes. Object.entries() gives each size with its pair of numbers.
• It prints 16 visits: 384 to 948 ms, 564 ms wide and 64 visits: 576 to 880 ms, 304 ms wide.
• Then 256 visits: 640 to 752 ms, 112 ms wide and 1024 visits: 680 to 744 ms, 64 ms wide.

Each step has four times the visits. The spread got 1.9, 2.7 and 1.8 times narrower: about half each time, as the square root says, though not exactly. From 16 visits to 1,024, it got 8.8 times narrower. The formula for a mean said 8.

Now turn that into a rule you can use: the error bar must be narrower than the change you're looking for. Say you want to notice the p75 getting 100 ms worse. With 16 or 64 visits, the spread alone is more than three times wider than that. With 1,024 visits, it's 64 ms wide, so a 100 ms change is bigger than the wobble.

Higher percentiles need more visits still. web.dev explains why the Core Web Vitals use the 75th percentile, not the 95th. With 100 visits, "it would take just 5 outlier samples for the 95th percentile value to be affected by the outliers." CrUX only includes a page once it has a minimum number of visitors. "An exact number is not disclosed", it says, but it was chosen so there are "enough samples to be confident in the statistical distributions".

Building a RUM pipeline saw a p75 of four visits swing from 380 to 684 ms. Segmenting field data said four visits are too few to judge. Now you can see why: here, even 16 visits gave a spread 564 ms wide.

Seasonality and traffic mix

More visits shrink the wobble. They can't fix the other thing that moved the dashboard: the mix. Over the day, the hours' p75s ran from 232 to 1,008 ms. Some of that was chance. To see the rest, put the hours together into four quarters of six hours each, with 96 visits per quarter:

1// Each quarter of the day: how many of its 96 visits were phones, and p75 LCP for everyone, in ms.2const quarters = {3  night: { phones: 72, p75: 872 },4  morning: { phones: 40, p75: 616 },5  afternoon: { phones: 32, p75: 484 },6  evening: { phones: 66, p75: 736 },7};8 9for (const [name, quarter] of Object.entries(quarters)) {10  console.log(`${name}: ${Math.round((quarter.phones / 96) * 100)}% phones, p75 ${quarter.p75} ms`);11}

• Lines 2 to 7 are recorded: how many of each quarter's visits came from phones, and its p75 LCP.
• Line 10 works out the share of phones, and prints it with the p75.
• It prints night: 75% phones, p75 872 ms and morning: 42% phones, p75 616 ms.
• Then afternoon: 33% phones, p75 484 ms and evening: 69% phones, p75 736 ms.

Sort the quarters by their share of phones, and you've sorted them by p75 too. The more phones, the slower. But did the page get slower at night? Split each quarter by device:

1// The same quarters, one device at a time: p75 LCP in ms.2const quarters = {3  night: { phones: 1008, desktops: 208 },4  morning: { phones: 1000, desktops: 224 },5  afternoon: { phones: 936, desktops: 220 },6  evening: { phones: 868, desktops: 208 },7};8 9for (const [name, p75] of Object.entries(quarters)) {10  console.log(`${name}: phones ${p75.phones} ms, desktops ${p75.desktops} ms`);11}

• Lines 2 to 7 are recorded: each quarter's p75 for phones alone, and for desktops alone.
• It prints night: phones 1008 ms, desktops 208 ms and morning: phones 1000 ms, desktops 224 ms.
• Then afternoon: phones 936 ms, desktops 220 ms and evening: phones 868 ms, desktops 208 ms.

Desktops stayed between 208 and 224 ms all day. Phones stayed between 868 and 1,008 ms. Neither got slower at night. What changed was how many of each came. The afternoon looked quick because two visits in three came from desktops.

A pattern like this has a name: seasonality. NIST's handbook defines it as "periodic fluctuations": ups and downs that come back on a schedule. The day here had one by the hour. Real traffic can also change by weekday and by time of year. A sale, a holiday or an advertising campaign can bring in a different crowd. web.dev warns that over short periods, "RUM data can be unduly impacted by fluctuations in website traffic and visitors."

Segmenting field data showed the same trap with a release: its p75 looked eight times better, because most of its visits came from desktops. Here it happened with no release at all, just the clock. Three habits keep the mix from fooling you:

• Compare the same times. Put this Monday evening next to last Monday evening, not next to this Monday afternoon.
• Use whole weeks. A window of whole weeks holds the same number of each weekday. CrUX's window is a whole number of weeks. The exercises ask how many days it covers.
• Split by segment. Compare phones with phones, and desktops with desktops. Here, each device stayed put while the total moved.

Two things moved the day's p75, and neither was the page. Read across each row to tell them apart:

Why it matters: check both before you blame the page.

Which tool answers it?

Each question below is answered best by one of the tools in this lesson. Sort it to that tool.

Common mix-ups

• "A percentile has one right formula." No. The same 16 visits gave 772, 841 and 795 ms. Tools pick differently when the spot falls between two visits.
• "A p75 from an hour of visits is exact." No. Hours with the same page and the same mix ran from 640 to 1,008 ms. One hour's 90% interval ran from 652 to 892 ms.
• "Twice the visits, half the error." No. It takes about four times the visits, because the error shrinks with the square root of the count.
• "A worse evening means a regression." No. The evening had 69% phones, and the afternoon 33%. Each device's p75 barely moved.
• "More visits fix everything." No. They shrink the wobble from chance. A different mix stays different.
• "With enough visits, the mean is fair." No. With 384 visits, the mean was still slower than 233 of them. More visits don't remove a long tail.

Practice exercises

Answer first, then use the hints.

// The day's 384 LCPs, counted in 250 ms bins: under 250 ms, 250 to 499 ms, and so on.
const counts = [148, 80, 73, 39, 23, 12, 2, 3, 4];

let below = 0;
for (const [bin, count] of counts.entries()) {
  below += count;
  console.log(`Under ${(bin + 1) * 250} ms: ${Math.round((below / 384) * 100)}% of visits`);
  if (below / 384 >= 0.5) break;
}

// The same 16 visits, sorted, in ms.
const lcps = [128, 148, 156, 236, 320, 396, 436, 560, 652, 672, 680, 772, 864, 892, 916, 932];

// The value at a spot such as 8.5: halfway from visit 8 to visit 9.
const at = (spot) => {
  const k = Math.floor(spot);
  return lcps[k - 1] + (spot - k) * (lcps[k] - lcps[k - 1]);
};
console.log(`NIST, spot ${0.5 * (lcps.length + 1)}: ${at(0.5 * (lcps.length + 1))} ms`);
console.log(`Excel and R, spot ${1 + 0.5 * (lcps.length - 1)}: ${at(1 + 0.5 * (lcps.length - 1))} ms`);

Check your understanding

For each question, ask three things. What's the shape of the data? How sure is the number? Did the mix change?

Key takeaways

• A p75 can move because the page changed, the mix changed, or chance brought different visitors.
• A histogram shows the shape. A cumulative chart gives any percentile, and the share under any time.
• Percentile methods differ on small samples. Know which one your tool uses.
• Every percentile has an error bar. Bootstrap it: 500 resamples, then keep the middle 90%.
• Four times the visits make the error bar about half as wide. It must be narrower than the change you look for.
• The mix moves the number on its own. Compare the same hours and days, or split by segment.

Sources

The facts on this page come from these docs:

• Histogram, NIST/SEMATECH e-Handbook of Statistical Methods.
• Histogram Interpretation: Skewed (Non-Normal) Right, NIST/SEMATECH e-Handbook of Statistical Methods.
• CrUX methodology, Chrome for Developers.
• Percentiles, NIST/SEMATECH e-Handbook of Statistical Methods.
• Bootstrap Plot, NIST/SEMATECH e-Handbook of Statistical Methods.
• Sample sizes required, NIST/SEMATECH e-Handbook of Statistical Methods.
• How the Core Web Vitals metrics thresholds were defined, web.dev.
• Seasonality, NIST/SEMATECH e-Handbook of Statistical Methods.
• Why is CrUX data different from my RUM data?, web.dev.

Coming next: Proving a change worked. You'll compare two sets of visits, before and after a change, and say how sure you are that it helped.