completefrontend
Open lab My progress
THE WEB PERFORMANCE FIELD GUIDE

The business case for performance

Show what speed is worth with your own data: link speed to sales without being fooled, read published studies critically, price a fix, and make the case.

By the end, you can
  • 01
    Link speed and sales in your own dataGroup real visits by LCP, find what else differs between fast and slow visits, and compare like with like.
  • 02
    Read a published study criticallyAsk how it measured, who wrote it, what its numbers mean in points, and which studies never got published.
  • 03
    Price a fix and make the caseTurn a fair test into pounds with a range, write down every assumption, and put the case on one page.

What is a faster page worth to us?

Ravi's test from lesson 40.3 is over. The faster home page won: 48% of phone visits clicked "See all our teas", against 33%. The owner is pleased. Then she asks two new questions.

"Clicks are nice. But what are they worth in pounds? And a friend sent me a study. It says every tenth of a second is worth millions. Is that true for us?"

Both questions are fair. The owner decides where Meera's hours go. So she needs a number she can trust, and she needs to know which numbers not to trust. This lesson builds Ravi's answer, one step at a time.

Ravi starts with what he already has. His RUM, short for real user monitoring, keeps a record of every phone visit to the home page. Each record holds the visit's LCP: the moment the biggest image or text appears. Here, that's the photo. The record also says whether the visitor clicked "See all our teas". That click is the home page's conversion: the thing the shop wants a visitor to do there.

Here is the month before the fix, in Google's three LCP groups:

A month of clicks, by LCPPop out in the code editor (opens in a new tab)JavaScript
// A month of phone visits to the home page: visits and clicks, by LCP.const month = [  { lcp: "good, up to 2.5 s", visits: 1835, clicks: 791 },  { lcp: "needs improvement", visits: 1177, clicks: 333 },  { lcp: "poor, over 4 s", visits: 588, clicks: 64 },]; for (const { lcp, visits, clicks } of month) {  console.log(`${lcp}: ${Math.round((clicks / visits) * 100)}% clicked`);}
  • Line 1 is a comment.
  • Lines 3 to 5 are recorded. Each holds one LCP group, its visits, and how many of them clicked. Google calls an LCP good up to 2.5 seconds, and poor over 4. Between the two, it "needs improvement".
  • Line 8 goes through the groups. Line 9 prints the share of each group that clicked.
  • It prints good, up to 2.5 s: 43% clicked, needs improvement: 28% clicked and poor, over 4 s: 11% clicked.
A bar chart of Ravi's month of phone visits, by LCP. good (up to 2.5 s): 1,835 visits, 43% clicked. needs improvement (2.5 to 4 s): 1,177 visits, 28% clicked. poor (over 4 s): 588 visits, 11% clicked.43% clickedgoodup to 2.5 s1,835 visits28% clickedneeds improvement2.5 to 4 s1,177 visits11% clickedpoorover 4 s588 visits
The month's phone visits in Google's three LCP groups. The taller the bar, the more of that group clicked "See all our teas". Good visits clicked about 4 times as often as poor ones.

Good visits clicked about 4 times as often as poor ones. The owner reads the chart the quick way: "So a fast page brings 4 times the clicks?" Not quite. Some of that gap isn't speed at all. The next section finds out how much.

The friend's study has a different problem. It measured other people's sites, not the Leaf and Kettle.

Real-life analogyA suit off the rack, or one made to measure

You need a suit. A shop has one off the rack in size 40. It fits someone, but maybe not you. A tailor takes out a tape and measures your own shoulders. That takes longer, but the suit fits you.

In real life: A suit off the rack, in someone's size
In a business case: A number from a published study about another site
In real life: The tailor's tape on your own shoulders
In a business case: A fair test on your own visitors
In real life: Trying it on before you pay
In a business case: Checking the gain after the fix ships
The main idea

A business case says what speed is worth to your shop, in money, with a range. Its number comes from a fair test on your own visitors. Your own month shows that speed and sales go together. Published studies show what happened somewhere else. Read both with care, and price from the test.

Here is Ravi's whole case in one picture:

Ravi's case in five steps. month: all phones: 8.4 points a second. by phone: one kind: 5.8 and 5.6 a second. fair test: lesson 40.3: 14.3 more clicks per 100 visits. price: about £930 a month, with a range. one page: answer first, then proof, then the ask.what goes togetherwhat the fix does, and is worthmonthall phones:8.4 pointsa secondby phoneone kind:5.8 and 5.6a secondfair testlesson 40.3:14.3 more clicksper 100 visitspriceabout £930a month,with a rangeone pageanswer first,then proof,then the ask
The whole lesson. Your own month shows that speed and clicks go together. Comparing within one kind of phone brings the number down. A fair test gives the fix's own size. Then price it, and put it on one page.

The next four sections take one step each: your own month, other people's studies, the price, and the page.

How the course recorded it

  • The page: Ravi's home page as it was before the fix, from lesson 40.1. It was served over HTTPS from the course's computer, at a made-up address. Each page sent its own beacon, as in lesson 40.3.
  • The phones: three kinds, each with its own CPU speed and its own typical network. Chrome 157 loaded the page 100 times for each kind. Each load drew its own network around its kind's typical one, the way lesson 13.1 drew phones.
  • The month: 3,600 phone visits, 120 a day for 30 days, as in lesson 40.3. A quarter came from new phones, half from mid-range phones, and a quarter from budget phones. Each visit took the LCP of one recorded load of its kind, picked at random.
  • The test: every number about the fix comes from lesson 40.3's week.

The clicks come from a model, as in lesson 40.3. That lesson started every phone at a 50% chance to click. This lesson adds one thing: who owns the phone.

The course's model of a click, by phonePop out in the code editor (opens in a new tab)JavaScript
// The course's model: each kind of phone starts at its own chance.const start = { new: 0.6, "mid-range": 0.5, budget: 0.4 };function clickChance(kind, lcpMs) {  return Math.max(0.05, start[kind] - 0.06 * (lcpMs / 1000));} console.log(`New phone, photo at 2 s: ${Math.round(clickChance("new", 2000) * 100)}%`);console.log(`Budget phone, photo at 2 s: ${Math.round(clickChance("budget", 2000) * 100)}%`);
  • Line 2 gives each kind of phone its own starting chance. In this model, owners of budget phones click less at any speed.
  • Line 4 is lesson 40.3's rule: 6 points less for each second of waiting, and never below 5%.
  • Lines 7 and 8 print New phone, photo at 2 s: 48% and Budget phone, photo at 2 s: 28%. The photo came at the same moment, and the chances are still far apart.

The rule is the course's own. It keeps one thing known: speed takes 6 points a second, on every phone. So you can check each method against the true answer. Everything else is real: the page, the loads, the LCPs and the beacons.

Speed and clicks in your own data

Ravi's chart shows that speed and clicks go together. That's called a correlation. It doesn't say the speed caused the clicks. Lesson 1.2 met the reason. A hidden difference, called a confounder, can change both at once. Is there one in Ravi's month?

Who made the slow visits?

Start with what a visitor sees. Here is the home page as it was, on a new phone and on a budget phone, each on its typical network:

The home page as it was, on a new phone and a budget phone, at four moments. At 1.3 s: new phone, the whole photo; budget phone, a strip of the photo. At 2.5 s: new phone, the whole photo; budget phone, about half the photo. At 3.0 s: new phone, the whole photo; budget phone, most of the photo. At 3.6 s: new phone, the whole photo; budget phone, the whole photo.new phonebudget phone1.3 sThe home page on a new phone, showing the whole photoThe home page on a budget phone, showing a strip of the photonew phone: whole2.5 sThe home page on a new phone, showing the whole photoThe home page on a budget phone, showing about half the photogood LCP line3.0 sThe home page on a new phone, showing the whole photoThe home page on a budget phone, showing most of the photo3.6 sThe home page on a new phone, showing the whole photoThe home page on a budget phone, showing the whole photobudget phone: whole
The same page, each phone on its kind's typical network. Each column is one moment. The new phone's photo is whole at 1.2 s, the budget phone's at 3.5 s.

On the new phone, the photo is whole after 1.2 seconds. The budget phone needs 3.5. In the course's model, a budget phone also sits on a slower network, so its photo comes later. Now count the whole month, one kind of phone at a time:

The month, one kind of phone at a timePop out in the code editor (opens in a new tab)JavaScript
// The same month, one kind of phone at a time.const phones = {  new: { visits: 913, poor: 0, good: 895, goodClicks: 467 },  "mid-range": { visits: 1764, poor: 222, good: 814, goodClicks: 291 },  budget: { visits: 923, poor: 366, good: 126, goodClicks: 33 },}; for (const [kind, p] of Object.entries(phones)) {  const poor = Math.round((p.poor / p.visits) * 100);  const clicked = Math.round((p.goodClicks / p.good) * 100);  console.log(`${kind}: ${poor}% poor; ${clicked}% of good visits clicked`);}
  • Lines 3 to 5 are recorded. Each holds one kind's visits, its poor visits, its good visits, and how many good visits clicked.
  • Line 8 goes through the kinds. Line 9 works out the share of its visits that were poor.
  • Line 10 works out the share of its good visits that clicked.
  • It prints new: 0% poor; 52% of good visits clicked, mid-range: 13% poor; 36% of good visits clicked and budget: 40% poor; 26% of good visits clicked.

Two things stand out. First, budget phones made 62% of the month's poor visits, though they made only a quarter of all visits. Second, look at good visits only. On new phones, 52% of them clicked. On budget phones, 26%. The speed was good on both, and the clicks were far apart.

Why it matters: the poor group is full of budget phones, whose owners click less anyway. So the chart's gap holds two things at once: the wait, and who was waiting.

Real-life analogyShoppers with a basket spend more

A shop notices that shoppers with a basket spend three times as much as shoppers without one. Should it hand everyone a basket? No. People who came for a big shop pick up a basket. People who popped in for milk don't. The basket and the big spend both come from the reason for the trip.

In real life: Carrying a basket
In a business case: A visit with a good LCP
In real life: Coming in for a big shop
In a business case: Owning a new phone, whose owner clicks more anyway
In real life: Handing every shopper a basket
In a business case: Making the page faster for everyone
In real life: Comparing big shoppers with and without a basket
In a business case: Comparing good and poor visits on one kind of phone

Compare like with like

So compare visits on the same kind of phone. Then the phone can't make the gap. Use lesson 40.3's sum: the gap in clicks between good and poor visits, over the gap in their mean LCP. The mean is the average. The sum gives points per second. A point is one percent: 40% minus 6 points is 34%.

Points per second, all phones and one kind at a timePop out in the code editor (opens in a new tab)JavaScript
// Good and poor visits: the share that clicked, and their mean LCP in seconds.const all = { good: { clicked: 0.431, lcp: 1.61 }, poor: { clicked: 0.109, lcp: 5.45 } };const midRange = { good: { clicked: 0.357, lcp: 1.96 }, poor: { clicked: 0.167, lcp: 5.21 } };const budget = { good: { clicked: 0.262, lcp: 2.24 }, poor: { clicked: 0.074, lcp: 5.59 } }; const perSecond = ({ good, poor }) => ((good.clicked - poor.clicked) / (poor.lcp - good.lcp)) * 100;console.log(`All phones: ${perSecond(all).toFixed(1)} points per second`);console.log(`Mid-range phones: ${perSecond(midRange).toFixed(1)} points per second`);console.log(`Budget phones: ${perSecond(budget).toFixed(1)} points per second`);
  • Lines 2 to 4 are recorded. For all phones, then mid-range and budget phones, each holds two groups. Each group has the share that clicked, and its mean LCP in seconds.
  • Line 6 is the sum: the drop in clicks, over the extra seconds, in points.
  • Lines 7 to 9 print All phones: 8.4 points per second, Mid-range phones: 5.8 points per second and Budget phones: 5.6 points per second.

Across all phones, a second seems to cost 8.4 points. Inside one kind of phone, it costs 5.8 and 5.6. New phones aren't on the list. None of their 913 visits had a poor LCP, so there's nothing to compare.

Why it matters: mixing the phones made each second look costlier than it is. A price built on the mixed number would promise too much. Try each kind of phone yourself:

Playground: the month, one kind of phone at a time
All phones
  1. Visits in the month3,600
  2. Good LCP, up to 2.5 s1835 visits, 791 clicked (43%)
  3. Needs improvement1177 visits, 333 clicked (28%)
  4. Poor LCP, over 4 s588 visits, 64 clicked (11%)
  5. Points per second, good against poor8.4
Try it yourself
All phones: 8.4 points per second

Each row counts the month's visits in one of Google's LCP groups, and how many of them clicked. The last row is the lesson's sum: the gap in clicks between good and poor visits, over the gap in their mean LCP.

Ravi's month of phone visits, built from real loads in Chrome. The clicks come from the course's model.

What the fair test said

Splitting by phone took out one hidden difference. Others can still hide in the month: the network, returning visitors, the time of day. You can only split by what you recorded.

Ron Kohavi's guide to experiments on the web says it plainly. Patterns found by mining data "are correlational". A test that splits visitors by chance is different: it spreads the confounders "equally over all values through random assignment". That includes the ones nobody recorded.

Ravi already has such a test: lesson 40.3's week. There, the fix brought the photo 2.7 seconds sooner on average. It added 14.3 clicks per 100 visits. That's 5.3 points a second. Here, the course's model knows the true answer, so put them side by side:

  • The month, all phones mixed: 8.4 points a second.
  • The month, one kind of phone at a time: 5.8 and 5.6.
  • The fair test: 5.3.
  • The model's own rule: 6.

The mixed month is the furthest off. Why it matters: use your month to find where to look. Use a fair test to say what a fix is worth.

Read published studies with care

Now the owner's second question: her friend's study. Lesson 1.2 checked where some famous numbers come from. This time, read a whole study. Ask it five questions:

  • How did it measure? A split by chance, a before and after, or fast and slow compared?
  • Who wrote it, and who paid?
  • What exactly did it measure? Which speed number, and which business number?
  • How big is the change, in points as well as in percent?
  • Which studies are missing?

The friend's study

The friend's study is "Milliseconds Make Millions". Lesson 1.2 met its headline number. Its full report answers the five questions itself:

  • How: the report says it "examined the correlation between mobile site speed and mobile site user behaviours". Speed went up and down by itself. Nobody changed it on purpose.
  • Who: Deloitte wrote it, with data from a firm called Fifty-Five. The report "has been commissioned by Google". It also says Deloitte "did not audit nor validate this data".
  • What: each hour's speed came from Lighthouse, a lab tool, not from each visitor's own load. The report says this was to set aside "connection type, devices used, etc." So the visitors' own phones and networks aren't in its speed numbers.
  • Missing: over 70 brands were asked, and 37 qualified. A result counted only if it reached "95% statistical significance". Results for computers were left out: the report found "a lot of contradicting parameters".

None of this makes the study wrong. It makes it a correlation across 37 other sites. Why it matters: it can say speed and sales moved together there. It can't say what a faster photo is worth at the Leaf and Kettle.

A study that warns you itself

Contentsquare sells tools that record visits. It grouped visits on 997 customer sites by their INP: how fast a page answers a tap. In retail, visits with a good INP converted at 2.5%, and the rest at 2.0%. It calls that "a 25% better conversion rate". Then it warns: "Correlation does not equal causation."

Its own numbers show why. Split by device, computers converted at 4.2% and 2.9%. Phones converted at 1.9% and 1.7%. It also says phones had worse INP. So the good-INP group holds more computers, which convert more anyway. That's Ravi's month again, with computers in place of new phones. Yet its last section says INP "has a significant impact". Why it matters: read the method, not just the ending.

One study, two kinds of number

Rakuten 24's study, from lesson 40.3, holds both kinds. First, it compared real visits by their LCP. On its home page, it says a good LCP "can lead to" up to 61.13% more conversions. Then it ran an A/B test on one landing page for a month. The faster version converted 33.13% better.

The pages differ, so the two numbers don't line up exactly. But only the second came from a split by chance. Why it matters: when a study gives both kinds, quote the test.

Before and after

Many case studies compare the numbers before a speed project with the numbers after it. That's easier than a test. But everything else that changed comes along too:

  • NDTV, a news site, made its LCP 55% better. Its bounce rate fell by half. The study adds a note: NDTV "made other product changes" at the same time. So "it is not possible to conclusively say" that speed was the only cause.
  • Agrofy, a farm marketplace, moved to a new framework, then spent a month on speed. Its LCP got 70% better. Its load abandonment on product pages fell 76%.

Now check the percentages. A change can be measured against the start or the end. A rate can fall by points or by percent:

Two published numbers, checkedPop out in the code editor (opens in a new tab)JavaScript
// Two numbers from published case studies, checked.const lcp = { before: 4.5, after: 2.5 }; // secondsconst abandoned = { before: 3.8, after: 0.9 }; // % of loadsconst drop = (x) => x.before - x.after; console.log(`LCP: ${Math.round((drop(lcp) / lcp.before) * 100)}% less than before`);console.log(`LCP: ${Math.round((drop(lcp) / lcp.after) * 100)}% of the new value`);console.log(`Abandoned: ${drop(abandoned).toFixed(1)} points fewer`);console.log(`Abandoned: ${Math.round((drop(abandoned) / abandoned.before) * 100)}% fewer`);
  • Line 2 is The Economic Times' LCP, in seconds. Its study says LCP "improved by 80% from 4.5 seconds to 2.5 seconds".
  • Line 3 is Agrofy's load abandonment: from 3.8% of loads to 0.9%.
  • Line 4 works out the drop: before, minus after.
  • Lines 6 and 7 print LCP: 44% less than before and LCP: 80% of the new value.
  • Lines 8 and 9 print Abandoned: 2.9 points fewer and Abandoned: 76% fewer.

The Economic Times' 80% is the drop measured against the new value, 2.5 seconds. Against where it started, LCP fell 44%. Agrofy's 76% is true too. But it's 2.9 points of a small number. Why it matters: work out the points before you repeat a percent.

The studies you never see

Real-life analogyReviews in a restaurant window

A restaurant tapes its best reviews to the window. Every one of them is real. But the one-star reviews stay inside. So the window can't tell you how often a meal goes wrong.

In real life: The reviews taped to the window
In a business case: Published case studies
In real life: The one-star reviews kept inside
In a business case: Speed projects that changed nothing, never written up
In real life: The owner picks which ones to show
In a business case: A page that asks for "a compelling business win"
In real life: Trying the food yourself
In a business case: A test on your own visitors

web.dev, Google's site for web developers, has a page on the business impact of Core Web Vitals. It ends with an invitation: "If you also have a compelling business win", you can send it in. Wins get written up. Projects that changed nothing rarely do.

How often do changes win? Kohavi's team counted at Microsoft. Of well-run experiments built to improve a key number, "only about one-third were successful". So a page of wins isn't a fair sample of what speed work does. Why it matters: a published win shows what can happen, not what usually happens.

One more place to look: WPO Stats collects one-paragraph summaries of performance case studies, each with a link. Use it as an index. Then open the study itself, and ask the five questions. Try them on six studies:

Playground: read a study with five questions
Rakuten 24
  1. How it measuredSplit by chance
  2. Who wrote itweb.dev, Google's site for web developers, with Rakuten 24
  3. What it measuredA landing page for a month, with half the visitors on each version.
  4. What it saysA 33.13% higher conversion rate on the faster version.
  5. In pointsNot given: only the relative change.
  6. What you can sayA fair test. Its size belongs to that page and those visitors.
Try it yourself
Rakuten 24: split by chance

Each row answers one question about the study, from its own pages. Read the first row before you read the numbers.

Six published studies, as their own write-ups describe them.

Read this table across: each way to link speed and money, what it can show, and what can fool it.

Three ways to link speed and money
WayWhat it can showWhat can fool it
Compare fast and slow visitsThat speed and sales go together, on many visits at once.Fast visits come from different visitors: newer phones, better networks. They may buy more anyway.
Compare before and after a changeThat the numbers moved after the work shipped.Everything else that changed in between: seasons, sales, a redesign, a new framework.
Split visitors by chanceWhat the change itself is worth, on those visitors.Too few visitors, uneven groups, or looking too early, as lesson 40.3 showed.

Put a price on the fix

Back to the owner's first question: what is the fix worth in pounds? Ravi needs three numbers, all his own:

  • Visits: 3,600 phone visits to the home page a month.
  • The gain: lesson 40.3's 14.3 more clicks per 100 visits. Its 90% interval runs from 7.6 to 20.7.
  • What a click is worth: his shop's analytics say one tea-list visit in ten ends in an order. The average order is £18.
The fix, pricedPop out in the code editor (opens in a new tab)JavaScript
// The home photo fix, priced: lesson 40.3's test, and Ravi's own numbers.const visits = 3600; // phone visits to the home page in a monthconst gain = { low: 0.076, middle: 0.143, high: 0.207 }; // more clicks per visitconst orderShare = 0.1; // tea-list visits that end in an orderconst orderValue = 18; // the average order, in pounds for (const [name, more] of Object.entries(gain)) {  const pounds = visits * more * orderShare * orderValue;  console.log(`${name}: £${Math.round(pounds).toLocaleString("en-US")} a month`);}
  • Line 2 is the month's phone visits.
  • Line 3 is the test's gain in clicks per visit: the low end of its interval, the middle, and the high end.
  • Line 4 says one tea-list visit in ten ends in an order. Line 5 is the average order, in pounds.
  • Line 8 multiplies them, one estimate at a time. Line 9 prints each one, rounded to a pound.
  • It prints low: £492 a month, middle: £927 a month and high: £1,341 a month.
The price of the fix, step by step. 3,600 visits a month, × 14.3 more clicks per 100, 515 more clicks, × 1 order in 10, 51 more orders, × £18 an order: £927 a month. From the test's 90% interval: low end £492, middle £927, high end £1,341.3,600 visits a month× 14.3 more clicks per 100515 more clicks× 1 order in 1051 more orders× £18 an order£927 a monththe test's range, a monthlow end: £492middle: £927high end: £1,341£492 to £1,341
Left: the sum, with the test's middle gain. Right: the same sum with each end of the test's 90% interval. Plan with the range, not one number.
Real-life analogyA taxi fare, given as a range

Before you book a taxi, the app shows a fare of "£14 to £18". It knows the distance. It can't know the traffic yet. So you bring £18, and you're glad when it costs £14.

In real life: The distance
In a business case: The 3,600 visits a month
In real life: The traffic nobody knows yet
In a business case: The test's chance error, its 90% interval
In real life: "£14 to £18"
In a business case: The low end to the high end, a month
In real life: Bringing enough for the worst fare
In a business case: Deciding with the low end

Even the low end, £492 a month, is more than twice what the fix cost. Meera's 5 hours cost the shop £200, at £40 an hour. The middle estimate pays that back in about 6 days. Why it matters: when even the low end is a clear yes, the owner doesn't need to trust the middle.

Write down every assumption

Each number in the sum stands for something Ravi assumed. He writes them down, so the owner can question them:

  • Phones only. Computers weren't in the test, so they add nothing here.
  • New clicks order as often as today's. If the extra visitors are less keen, the value is lower.
  • The gain lasts. Kohavi's guide notes that a test measures the effect "during the experimentation period, typically a few weeks." So Ravi says "a month", and checks it each month.
  • The test can miss. A 90% interval misses one time in ten, as lesson 40.3 showed.

What the month would have promised

What if Ravi had priced the fix from his month instead? The month says 8.4 points a second. The fix saved 2.7 seconds. So the month promises 22.7 more clicks per 100 visits: £1,471 a month. Try each estimate:

Playground: what the fix is worth, from each estimate
Test: middle
  1. Where the gain comes fromThe gain the test measured
  2. More clicks per 100 visits14.3
  3. More clicks a month515
  4. More orders a month, 1 in 1051
  5. More sales a month, at £18 an order£927
  6. Twelve months, if the gain lasts£11,120
Try it yourself
Test: middle: £927 a month

Each row is one step of the lesson's sum: 3,600 phone visits a month, times the gain, times 1 order in 10, times £18.

The test's numbers come from lesson 40.3's week. The month's comes from this lesson's month of visits.

The month's price is 59% above the test's middle, and above even its high end. Why it matters: promise what the test showed. Then the monthly check will back you up.

Make the case on one page

The owner wants a decision she can make in a minute. Lesson 40.2 gave Ravi's audit report its order: the answer first, then the proof, then the method. A business case uses the same order, on one page.

Real-life analogyAsking the bank for a loan

You ask the bank for a loan for a new coffee machine. The manager wants four things: how much, what it will earn, your proof, and what happens if sales are slow. Bring all four on one sheet, and the meeting takes ten minutes.

In real life: How much you want
In a business case: The ask: Meera's time
In real life: What the machine will earn
In a business case: The value, with its range
In real life: Your bank statements
In a business case: The test on your own visitors
In real life: "What if sales are slow?"
In a business case: The low end, and the assumptions

Here is Ravi's page. Every number comes from the sums above:

Ravi's one-page caseText
# Speed pays: the faster home photoAnswer: it brings about £930 of tea orders a month, between £490 and £1,340.Cost: 5 hours of Meera's work, £200. It paid that back in about 6 days.Evidence: our own test of 570 phone visits, split by chance: 14 more clicks on our teas per 100 visits.Sum: 3,600 phone visits a month × 14.3 more clicks per 100 × 1 order in 10 × £18 an order.Assumptions: phones only; new clicks order as often as today's; the gain lasts.Not used: our month of visits, which says 8.4 points a second. Budget phones make most slow visits.Ask: one day of Meera's time a month for speed, each fix tested like this one.Check: phone orders each month, and the next test's result.
  • Line 1 is the title. It says the answer, not just the topic.
  • Line 2 is the answer, with its range. Money is rounded to £10, because it's an estimate.
  • Line 3 is the cost, and how fast the fix paid it back.
  • Line 4 is the evidence: a test on the shop's own visitors.
  • Line 5 shows the sum, so anyone can redo it.
  • Line 6 lists the assumptions.
  • Line 7 says what Ravi left out, and why.
  • Line 8 is the ask: what Ravi needs from the owner.
  • Line 9 says how everyone will know it worked.

Notice what isn't there: no famous study, and no "speed is worth millions". If the owner asks about her friend's study, Ravi's answer is ready. It measured other sites, and it measured a correlation. Why it matters: a case built on your own test still stands after the owner's questions.

Two readers, one page

Lesson 40.2 wrote for two readers, and so does this page. The owner reads lines 2, 3 and 8, and decides. Meera reads lines 4 to 7, and checks the work. Both read numbers in pounds and orders, not points per second. Those stay in Ravi's notes, with a link to the test.

After the yes

The case ends with a check, so Ravi checks. His RUM already counts phone visits and clicks. Each month, his report gets one more line: phone orders this month, against the month before the fix. The next fix gets its own test, the way lesson 40.3 ran one. Why it matters: each tested fix adds proof, so the next case is easier to make.

How did this study measure?

Each line below describes one study from this course. Sort it by how it measured:

How did this study measure?
  • Ravi's week: the edge picks each phone visitor's page with a coin
  • Rakuten 24: half the visitors get the faster landing page for a month
  • Google holds its results back 400 ms for one group of users
  • NDTV: bounce rate before and after a month of LCP work
  • Yahoo! JAPAN News: page views per session before and after fixing layout shifts
  • Agrofy: load abandonment across a new framework and a month of speed work
  • Ravi's month: good-LCP visits against poor-LCP visits
  • Milliseconds Make Millions: hourly speed against each brand's analytics, on 37 sites
  • Contentsquare: visits with a good INP against visits with a poor one
Try it yourself
0 of 9 correct

Sort each study by how it measured.

Choose a category for every card. You can change an answer at any time; Reset clears them all.

Common mix-ups

  • "Good-LCP visits clicked far more often, so a fast page multiplies clicks." No. Budget phones made most of the poor visits, and their owners click less at any speed. Inside one kind of phone, the gap shrinks.
  • "Splitting by device removes every hidden difference." No. It removes only the one you split by. A split by chance balances the ones you never recorded, too.
  • "A study from a famous firm is a fair test." No. Milliseconds Make Millions is a correlation over natural changes in speed, as its own report says.
  • "76% fewer means the rate fell by 76." No. Agrofy's rate fell from 3.8% to 0.9%: 2.9 points.
  • "Published case studies show what speed work usually does." No. Wins get written up. At Microsoft, only about a third of well-run experiments improved their key number.
  • "The value of a fix is one number." No. Give the range from the test's interval, and the assumptions behind it.
  • "Multiply a month by twelve for the year." No. A test measures the weeks it ran. A year assumes the gain lasts, so say so, and check it.

Practice exercises

Answer first, then use the hints.

Exercise 1 · Warm-upBigger orders

The average order grows to £20. What does the middle line print?

Starter codePop out in the code editor (opens in a new tab)JavaScript
// The home photo fix, priced: lesson 40.3's test, and Ravi's own numbers.
const visits = 3600; // phone visits to the home page in a month
const gain = { low: 0.076, middle: 0.143, high: 0.207 }; // more clicks per visit
const orderShare = 0.1; // tea-list visits that end in an order
const orderValue = 20; // the average order, in pounds

for (const [name, more] of Object.entries(gain)) {
  const pounds = visits * more * orderShare * orderValue;
  console.log(`${name}: £${Math.round(pounds).toLocaleString("en-US")} a month`);
}

Answer, then press Check. Spacing and letter case don’t matter.

    Exercise 2 · Warm-upA made-up case study

    A made-up study says LCP went from 5 seconds to 2. What does the first line print?

    Starter codePop out in the code editor (opens in a new tab)JavaScript
    // A made-up case study, checked.
    const lcp = { before: 5, after: 2 }; // seconds
    const abandoned = { before: 4, after: 3 }; // % of loads
    const drop = (x) => x.before - x.after;
    
    console.log(`LCP: ${Math.round((drop(lcp) / lcp.before) * 100)}% less than before`);
    console.log(`LCP: ${Math.round((drop(lcp) / lcp.after) * 100)}% of the new value`);
    console.log(`Abandoned: ${drop(abandoned).toFixed(1)} points fewer`);
    console.log(`Abandoned: ${Math.round((drop(abandoned) / abandoned.before) * 100)}% fewer`);

    Answer, then press Check. Spacing and letter case don’t matter.

      Exercise 3 · PracticeNew phones in the middle

      In the month, how many new-phone visits had an LCP that needs improvement?

      Answer, then press Check. Spacing and letter case don’t matter.

        Exercise 4 · PracticeOrders at the high end

        At the high end of the test's interval, how many more orders a month does the fix bring?

        Answer, then press Check. Spacing and letter case don’t matter.

          Exercise 5 · ChallengeThe budget phone's wait

          Open the budget phone's trace in DevTools' Performance panel. In its LCP breakdown, which part takes longest?

          Answer, then press Check. Spacing and letter case don’t matter.

            Exercise 6 · ChallengeDeloitte's model

            In the full Milliseconds Make Millions report, what kind of regression model did the study use?

            Answer, then press Check. Spacing and letter case don’t matter.

              Check your understanding

              For each question, ask: what else differs, how was it measured, and what was assumed?

              The business case quiz · 7 questionsScore: first tries count
              1. Question 1 of 7In Ravi's month, good-LCP visits clicked far more often than poor ones. Does that show how much a faster page would add?

                Choose an answer to see the explanation.

              2. Question 2 of 7Why does Ravi compare good and poor visits inside one kind of phone?

                Choose an answer to see the explanation.

              3. Question 3 of 7Which number should Ravi use to price the home photo fix?

                Choose an answer to see the explanation.

              4. Question 4 of 7What does this made-up case study print?

                Read the code, then predictPop out in the code editor (opens in a new tab)JavaScript
                // A made-up case study: the bounce rate before and after, in %.
                const bounce = { before: 50, after: 40 };
                const drop = bounce.before - bounce.after;
                console.log(`${drop} points, or ${Math.round((drop / bounce.before) * 100)}% fewer`);

                Choose an answer to see the explanation.

              5. Question 5 of 7A page of case studies shows only wins. Why should that make you careful?

                Choose an answer to see the explanation.

              6. Question 6 of 7What goes first on Ravi's one-page case?

                Choose an answer to see the explanation.

              7. Question 7 of 7The fix brings about a month's worth of orders. Why doesn't Ravi just multiply by twelve and lead with a yearly figure?

                Choose an answer to see the explanation.

              Key takeaways

              • Your own RUM shows that speed and sales go together. It can't say how much speed causes.
              • Look for what else differs between fast and slow visits, like the phone, and compare like with like.
              • Price a fix from a fair test on your own visitors: the gain, times your visits, times what a visit is worth.
              • Give a range from the test's interval, and write down every assumption.
              • Read a study by how it measured, who paid, what its numbers mean in points, and which studies are missing.
              • Put the case on one page: the answer and its range, the cost, the proof, the ask, and the check.

              Remember the one-liner.
              Price speed from your own fair test, with a range, and read everyone else's numbers by how they were made.

              Sources

              The facts on this page come from these papers, studies and docs:

              Coming next: Search and Core Web Vitals. The owner's next question: will a faster page move the shop up Google's results?

              CompleteFrontend Clear concepts. Working examples.