The business case for performance
Show what speed is worth with your own data: link speed to sales without being fooled, read published studies critically, price a fix, and make the case.
- 01Link speed and sales in your own dataGroup real visits by LCP, find what else differs between fast and slow visits, and compare like with like.
- 02Read a published study criticallyAsk how it measured, who wrote it, what its numbers mean in points, and which studies never got published.
- 03Price a fix and make the caseTurn a fair test into pounds with a range, write down every assumption, and put the case on one page.
What is a faster page worth to us?
Ravi's test from lesson 40.3 is over. The faster home page won: 48% of phone visits clicked "See all our teas", against 33%. The owner is pleased. Then she asks two new questions.
"Clicks are nice. But what are they worth in pounds? And a friend sent me a study. It says every tenth of a second is worth millions. Is that true for us?"
Both questions are fair. The owner decides where Meera's hours go. So she needs a number she can trust, and she needs to know which numbers not to trust. This lesson builds Ravi's answer, one step at a time.
Ravi starts with what he already has. His RUM, short for real user monitoring, keeps a record of every phone visit to the home page. Each record holds the visit's LCP: the moment the biggest image or text appears. Here, that's the photo. The record also says whether the visitor clicked "See all our teas". That click is the home page's conversion: the thing the shop wants a visitor to do there.
Here is the month before the fix, in Google's three LCP groups:
// A month of phone visits to the home page: visits and clicks, by LCP.const month = [ { lcp: "good, up to 2.5 s", visits: 1835, clicks: 791 }, { lcp: "needs improvement", visits: 1177, clicks: 333 }, { lcp: "poor, over 4 s", visits: 588, clicks: 64 },]; for (const { lcp, visits, clicks } of month) { console.log(`${lcp}: ${Math.round((clicks / visits) * 100)}% clicked`);}- Line 1 is a comment.
- Lines 3 to 5 are recorded. Each holds one LCP group, its visits, and how many of them clicked. Google calls an LCP good up to 2.5 seconds, and poor over 4. Between the two, it "needs improvement".
- Line 8 goes through the groups. Line 9 prints the share of each group that clicked.
- It prints
good, up to 2.5 s: 43% clicked,needs improvement: 28% clickedandpoor, over 4 s: 11% clicked.
Good visits clicked about 4 times as often as poor ones. The owner reads the chart the quick way: "So a fast page brings 4 times the clicks?" Not quite. Some of that gap isn't speed at all. The next section finds out how much.
The friend's study has a different problem. It measured other people's sites, not the Leaf and Kettle.
You need a suit. A shop has one off the rack in size 40. It fits someone, but maybe not you. A tailor takes out a tape and measures your own shoulders. That takes longer, but the suit fits you.
- In real life: A suit off the rack, in someone's size
- In a business case: A number from a published study about another site
- In real life: The tailor's tape on your own shoulders
- In a business case: A fair test on your own visitors
- In real life: Trying it on before you pay
- In a business case: Checking the gain after the fix ships
A business case says what speed is worth to your shop, in money, with a range. Its number comes from a fair test on your own visitors. Your own month shows that speed and sales go together. Published studies show what happened somewhere else. Read both with care, and price from the test.
Here is Ravi's whole case in one picture:
The next four sections take one step each: your own month, other people's studies, the price, and the page.
How the course recorded it
- The page: Ravi's home page as it was before the fix, from lesson 40.1. It was served over HTTPS from the course's computer, at a made-up address. Each page sent its own beacon, as in lesson 40.3.
- The phones: three kinds, each with its own CPU speed and its own typical network. Chrome 157 loaded the page 100 times for each kind. Each load drew its own network around its kind's typical one, the way lesson 13.1 drew phones.
- The month: 3,600 phone visits, 120 a day for 30 days, as in lesson 40.3. A quarter came from new phones, half from mid-range phones, and a quarter from budget phones. Each visit took the LCP of one recorded load of its kind, picked at random.
- The test: every number about the fix comes from lesson 40.3's week.
The clicks come from a model, as in lesson 40.3. That lesson started every phone at a 50% chance to click. This lesson adds one thing: who owns the phone.
// The course's model: each kind of phone starts at its own chance.const start = { new: 0.6, "mid-range": 0.5, budget: 0.4 };function clickChance(kind, lcpMs) { return Math.max(0.05, start[kind] - 0.06 * (lcpMs / 1000));} console.log(`New phone, photo at 2 s: ${Math.round(clickChance("new", 2000) * 100)}%`);console.log(`Budget phone, photo at 2 s: ${Math.round(clickChance("budget", 2000) * 100)}%`);- Line 2 gives each kind of phone its own starting chance. In this model, owners of budget phones click less at any speed.
- Line 4 is lesson 40.3's rule: 6 points less for each second of waiting, and never below 5%.
- Lines 7 and 8 print
New phone, photo at 2 s: 48%andBudget phone, photo at 2 s: 28%. The photo came at the same moment, and the chances are still far apart.
The rule is the course's own. It keeps one thing known: speed takes 6 points a second, on every phone. So you can check each method against the true answer. Everything else is real: the page, the loads, the LCPs and the beacons.
Speed and clicks in your own data
Ravi's chart shows that speed and clicks go together. That's called a correlation. It doesn't say the speed caused the clicks. Lesson 1.2 met the reason. A hidden difference, called a confounder, can change both at once. Is there one in Ravi's month?
Who made the slow visits?
Start with what a visitor sees. Here is the home page as it was, on a new phone and on a budget phone, each on its typical network:
On the new phone, the photo is whole after 1.2 seconds. The budget phone needs 3.5. In the course's model, a budget phone also sits on a slower network, so its photo comes later. Now count the whole month, one kind of phone at a time:
// The same month, one kind of phone at a time.const phones = { new: { visits: 913, poor: 0, good: 895, goodClicks: 467 }, "mid-range": { visits: 1764, poor: 222, good: 814, goodClicks: 291 }, budget: { visits: 923, poor: 366, good: 126, goodClicks: 33 },}; for (const [kind, p] of Object.entries(phones)) { const poor = Math.round((p.poor / p.visits) * 100); const clicked = Math.round((p.goodClicks / p.good) * 100); console.log(`${kind}: ${poor}% poor; ${clicked}% of good visits clicked`);}- Lines 3 to 5 are recorded. Each holds one kind's visits, its poor visits, its good visits, and how many good visits clicked.
- Line 8 goes through the kinds. Line 9 works out the share of its visits that were poor.
- Line 10 works out the share of its good visits that clicked.
- It prints
new: 0% poor; 52% of good visits clicked,mid-range: 13% poor; 36% of good visits clickedandbudget: 40% poor; 26% of good visits clicked.
Two things stand out. First, budget phones made 62% of the month's poor visits, though they made only a quarter of all visits. Second, look at good visits only. On new phones, 52% of them clicked. On budget phones, 26%. The speed was good on both, and the clicks were far apart.
Why it matters: the poor group is full of budget phones, whose owners click less anyway. So the chart's gap holds two things at once: the wait, and who was waiting.
A shop notices that shoppers with a basket spend three times as much as shoppers without one. Should it hand everyone a basket? No. People who came for a big shop pick up a basket. People who popped in for milk don't. The basket and the big spend both come from the reason for the trip.
- In real life: Carrying a basket
- In a business case: A visit with a good LCP
- In real life: Coming in for a big shop
- In a business case: Owning a new phone, whose owner clicks more anyway
- In real life: Handing every shopper a basket
- In a business case: Making the page faster for everyone
- In real life: Comparing big shoppers with and without a basket
- In a business case: Comparing good and poor visits on one kind of phone
Compare like with like
So compare visits on the same kind of phone. Then the phone can't make the gap. Use lesson 40.3's sum: the gap in clicks between good and poor visits, over the gap in their mean LCP. The mean is the average. The sum gives points per second. A point is one percent: 40% minus 6 points is 34%.
// Good and poor visits: the share that clicked, and their mean LCP in seconds.const all = { good: { clicked: 0.431, lcp: 1.61 }, poor: { clicked: 0.109, lcp: 5.45 } };const midRange = { good: { clicked: 0.357, lcp: 1.96 }, poor: { clicked: 0.167, lcp: 5.21 } };const budget = { good: { clicked: 0.262, lcp: 2.24 }, poor: { clicked: 0.074, lcp: 5.59 } }; const perSecond = ({ good, poor }) => ((good.clicked - poor.clicked) / (poor.lcp - good.lcp)) * 100;console.log(`All phones: ${perSecond(all).toFixed(1)} points per second`);console.log(`Mid-range phones: ${perSecond(midRange).toFixed(1)} points per second`);console.log(`Budget phones: ${perSecond(budget).toFixed(1)} points per second`);- Lines 2 to 4 are recorded. For all phones, then mid-range and budget phones, each holds two groups. Each group has the share that clicked, and its mean LCP in seconds.
- Line 6 is the sum: the drop in clicks, over the extra seconds, in points.
- Lines 7 to 9 print
All phones: 8.4 points per second,Mid-range phones: 5.8 points per secondandBudget phones: 5.6 points per second.
Across all phones, a second seems to cost 8.4 points. Inside one kind of phone, it costs 5.8 and 5.6. New phones aren't on the list. None of their 913 visits had a poor LCP, so there's nothing to compare.
Why it matters: mixing the phones made each second look costlier than it is. A price built on the mixed number would promise too much. Try each kind of phone yourself:
- Visits in the month3,600
- Good LCP, up to 2.5 s1835 visits, 791 clicked (43%)
- Needs improvement1177 visits, 333 clicked (28%)
- Poor LCP, over 4 s588 visits, 64 clicked (11%)
- Points per second, good against poor8.4
Each row counts the month's visits in one of Google's LCP groups, and how many of them clicked. The last row is the lesson's sum: the gap in clicks between good and poor visits, over the gap in their mean LCP.
What the fair test said
Splitting by phone took out one hidden difference. Others can still hide in the month: the network, returning visitors, the time of day. You can only split by what you recorded.
Ron Kohavi's guide to experiments on the web says it plainly. Patterns found by mining data "are correlational". A test that splits visitors by chance is different: it spreads the confounders "equally over all values through random assignment". That includes the ones nobody recorded.
Ravi already has such a test: lesson 40.3's week. There, the fix brought the photo 2.7 seconds sooner on average. It added 14.3 clicks per 100 visits. That's 5.3 points a second. Here, the course's model knows the true answer, so put them side by side:
- The month, all phones mixed: 8.4 points a second.
- The month, one kind of phone at a time: 5.8 and 5.6.
- The fair test: 5.3.
- The model's own rule: 6.
The mixed month is the furthest off. Why it matters: use your month to find where to look. Use a fair test to say what a fix is worth.
Read published studies with care
Now the owner's second question: her friend's study. Lesson 1.2 checked where some famous numbers come from. This time, read a whole study. Ask it five questions:
- How did it measure? A split by chance, a before and after, or fast and slow compared?
- Who wrote it, and who paid?
- What exactly did it measure? Which speed number, and which business number?
- How big is the change, in points as well as in percent?
- Which studies are missing?
The friend's study
The friend's study is "Milliseconds Make Millions". Lesson 1.2 met its headline number. Its full report answers the five questions itself:
- How: the report says it "examined the correlation between mobile site speed and mobile site user behaviours". Speed went up and down by itself. Nobody changed it on purpose.
- Who: Deloitte wrote it, with data from a firm called Fifty-Five. The report "has been commissioned by Google". It also says Deloitte "did not audit nor validate this data".
- What: each hour's speed came from Lighthouse, a lab tool, not from each visitor's own load. The report says this was to set aside "connection type, devices used, etc." So the visitors' own phones and networks aren't in its speed numbers.
- Missing: over 70 brands were asked, and 37 qualified. A result counted only if it reached "95% statistical significance". Results for computers were left out: the report found "a lot of contradicting parameters".
None of this makes the study wrong. It makes it a correlation across 37 other sites. Why it matters: it can say speed and sales moved together there. It can't say what a faster photo is worth at the Leaf and Kettle.
A study that warns you itself
Contentsquare sells tools that record visits. It grouped visits on 997 customer sites by their INP: how fast a page answers a tap. In retail, visits with a good INP converted at 2.5%, and the rest at 2.0%. It calls that "a 25% better conversion rate". Then it warns: "Correlation does not equal causation."
Its own numbers show why. Split by device, computers converted at 4.2% and 2.9%. Phones converted at 1.9% and 1.7%. It also says phones had worse INP. So the good-INP group holds more computers, which convert more anyway. That's Ravi's month again, with computers in place of new phones. Yet its last section says INP "has a significant impact". Why it matters: read the method, not just the ending.
One study, two kinds of number
Rakuten 24's study, from lesson 40.3, holds both kinds. First, it compared real visits by their LCP. On its home page, it says a good LCP "can lead to" up to 61.13% more conversions. Then it ran an A/B test on one landing page for a month. The faster version converted 33.13% better.
The pages differ, so the two numbers don't line up exactly. But only the second came from a split by chance. Why it matters: when a study gives both kinds, quote the test.
Before and after
Many case studies compare the numbers before a speed project with the numbers after it. That's easier than a test. But everything else that changed comes along too:
- NDTV, a news site, made its LCP 55% better. Its bounce rate fell by half. The study adds a note: NDTV "made other product changes" at the same time. So "it is not possible to conclusively say" that speed was the only cause.
- Agrofy, a farm marketplace, moved to a new framework, then spent a month on speed. Its LCP got 70% better. Its load abandonment on product pages fell 76%.
Now check the percentages. A change can be measured against the start or the end. A rate can fall by points or by percent:
// Two numbers from published case studies, checked.const lcp = { before: 4.5, after: 2.5 }; // secondsconst abandoned = { before: 3.8, after: 0.9 }; // % of loadsconst drop = (x) => x.before - x.after; console.log(`LCP: ${Math.round((drop(lcp) / lcp.before) * 100)}% less than before`);console.log(`LCP: ${Math.round((drop(lcp) / lcp.after) * 100)}% of the new value`);console.log(`Abandoned: ${drop(abandoned).toFixed(1)} points fewer`);console.log(`Abandoned: ${Math.round((drop(abandoned) / abandoned.before) * 100)}% fewer`);- Line 2 is The Economic Times' LCP, in seconds. Its study says LCP "improved by 80% from 4.5 seconds to 2.5 seconds".
- Line 3 is Agrofy's load abandonment: from 3.8% of loads to 0.9%.
- Line 4 works out the drop: before, minus after.
- Lines 6 and 7 print
LCP: 44% less than beforeandLCP: 80% of the new value. - Lines 8 and 9 print
Abandoned: 2.9 points fewerandAbandoned: 76% fewer.
The Economic Times' 80% is the drop measured against the new value, 2.5 seconds. Against where it started, LCP fell 44%. Agrofy's 76% is true too. But it's 2.9 points of a small number. Why it matters: work out the points before you repeat a percent.
The studies you never see
A restaurant tapes its best reviews to the window. Every one of them is real. But the one-star reviews stay inside. So the window can't tell you how often a meal goes wrong.
- In real life: The reviews taped to the window
- In a business case: Published case studies
- In real life: The one-star reviews kept inside
- In a business case: Speed projects that changed nothing, never written up
- In real life: The owner picks which ones to show
- In a business case: A page that asks for "a compelling business win"
- In real life: Trying the food yourself
- In a business case: A test on your own visitors
web.dev, Google's site for web developers, has a page on the business impact of Core Web Vitals. It ends with an invitation: "If you also have a compelling business win", you can send it in. Wins get written up. Projects that changed nothing rarely do.
How often do changes win? Kohavi's team counted at Microsoft. Of well-run experiments built to improve a key number, "only about one-third were successful". So a page of wins isn't a fair sample of what speed work does. Why it matters: a published win shows what can happen, not what usually happens.
One more place to look: WPO Stats collects one-paragraph summaries of performance case studies, each with a link. Use it as an index. Then open the study itself, and ask the five questions. Try them on six studies:
- How it measuredSplit by chance
- Who wrote itweb.dev, Google's site for web developers, with Rakuten 24
- What it measuredA landing page for a month, with half the visitors on each version.
- What it saysA 33.13% higher conversion rate on the faster version.
- In pointsNot given: only the relative change.
- What you can sayA fair test. Its size belongs to that page and those visitors.
Each row answers one question about the study, from its own pages. Read the first row before you read the numbers.
Read this table across: each way to link speed and money, what it can show, and what can fool it.
| Way | What it can show | What can fool it |
|---|---|---|
| Compare fast and slow visits | That speed and sales go together, on many visits at once. | Fast visits come from different visitors: newer phones, better networks. They may buy more anyway. |
| Compare before and after a change | That the numbers moved after the work shipped. | Everything else that changed in between: seasons, sales, a redesign, a new framework. |
| Split visitors by chance | What the change itself is worth, on those visitors. | Too few visitors, uneven groups, or looking too early, as lesson 40.3 showed. |
Put a price on the fix
Back to the owner's first question: what is the fix worth in pounds? Ravi needs three numbers, all his own:
- Visits: 3,600 phone visits to the home page a month.
- The gain: lesson 40.3's 14.3 more clicks per 100 visits. Its 90% interval runs from 7.6 to 20.7.
- What a click is worth: his shop's analytics say one tea-list visit in ten ends in an order. The average order is £18.
// The home photo fix, priced: lesson 40.3's test, and Ravi's own numbers.const visits = 3600; // phone visits to the home page in a monthconst gain = { low: 0.076, middle: 0.143, high: 0.207 }; // more clicks per visitconst orderShare = 0.1; // tea-list visits that end in an orderconst orderValue = 18; // the average order, in pounds for (const [name, more] of Object.entries(gain)) { const pounds = visits * more * orderShare * orderValue; console.log(`${name}: £${Math.round(pounds).toLocaleString("en-US")} a month`);}- Line 2 is the month's phone visits.
- Line 3 is the test's gain in clicks per visit: the low end of its interval, the middle, and the high end.
- Line 4 says one tea-list visit in ten ends in an order. Line 5 is the average order, in pounds.
- Line 8 multiplies them, one estimate at a time. Line 9 prints each one, rounded to a pound.
- It prints
low: £492 a month,middle: £927 a monthandhigh: £1,341 a month.
Before you book a taxi, the app shows a fare of "£14 to £18". It knows the distance. It can't know the traffic yet. So you bring £18, and you're glad when it costs £14.
- In real life: The distance
- In a business case: The 3,600 visits a month
- In real life: The traffic nobody knows yet
- In a business case: The test's chance error, its 90% interval
- In real life: "£14 to £18"
- In a business case: The low end to the high end, a month
- In real life: Bringing enough for the worst fare
- In a business case: Deciding with the low end
Even the low end, £492 a month, is more than twice what the fix cost. Meera's 5 hours cost the shop £200, at £40 an hour. The middle estimate pays that back in about 6 days. Why it matters: when even the low end is a clear yes, the owner doesn't need to trust the middle.
Write down every assumption
Each number in the sum stands for something Ravi assumed. He writes them down, so the owner can question them:
- Phones only. Computers weren't in the test, so they add nothing here.
- New clicks order as often as today's. If the extra visitors are less keen, the value is lower.
- The gain lasts. Kohavi's guide notes that a test measures the effect "during the experimentation period, typically a few weeks." So Ravi says "a month", and checks it each month.
- The test can miss. A 90% interval misses one time in ten, as lesson 40.3 showed.
What the month would have promised
What if Ravi had priced the fix from his month instead? The month says 8.4 points a second. The fix saved 2.7 seconds. So the month promises 22.7 more clicks per 100 visits: £1,471 a month. Try each estimate:
- Where the gain comes fromThe gain the test measured
- More clicks per 100 visits14.3
- More clicks a month515
- More orders a month, 1 in 1051
- More sales a month, at £18 an order£927
- Twelve months, if the gain lasts£11,120
Each row is one step of the lesson's sum: 3,600 phone visits a month, times the gain, times 1 order in 10, times £18.
The month's price is 59% above the test's middle, and above even its high end. Why it matters: promise what the test showed. Then the monthly check will back you up.
Make the case on one page
The owner wants a decision she can make in a minute. Lesson 40.2 gave Ravi's audit report its order: the answer first, then the proof, then the method. A business case uses the same order, on one page.
You ask the bank for a loan for a new coffee machine. The manager wants four things: how much, what it will earn, your proof, and what happens if sales are slow. Bring all four on one sheet, and the meeting takes ten minutes.
- In real life: How much you want
- In a business case: The ask: Meera's time
- In real life: What the machine will earn
- In a business case: The value, with its range
- In real life: Your bank statements
- In a business case: The test on your own visitors
- In real life: "What if sales are slow?"
- In a business case: The low end, and the assumptions
Here is Ravi's page. Every number comes from the sums above:
# Speed pays: the faster home photoAnswer: it brings about £930 of tea orders a month, between £490 and £1,340.Cost: 5 hours of Meera's work, £200. It paid that back in about 6 days.Evidence: our own test of 570 phone visits, split by chance: 14 more clicks on our teas per 100 visits.Sum: 3,600 phone visits a month × 14.3 more clicks per 100 × 1 order in 10 × £18 an order.Assumptions: phones only; new clicks order as often as today's; the gain lasts.Not used: our month of visits, which says 8.4 points a second. Budget phones make most slow visits.Ask: one day of Meera's time a month for speed, each fix tested like this one.Check: phone orders each month, and the next test's result.- Line 1 is the title. It says the answer, not just the topic.
- Line 2 is the answer, with its range. Money is rounded to £10, because it's an estimate.
- Line 3 is the cost, and how fast the fix paid it back.
- Line 4 is the evidence: a test on the shop's own visitors.
- Line 5 shows the sum, so anyone can redo it.
- Line 6 lists the assumptions.
- Line 7 says what Ravi left out, and why.
- Line 8 is the ask: what Ravi needs from the owner.
- Line 9 says how everyone will know it worked.
Notice what isn't there: no famous study, and no "speed is worth millions". If the owner asks about her friend's study, Ravi's answer is ready. It measured other sites, and it measured a correlation. Why it matters: a case built on your own test still stands after the owner's questions.
Two readers, one page
Lesson 40.2 wrote for two readers, and so does this page. The owner reads lines 2, 3 and 8, and decides. Meera reads lines 4 to 7, and checks the work. Both read numbers in pounds and orders, not points per second. Those stay in Ravi's notes, with a link to the test.
After the yes
The case ends with a check, so Ravi checks. His RUM already counts phone visits and clicks. Each month, his report gets one more line: phone orders this month, against the month before the fix. The next fix gets its own test, the way lesson 40.3 ran one. Why it matters: each tested fix adds proof, so the next case is easier to make.
How did this study measure?
Each line below describes one study from this course. Sort it by how it measured:
- Ravi's week: the edge picks each phone visitor's page with a coin
- Rakuten 24: half the visitors get the faster landing page for a month
- Google holds its results back 400 ms for one group of users
- NDTV: bounce rate before and after a month of LCP work
- Yahoo! JAPAN News: page views per session before and after fixing layout shifts
- Agrofy: load abandonment across a new framework and a month of speed work
- Ravi's month: good-LCP visits against poor-LCP visits
- Milliseconds Make Millions: hourly speed against each brand's analytics, on 37 sites
- Contentsquare: visits with a good INP against visits with a poor one
Sort each study by how it measured.
Common mix-ups
- "Good-LCP visits clicked far more often, so a fast page multiplies clicks." No. Budget phones made most of the poor visits, and their owners click less at any speed. Inside one kind of phone, the gap shrinks.
- "Splitting by device removes every hidden difference." No. It removes only the one you split by. A split by chance balances the ones you never recorded, too.
- "A study from a famous firm is a fair test." No. Milliseconds Make Millions is a correlation over natural changes in speed, as its own report says.
- "76% fewer means the rate fell by 76." No. Agrofy's rate fell from 3.8% to 0.9%: 2.9 points.
- "Published case studies show what speed work usually does." No. Wins get written up. At Microsoft, only about a third of well-run experiments improved their key number.
- "The value of a fix is one number." No. Give the range from the test's interval, and the assumptions behind it.
- "Multiply a month by twelve for the year." No. A test measures the weeks it ran. A year assumes the gain lasts, so say so, and check it.
Practice exercises
Answer first, then use the hints.
The average order grows to £20. What does the middle line print?
// The home photo fix, priced: lesson 40.3's test, and Ravi's own numbers.
const visits = 3600; // phone visits to the home page in a month
const gain = { low: 0.076, middle: 0.143, high: 0.207 }; // more clicks per visit
const orderShare = 0.1; // tea-list visits that end in an order
const orderValue = 20; // the average order, in pounds
for (const [name, more] of Object.entries(gain)) {
const pounds = visits * more * orderShare * orderValue;
console.log(`${name}: £${Math.round(pounds).toLocaleString("en-US")} a month`);
}middle: £1,030 a month. A bigger order makes every estimate bigger by the same share: 20 is 18 plus a ninth.
A made-up study says LCP went from 5 seconds to 2. What does the first line print?
// A made-up case study, checked.
const lcp = { before: 5, after: 2 }; // seconds
const abandoned = { before: 4, after: 3 }; // % of loads
const drop = (x) => x.before - x.after;
console.log(`LCP: ${Math.round((drop(lcp) / lcp.before) * 100)}% less than before`);
console.log(`LCP: ${Math.round((drop(lcp) / lcp.after) * 100)}% of the new value`);
console.log(`Abandoned: ${drop(abandoned).toFixed(1)} points fewer`);
console.log(`Abandoned: ${Math.round((drop(abandoned) / abandoned.before) * 100)}% fewer`);LCP: 60% less than before. Against the new value, the same drop is 150%: that's why the base matters.
In the month, how many new-phone visits had an LCP that needs improvement?
18. Most new-phone visits were good, a few needed improvement, and almost none were poor.
At the high end of the test's interval, how many more orders a month does the fix bring?
75. 745 more clicks a month, and one in ten orders.
Open the budget phone's trace in DevTools' Performance panel. In its LCP breakdown, which part takes longest?
Resource load duration: downloading the photo itself. On a budget phone's network, the big photo takes most of the wait.
In the full Milliseconds Make Millions report, what kind of regression model did the study use?
A logarithmic regression model. It looks for numbers that move together with speed. It doesn't change anyone's speed.
Check your understanding
For each question, ask: what else differs, how was it measured, and what was assumed?
Question 1 of 7In Ravi's month, good-LCP visits clicked far more often than poor ones. Does that show how much a faster page would add?
Choose an answer to see the explanation.
Question 2 of 7Why does Ravi compare good and poor visits inside one kind of phone?
Choose an answer to see the explanation.
Question 3 of 7Which number should Ravi use to price the home photo fix?
Choose an answer to see the explanation.
Question 4 of 7What does this made-up case study print?
Read the code, then predict// A made-up case study: the bounce rate before and after, in %. const bounce = { before: 50, after: 40 }; const drop = bounce.before - bounce.after; console.log(`${drop} points, or ${Math.round((drop / bounce.before) * 100)}% fewer`);Choose an answer to see the explanation.
Question 5 of 7A page of case studies shows only wins. Why should that make you careful?
Choose an answer to see the explanation.
Question 6 of 7What goes first on Ravi's one-page case?
Choose an answer to see the explanation.
Question 7 of 7The fix brings about a month's worth of orders. Why doesn't Ravi just multiply by twelve and lead with a yearly figure?
Choose an answer to see the explanation.
Key takeaways
- Your own RUM shows that speed and sales go together. It can't say how much speed causes.
- Look for what else differs between fast and slow visits, like the phone, and compare like with like.
- Price a fix from a fair test on your own visitors: the gain, times your visits, times what a visit is worth.
- Give a range from the test's interval, and write down every assumption.
- Read a study by how it measured, who paid, what its numbers mean in points, and which studies are missing.
- Put the case on one page: the answer and its range, the cost, the proof, the ask, and the check.
Remember the one-liner.
Price speed from your own fair test, with a range, and read everyone else's numbers by how they were made.
Sources
The facts on this page come from these papers, studies and docs:
- Practical Guide to Controlled Experiments on the Web, Ron Kohavi, Randal M. Henne and Dan Sommerfield.
- Online Controlled Experiments and A/B Tests, Ron Kohavi and Roger Longbotham.
- Online Experimentation at Microsoft, Ron Kohavi, Thomas Crook and Roger Longbotham.
- Milliseconds Make Millions (the full report), Deloitte Ireland, commissioned by Google.
- Milliseconds make millions, web.dev.
- Interaction to next paint: business value, Contentsquare.
- The business impact of Core Web Vitals, web.dev.
- NDTV achieved a 55% improvement in LCP by optimizing for Core Web Vitals, web.dev.
- Agrofy: A 70% improvement in LCP correlated to a 76% reduction in load abandonment, web.dev.
- How CLS optimizations increased Yahoo! JAPAN News's page views per session by 15%, web.dev.
- How The Economic Times passed Core Web Vitals thresholds and achieved an overall 43% better bounce rate, web.dev.
- Rakuten 24's investment in Core Web Vitals, web.dev.
- WPO Stats, an index of performance case studies.
- Largest Contentful Paint (LCP), web.dev.
Coming next: Search and Core Web Vitals. The owner's next question: will a faster page move the shop up Google's results?