Lab data vs field data

Two ways to measure a page

Ravi looks after the Leaf and Kettle website. In lesson 1.3, he learned not to judge the teas page on his fast laptop. So now he tests it much the way Lighthouse, the test tool from lesson 1.1, tests a phone. His laptop pretends to be a mid-range phone on a slow mobile network. The browser starts with no saved files, as if it had never seen the page.

The teas appear after 2,228 ms. That's the page's LCP, Largest Contentful Paint: the moment the main content appears. It's under LCP's good limit of 2.5 seconds, from lesson 3.1. Ravi is pleased.

Then he wonders: is that what his visitors get? The same morning, other people open the same page. Someone on a new phone. A regular who has visited before. And Asha, from lesson 1.3, on her cheap phone on the bus.

The course can't watch your real visitors. So it did the next best thing. It loaded the same page four times in Chrome 156, each time set up like one of these people. Here is when the teas appeared each time:

1// When the teas appeared, in ms, in four loads of one page2const loads = [3  { who: "Ravi's test", ms: 2228 },4  { who: "A visitor on a new phone", ms: 564 },5  { who: "A regular, back again", ms: 808 },6  { who: "Asha on her cheap phone", ms: 7628 },7];8 9for (const load of loads) {10  console.log(`${load.who}: the teas appeared at ${load.ms} ms.`);11}

• Line 1 is a comment. Times are in milliseconds (ms), thousandths of a second. Each one counts from the moment that person opened the page.
• Line 3 is Ravi's test: a mid-range phone, a slow network, and no saved files.
• Lines 4 to 6 are three visitors. One has a new phone on a fast network. One is a regular whose phone kept the page's files. And Asha has a cheap phone on a slow 3G network.
• Lines 9 to 11 print one sentence per load. The first two are Ravi's test: the teas appeared at 2228 ms. and A visitor on a new phone: the teas appeared at 564 ms.
• The last two are A regular, back again: the teas appeared at 808 ms. and Asha on her cheap phone: the teas appeared at 7628 ms.

Same page, same code. Yet the teas took anywhere from about half a second to more than seven and a half seconds. Here is what each screen showed two seconds after its visitor opened the page:

At the same moment, two visitors were already reading about teas. Ravi's test showed only the shop's name. Asha was still looking at a blank screen.

So which number is the page's speed? Each one is true, for one load. But Ravi's number and the visitors' numbers come from two different kinds of measuring. They have names:

• Lab data comes from a load you run yourself, under conditions you choose. One such load is a lab test. Ravi's test is lab data.
• Field data comes from real visits, on the visitors' own phones and networks. The three visitors in the picture stand in for it.

The conditions are everything about a load except the page itself. That means the device, the network, the place it runs from, and the cache. The cache is the browser's store of files it saved on earlier visits.

You've met both kinds before. Lesson 1.1 compared them to a café owner's stopwatch and the till's record of every wait. Lesson 3.1 compared them to a bus timetable and the real waits at the stops. Here is why each kind is good at something different:

The lab's strength is that it repeats. The course ran Ravi's test five times. The teas appeared between 2,216 ms and 2,264 ms every time. Lesson 1.3 recorded the same test on another day, and got 2,212 ms. Why it matters: when a change to the page moves a lab number, you know the change did it. Lesson 9.4, Noise and variability, shows how much lab runs can still differ.

The field's strength is that it's real. It's what visitors got, on the phones and networks they really have. That's why lesson 3.1 judged a page by its field data, at the 75th percentile.

Read each row of this table across: one question, answered for lab data and for field data.

Synthetic testing vs real user monitoring

Where does each kind of data come from? Each has its own way of measuring, with its own name.

Synthetic testing makes lab data

Synthetic testing means a tool makes the visit itself. It loads the page as if it were a visitor, under the conditions you set. Synthetic means made, not natural: no real person is involved. Three tools do it:

• Lighthouse runs in Chrome's DevTools, in PageSpeed Insights, and from the command line. PageSpeed Insights is Google's website for testing any public page. Module 8 covers Lighthouse.
• WebPageTest is another testing tool. Module 9 covers it.
• DevTools' Performance panel records one load in full detail. Lesson 7.1, The Performance panel, tours it.

Nobody has to visit, so you can run these tests before release, on every change, or every hour. That's the forecast: a look at a load before real people make it.

Real user monitoring collects field data

Real user monitoring, or RUM, is the rain gauge. A small script on your page measures each visit in the visitor's own browser. Then it sends the numbers to you. web.dev says field data and RUM data are two names for the same thing.

The script has two parts. The first part measures LCP, with the browser tool the course has used since lesson 1.4:

1let lcp;2new PerformanceObserver((list) => {3  lcp = list.getEntries().at(-1).startTime;4}).observe({ type: "largest-contentful-paint", buffered: true });

• Line 1 makes a variable to hold the latest LCP.
• Line 2 makes a PerformanceObserver: an object the browser calls each time it records a new entry of the kind you ask for.
• Line 3 keeps the time of the newest entry. The browser reports a new LCP entry each time something bigger is drawn, so the last one is the LCP.
• Line 4 asks for LCP entries. buffered: true also hands over the entries from before the script ran.

The second part sends the number home when the visitor leaves:

1document.addEventListener("visibilitychange", () => {2  if (document.visibilityState === "hidden" && lcp !== undefined) {3    navigator.sendBeacon("/analytics", JSON.stringify({ page: location.pathname, lcp }));4  }5});

• Line 1 listens for visibilitychange, an event that fires when the page is hidden or shown again.
• Line 2 checks that the page is now hidden. The visitor switched tabs, went to another app, or left. Chrome's guide to the page lifecycle says hidden is often the last change a page can reliably see. So it's the moment to send.
• Line 3 sends the page's path and its LCP with navigator.sendBeacon(). It hands the browser a small message to deliver, without holding up the page as it closes.

Real visits have cases this small script misses. A page can open in a background tab. It can come back from memory when the visitor presses Back. The web-vitals library, a small library from Chrome's team, handles cases like these. Module 12 builds a whole RUM setup, starting with lesson 12.1, The web-vitals library.

Chrome's own field data

Chrome collects field data too, from some of its users. It's called the Chrome User Experience Report, or CrUX. PageSpeed Insights shows it next to its Lighthouse result. Google Search Console shows it too. It's real, but it isn't everyone:

• It comes from Chrome on computers and on Android.
• It only counts people who share usage statistics with Google and sync their browsing history.
• Chrome on iPhones, Android apps that show web pages inside them, and other browsers such as Edge send nothing to it.
• Only public pages with enough visitors are in it. It covers the last 28 days.

PageSpeed Insights' FAQ calls its field data "a historical report about how a particular URL has performed." That's the rain gauge: it records what already happened. Lesson 10.1, The Chrome User Experience Report, opens it up.

Your visit is field data too

Your own visit to this page is one field sample: one real load, on your device and your network. Measure it:

1const [navigation] = performance.getEntriesByType("navigation");2const files = performance.getEntriesByType("resource").filter((file) => file.decodedBodySize > 0);3const cached = files.filter((file) => file.transferSize === 0);4 5const visit = {6  ttfb: navigation.responseStart,7  lcp: latestLcp.startTime,8  fromCache: `${cached.length} of ${files.length} files`,9  screen: `${innerWidth} × ${innerHeight}`,10  cores: navigator.hardwareConcurrency,11  network: navigator.connection?.effectiveType,12};

• Line 3 of the panel's code counts the files that came from your cache. A file from the cache crossed no network, so its transferSize is 0.
• Lines 9 and 10 read two of your conditions: the window's size and the number of CPU cores. A core is one worker inside the processor, and more cores can do more at once.
• Line 11 reads the browser's guess at your network. Only Chromium browsers have it. Chromium is the browser code Chrome and Edge are built on.

Why your Lighthouse score isn't your users' experience

Lighthouse gives a page a score from 0 to 100. A green score, 90 or more, looks like a verdict on your page. But it's a verdict on one load, under these settings:

1const defaultSettings = {2  formFactor: "mobile",3  screenEmulation: { mobile: true, width: 412, height: 823, deviceScaleFactor: 1.75 },4  throttlingMethod: "simulate",5  throttling: { rttMs: 150, throughputKbps: 1.6 * 1024, cpuSlowdownMultiplier: 4 },6  disableStorageReset: false,7};

• Line 2 says Lighthouse tests as a phone, unless you ask for a computer.
• Line 3 is the phone's screen: 412 by 823 CSS pixels, the size of a Moto G Power. Chrome emulates it: it pretends to have that screen. It doesn't become that phone.
• Line 4 picks how to slow the load down. Throttling means slowing something down on purpose. "simulate" means Lighthouse loads the page at full speed. Then it works out how long the load would have taken on a slower network and CPU.
• Line 5 is that slower network. A round trip is the time for a message to reach the server and for the answer to come back: here, 150 ms. The download speed is 1.6 megabits per second. Lighthouse's docs say that's roughly the slowest quarter of 4G connections and the fastest quarter of 3G.
• Line 5 also slows the CPU. The CPU is the processor that runs the page's code. Here it runs 4 times slower than on the computer doing the test.
• Line 6 leaves the reset on. So Lighthouse clears the cache before each run, and every run is a first visit.

Read each row across: one setting, in Lighthouse's default run and for your real users.

So one Lighthouse run is like one taste from a big pot:

web.dev shows a real PageSpeed Insights report where the two disagreed. The lab measured an LCP of 3.0 seconds. The field said 1.8 seconds, at the 75th percentile. Here is a made-up page with the same shape: one lab load, and ten field visits.

1const labLcp = 3000;2const fieldLcps = [600, 700, 800, 900, 1000, 1300, 1600, 1800, 3200, 4800];3 4function percentile(values, p) {5  const sorted = [...values].sort((a, b) => a - b);6  const rank = Math.ceil((p / 100) * sorted.length);7  return sorted[rank - 1];8}9 10console.log("lab:", labLcp, "ms from one load");11console.log("field:", percentile(fieldLcps, 75), "ms at the 75th percentile of", fieldLcps.length, "visits");

• Line 1 is the lab: one load, so one LCP.
• Line 2 is the field: ten visits' LCPs. Most are fast, and a few are slow.
• Lines 4 to 8 are the percentile function from lesson 3.1. It sorts the values, then takes the one p% of the way along.
• Line 10 prints lab: 3000 ms from one load.
• Line 11 prints field: 1800 ms at the 75th percentile of 10 visits.

The lab says 3 seconds. That's over LCP's good limit of 2.5, so it needs improvement. The field says 1.8 seconds: good. Neither is wrong. web.dev calls the lab's number "still a valid LCP value for this page". It's one possible visit, and slower than most.

PageSpeed Insights' own FAQ agrees. Any green score is good, it says, but "having good lab data does not necessarily mean real-user experiences will also be good." Its report shows both: CrUX's field data, and the Lighthouse result.

The gap goes both ways. Your visitors may have faster phones and full caches, like the regular. Or slower phones and worse networks, like Asha. Why it matters: use the score to compare versions of your page under the same settings. Use field data to judge what visitors get. Module 8 takes Lighthouse apart, starting with How Lighthouse works.

Cache, device, network, and interaction differences

Back to the four screens from the start of the lesson. Same page, same code, same browser. So what made them so different? Each visitor differed from Ravi's test in one or two ways. Read each row across: one recorded load, and its conditions.

All four were real loads in Chrome 156, on a phone-sized screen. The network names are DevTools' presets from lesson 1.1. "4 times slower" means the CPU was slowed down 4 times, as in Lighthouse. The new phone and Asha's phone come from lesson 1.3's recordings. Ravi's test and the regular were recorded for this lesson.

The cache

Ravi's test and the regular had the same phone and the same network. Only one thing was different. The regular's phone had loaded the page before.

The browser's cupboard is its HTTP cache: the store of files it saved from earlier visits. When the page asks for a file it saved, the browser can use its copy instead of downloading it again. Here is what that changed for the regular:

• styles.css and app.js came from the cache. Each one moved 0 bytes over the network.
• The teas appeared at 808 ms, against 2,228 ms in Ravi's test.
• app.js still had to run. A long task is a piece of work that keeps the browser busy for more than 50 ms, from lesson 3.2. Running app.js was one: 158 ms for the regular, and 160 ms in Ravi's test.

So the cache saved the walk to the shop, not the cooking. Why it matters: a full cache makes downloads disappear, but a big script still runs on every visit. And lab tests usually start with an empty cache. web.dev says "it's not possible for a lab tool to know what percentage of real-world visits occur from new versus returning users."

There's a second kind of cache. When you press the Back button, the browser can show the page you just left straight from memory, almost at once. That's the back/forward cache. web.dev says those visits are in your field data. Lab tests don't include them.

Devices and networks

The new phone and Asha's phone loaded exactly the same files. The teas came at 564 ms and at 7,628 ms: 13.5 times as long. Lesson 1.3 showed where that time went: a slower network and a slower CPU. A lab test picks one device and one network. Your visitors bring all of them.

Even the screen matters. LCP times the biggest text or picture on the screen, called the LCP element. A bigger screen shows more. So a visitor on a big computer screen can have a different LCP element than a lab test on a phone. Other things change what's on the screen too:

• Personalized content: parts of the page made for one visitor, like their name or their own suggestions.
• A/B tests: two versions of a page, shown to different visitors to compare them.
• The fonts installed on the visitor's device, which change how big the text is.
• Links to the middle of a page, so the visitor starts further down.

Interactions

A default lab test loads the page and waits. Nobody scrolls, taps or types. Real people do, and each Core Web Vital notices:

• LCP can come earlier in the field. The browser stops looking for a bigger element as soon as the visitor taps, scrolls or presses a key.
• INP, Interaction to Next Paint, measures how soon the page responds to a tap, click or key press. A default lab test has none. So lab tools show Total Blocking Time (TBT) instead: how long the page was too busy to respond while it loaded.
• TBT and INP can disagree. The slow store from lesson 1.4 had a good TBT, 154 ms. Yet its Add to cart click took 296 ms, over INP's good limit of 200 ms. Only a click shows that.
• CLS, Cumulative Layout Shift, counts how much things jump around. In the lab, it only sees jumps during the load, on the first screen. In the field, it counts the whole visit. Images without a set size, further down, can push text around as the visitor scrolls.

Who visits decides the field's number

So the field's number depends on who visits. A shop full of regulars gets a fast 75th percentile. A shop full of slow phones gets a slow one. Here are the four recorded loads, mixed into 100 visits:

1const visitors = [2  { who: "new phone", lcp: 564, share: 20 },3  { who: "regular, back again", lcp: 808, share: 60 },4  { who: "first visit", lcp: 2228, share: 15 },5  { who: "cheap phone on 3G", lcp: 7628, share: 5 },6];7 8function p75(visitors) {9  const sorted = [...visitors].sort((a, b) => a.lcp - b.lcp);10  let seen = 0;11  for (const visitor of sorted) {12    seen += visitor.share;13    if (seen >= 75) return visitor.lcp;14  }15}16 17console.log("lab LCP: 2228 ms");18console.log("field LCP at p75:", p75(visitors), "ms");

• Lines 2 to 5 are the four recorded LCPs. Each comes with a share: how many of every 100 visits that kind of visitor makes. The shares are made up. Here, most visits are from regulars.
• Line 9 sorts the visitors from fastest to slowest.
• Lines 11 to 13 count their visits, fastest first, until 75 of the 100 are counted. That visitor's LCP is the 75th percentile.
• Line 17 prints lab LCP: 2228 ms, Ravi's test. Line 18 prints field LCP at p75: 808 ms.

The new phones make 20 visits. The regulars bring the count to 80, past 75. So the regular's 808 ms is the 75th percentile. Now change who visits:

Ravi's number never moves, because the lab always loads the page the same way. The field's number moves with your visitors. With some mixes, the same page goes from good to poor.

Diagnosing in the lab, confirming in the field

So which do you trust? web.dev's rule: when you have both, let the field data set your priorities. It's what real people got. But field data has a weakness. It tells you that visits were slow. It rarely tells you why.

To diagnose a problem is to find its cause. You do that in the lab, because a lab test can record a trace: a record of everything the browser did during one load. Lesson 7.2, Reading a performance trace, shows how to read one. Together, the two kinds of data make a loop of four steps:

1. Find it in the field. Which metric fails at the 75th percentile? On which pages? And for whom: phones or computers, which countries, which connections?
2. Reproduce it in the lab. Make the problem happen again, under those visitors' conditions. DevTools' Performance panel can fetch CrUX data for your page and show it next to your own numbers. It also suggests network and CPU settings closer to your users'.
3. Fix it, and check in the lab. Find the cause in a trace and fix it. Then test again with the same settings, and compare before and after.
4. Confirm it in the field. Your own RUM shows the change once visitors get the new version. CrUX covers the last 28 days, so it changes slowly.

Step 1 asks "for whom?". Field data can answer that if each visit is sent with its conditions. Here, each visit was sent with the connection type the browser reported:

1const visits = [2  { lcp: 900, connection: "4g" },3  { lcp: 1300, connection: "4g" },4  { lcp: 1700, connection: "4g" },5  { lcp: 2100, connection: "4g" },6  { lcp: 2400, connection: "4g" },7  { lcp: 3800, connection: "3g" },8  { lcp: 4600, connection: "3g" },9  { lcp: 5900, connection: "3g" },10];11 12for (const connection of ["4g", "3g"]) {13  const group = visits.filter((visit) => visit.connection === connection);14  const poor = group.filter((visit) => visit.lcp > 4000).length;15  console.log(`${connection}: ${poor} of ${group.length} visits poor`);16}

• Lines 2 to 9 are eight visits. Each has its LCP and its connection type, as navigator.connection.effectiveType reports it. That's the browser's guess at the network: "4g", "3g" and so on.
• Line 12 goes through the two connection types, one at a time.
• Line 13 keeps that type's visits. Line 14 counts the poor ones: LCP over 4 seconds, LCP's poor limit.
• Line 15 prints 4g: 0 of 5 visits poor, then 3g: 2 of 3 visits poor.

So the slow visits are on 3G. Test with a 3G-like setting in the lab, not on your office Wi-Fi. Lesson 12.5, Segmenting field data, splits visits this way by device, network and country. Lesson 12.7, From field data to a fix, walks the whole loop. Lesson 13.2, Proving a change worked, shows how to compare before and after honestly.

Sometimes the lab can't reproduce what the field shows. Then the difference itself is a clue. Look for:

• a cache the lab doesn't have;
• content that moves after the load;
• a slow tap or click;
• a script that only some visitors get.

And when the field is good but the lab is poor, the lab is still worth a look. web.dev points out that field data only has the people who managed to load your page. The lab can speak for visitors on slower networks and devices.

Lab or field?

Each question below needs one kind of data. Ask yourself: does it need real visitors, or exact conditions? Then sort it:

Common mix-ups

• "Lighthouse measures my users." No. It loads the page once, as a pretend phone with an empty cache. Field data measures your users.
• "If lab and field disagree, one of them is broken." No. They measured different loads: one test load, and many real visits.
• "Field data includes everyone." No. CrUX only has some Chrome users. And no field data has the people whose page never loaded.
• "A full cache makes a page fast." Only partly. The regular skipped two downloads, but app.js still ran just as long.
• "If my lab settings match my users, the lab is the field." No. Matching settings reproduce one kind of visitor. The field is all of them.
• "The lab can never measure INP." Not in a default run, where nobody interacts. DevTools' live metrics show INP as you click. But only real visitors' clicks give the field's INP.

Practice exercises

Answer first, then use the hints.

const labLcp = 3000;
const fieldLcps = [600, 700, 800, 900, 1000, 1300, 1600, 1800, 3200, 4100, 4800, 5000];

function percentile(values, p) {
  const sorted = [...values].sort((a, b) => a - b);
  const rank = Math.ceil((p / 100) * sorted.length);
  return sorted[rank - 1];
}

console.log("lab:", labLcp, "ms from one load");
console.log("field:", percentile(fieldLcps, 75), "ms at the 75th percentile of", fieldLcps.length, "visits");

const visitors = [
  { who: "new phone", lcp: 564, share: 10 },
  { who: "regular, back again", lcp: 808, share: 10 },
  { who: "first visit", lcp: 2228, share: 50 },
  { who: "cheap phone on 3G", lcp: 7628, share: 30 },
];

function p75(visitors) {
  const sorted = [...visitors].sort((a, b) => a.lcp - b.lcp);
  let seen = 0;
  for (const visitor of sorted) {
    seen += visitor.share;
    if (seen >= 75) return visitor.lcp;
  }
}

console.log("lab LCP: 2228 ms");
console.log("field LCP at p75:", p75(visitors), "ms");

Check your understanding

For each question, ask: which loads does this number come from, and under whose conditions?

Key takeaways

• Lab data comes from a test load under conditions you choose. It's repeatable and detailed, and ready before release.
• Field data comes from real visits. It's one number per visit, summed up at the 75th percentile, and it's what people got.
• A Lighthouse score is one load: a pretend phone, a slow network and CPU, an empty cache, and nobody tapping.
• The cache, the device, the network, the screen and what people do all move field numbers away from the lab's, in both directions.
• A full cache saves downloads, not work. The regular's phone still ran app.js just as long.
• Let the field set priorities. Reproduce and fix in the lab, under your users' conditions. Then confirm in the field.

Sources

The facts on this page come from these docs and source code:

• Why lab and field data can be different (and what to do about it), web.dev.
• Largest Contentful Paint (LCP), web.dev.
• Navigator: sendBeacon(), MDN.
• Page Lifecycle API, Chrome for Developers.
• web-vitals: README, Chrome's team on GitHub.
• CrUX methodology, Chrome for Developers.
• About PageSpeed Insights, Google for Developers.
• constants.js: Lighthouse's default settings, Lighthouse source.
• Network and CPU throttling, Lighthouse documentation.
• Lighthouse CLI options, Lighthouse documentation.
• Performance panel: live metrics and field data, Chrome for Developers.

Coming next: Averages lie: reading distributions, the last lesson in Why speed matters. Field data is a spread of numbers. That lesson looks at the spread's shape, and why an average hides your slowest visitors.