Segmenting field data

Who is slow, and why?

Leaf and Kettle ships a new version of its mint tea page. The team calls it release b. The next day, a dashboard like the one in Building a RUM pipeline shows its number: a p75 LCP of 144 ms. The version before, release a, had 1224 ms.

Someone in the team says: "Release b is more than eight times faster!" Is it? And is anyone still waiting?

First, the two words in that number:

• LCP, Largest Contentful Paint, is when the biggest image or block of text appeared. On this page, it's the mint tea photo.
• The p75, the 75th percentile, is a time that three visits in four matched or beat.

Here are release b's 16 visits, in two groups:

1// LCP of release b's 16 visits, in ms.2const phones = [1228, 1228, 1224, 1232];3const desktops = [132, 124, 120, 140, 128, 144, 128, 140, 124, 128, 132, 132];4 5const p75 = (values) => [...values].sort((x, y) => x - y)[Math.ceil(values.length * 0.75) - 1];6console.log(`Everyone: p75 LCP ${p75([...phones, ...desktops])} ms`);7console.log(`Phones only: p75 LCP ${p75(phones)} ms`);

• Line 1 is a comment. Each number is one visit's LCP, in milliseconds (ms): thousandths of a second.
• Line 2 holds the 4 visits from phones. Line 3 holds the 12 from desktop computers.
• Line 5 works out a p75. It sorts a copy of the times, fastest first, and takes the one three quarters of the way along. For 16 visits, that's the 12th. For 4, it's the 3rd.
• Line 6 puts both groups together. It prints Everyone: p75 LCP 144 ms.
• Line 7 uses the phones alone. It prints Phones only: p75 LCP 1228 ms.

So the phone visits waited more than a second for the photo. Yet the number for everyone didn't show them at all. Why not? Put all 16 visits in a line, fastest first:

The p75 is the 12th visit in the line. The four phone visits were the four slowest: the 13th to the 16th. They filled the slowest quarter, and the p75 stopped just before them. A slow group of about a quarter of the visits, or fewer, can hide there completely.

Splitting visits into groups like this is called segmenting. Each group is a segment: the visits that share one thing, such as phones, one kind of page, or first visits. Averages lie: reading distributions used the word first.

Here, phones are slow. But why? A phone can differ from a desktop in more than one way. It may have a slower processor, and a slower network. To tell those causes apart, change one thing at a time and watch what moves.

That's what the course did. It loaded a copy of the mint tea page 120 times in Chrome. Each visit loaded the page, clicked Add to cart, and left. As it left, the page sent all its metrics in one beacon, as in Sending performance data. The visits came in five experiments, and each one changed only one thing:

• Device and network: a desktop or a phone, on a fast or a slow network.
• Page templates: the home page, or a product page with more photos.
• Cache and navigation type: a first visit, going Back, a reload, and a later visit.
• Logged in or not: the same page, with and without a member's cookie.
• Two releases: release a and release b, each with its own mix of phones and desktops. That's where the 144 ms came from.

Here is every experiment. Pick one, then a metric. Each row is a segment, with its p75 and how many visits it has:

The next four sections take one kind of split each. The last one comes back to release b, and to the team's question.

Device and network segments

Where do you start splitting? Think of a café with a pile of complaints.

Each way of sorting points at a different cause. This section sorts by the two things every visitor brings: a device and a network.

One thing at a time

The course made a phone and a slow network inside Chrome. It used Chrome's DevTools protocol, which lets a program steer Chrome from outside:

• The phone had a screen 412 pixels wide. Its processor, the CPU, ran 4 times slower than the desktop's. The protocol calls this a "slowdown factor".
• The slow network held every request for at least 300 ms before its answer began. All downloads together shared 200,000 bytes a second, about 200 KB.
• The fast network had no delay and no limit.

It made 8 visits in each of the four mixes: a desktop or a phone, on the fast or the slow network. Here is what the network and the phone did to LCP:

1// p75 LCP from 8 visits in each segment, in ms.2const desktopFast = 44;3const desktopSlow = 1212;4const phoneFast = 100;5 6console.log(`The slow network added ${desktopSlow - desktopFast} ms to LCP.`);7console.log(`The phone added ${phoneFast - desktopFast} ms to LCP.`);

• Lines 2 to 4 are recorded: the p75 LCP of three segments. desktopFast is the desktop on the fast network, and so on.
• Line 6 changes only the network: it compares the desktop with itself. It prints The slow network added 1168 ms to LCP.
• Line 7 changes only the device: both are on the fast network. It prints The phone added 56 ms to LCP.

Each line compares two segments that differ in one way, so each difference has one cause. For loading the page, the network mattered more than twenty times as much as the phone.

Why so much? The mint tea photo is 102,809 bytes. On the slow network, the photo alone needed about half a second just to arrive, after a wait of at least 300 ms to start.

Now the click. Every visit pressed Add to cart, and the button runs a loop of 30,000,000 turns: the same work on every device. INP, Interaction to Next Paint, is the time from a click until the next frame shows its result.

1// p75 INP from 8 visits in each segment, in ms.2const desktopFast = 64;3const desktopSlow = 64;4const phoneFast = 120;5 6console.log(`The slow network added ${desktopSlow - desktopFast} ms to INP.`);7console.log(`The phone added ${phoneFast - desktopFast} ms to INP.`);

• Lines 2 to 4 are recorded: the p75 INP of the same three segments.
• Line 6 prints The slow network added 0 ms to INP.
• Line 7 prints The phone added 56 ms to INP.

The click never touched the network. Its loop ran on the CPU, so only the slower CPU made it slower. Here are all four segments, with both metrics:

Why it matters: "phones are slow" is not a cause. Split phones by network too. If LCP is slow only on slow networks, look at the bytes the page sends. If INP is slow on phones on every network, look at the code the click runs.

Putting the device and network on each beacon

In the recording, each page knew its segment, because the course put it in the page's address. A real site has to work it out.

Field data from every browser labelled each beacon's engine from its user agent: the text a browser sends to say what it is. The device can come from the same text. MDN's advice is short: "you can look for the string Mobi anywhere in the UA string." UA is short for user agent.

1// Two user agents recorded in "Field data from every browser".2const laptop = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/156.0.0.0 Safari/537.36";3const iphone = "Mozilla/5.0 (iPhone; CPU iPhone OS 18_7 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/26.6 Mobile/15E148 Safari/604.1";4 5const deviceOf = (userAgent) => (userAgent.includes("Mobi") ? "phone" : "desktop");6console.log(`The laptop goes in the ${deviceOf(laptop)} segment.`);7console.log(`The iPhone goes in the ${deviceOf(iphone)} segment.`);

• Lines 2 and 3 hold two user agents recorded in that lesson: Chrome on a laptop, and WebKit acting as an iPhone.
• Line 5 is the test. Text that contains Mobi is a phone. Anything else is a desktop.
• Line 6 prints The laptop goes in the desktop segment. Line 7 prints The iPhone goes in the phone segment. The iPhone's user agent says Mobile/15E148.

The network is harder to label. Chromium browsers have navigator.connection.effectiveType: the browser's own guess at the kind of connection. MDN lists its four values: slow-2g, 2g, 3g and 4g. The browser picks one from the round trips and download speeds it saw recently. Before you split by it, know two things:

• Firefox and Safari don't have it. Their beacons have no answer, and no answer isn't the same as a fast or a slow one.
• It's coarse. The docs of the Chrome User Experience Report say most visits "are likely on connections faster than 3G and thus classified as 4G". So it mostly picks out the slowest connections.

The Chrome User Experience Report, or CrUX, splits its data too. Its ways of splitting are called dimensions, and The Chrome User Experience Report covered them:

• Form factor: phone, tablet or desktop, "inferred from the device User-Agent string".
• Country: worked out from the visitor's IP address, the number the visit came from.

CrUX used to split by effective connection type too. It removed that dimension in February 2025, and reports a round-trip time instead.

Your own collector can find each beacon's country the same way, from the address the beacon came from. Keep the country and drop the address. Sending performance data showed why to store as little as you can.

Page templates and routes

Leaf and Kettle has a page for every tea: /tea/mint, /tea/oolong, and so on. Each of those paths is a route: the part of the address after the site's name. Every tea's page is built from the same page template: one layout, with a photo, a price and an Add to cart button.

Should you split by route, or by template? By template, for two reasons:

• One route on its own may get only a few visits. A segment that small is noisy, as the last section shows. A template gathers the visits of all its routes.
• The fix lives in the template. Make the product template faster, and every tea's page gets faster.

So put each route in its template before you split. A small function can do it:

1// Every tea has its own page, but they all use one template.2function templateOf(path) {3  if (path === "/") return "home";4  if (path.startsWith("/tea/")) return "product";5  return "other";6}7 8console.log(`/tea/mint is a ${templateOf("/tea/mint")} page.`);9console.log(`/tea/oolong is a ${templateOf("/tea/oolong")} page.`);10console.log(`/ is the ${templateOf("/")} page.`);

• Line 2 starts a function that takes a path.
• Line 3 calls / the home page. Line 4 puts every path that starts with /tea/ in the product template. Line 5 calls anything else other.
• Lines 8 to 10 print /tea/mint is a product page., /tea/oolong is a product page. and / is the home page.

The collector can run a function like this on each beacon's path. Or the page can send its template's name itself, since it knows which template built it.

Now the recording. The course's product page had the same photo as the home page, plus three more photos below it. It loaded each page 8 times, on the phone and the slow network:

1// LCP from 8 visits to each page, on a phone and a slow network, in ms.2const home = [1224, 1228, 1220, 1232, 1216, 1228, 1232, 1228];3const product = [2360, 2376, 2368, 2368, 2384, 2368, 2356, 2372];4 5const p75 = (values) => [...values].sort((x, y) => x - y)[Math.ceil(values.length * 0.75) - 1];6console.log(`Home page: p75 LCP ${p75(home)} ms`);7console.log(`Product page: p75 LCP ${p75(product)} ms`);

• Lines 2 and 3 are recorded: the LCP of every visit to each page.
• Line 5 is the same p75 as before.
• Line 6 prints Home page: p75 LCP 1228 ms. Line 7 prints Product page: p75 LCP 2372 ms.

Same phone, same network, and the product page took almost twice as long. Why? Each photo is 102,809 bytes, so the product page's four photos add up to 411,236 bytes. Through 200,000 bytes a second, that's about two seconds of downloading. The photos shared the pipe, so each one arrived later.

Why it matters: a number for the whole site would only say "slow". Split by template, it says the product template is slow, and that's where the fix goes.

Cache state and navigation type

The first time Asha opens the mint tea page, her browser downloads everything. When she comes back, the browser can take the photo from its HTTP cache: a store of files it kept from earlier visits. That saves the download.

The course's server let the browser keep the photos and the web-vitals script for an hour. The page itself had to be fetched every time. On the phone and the slow network, the course made 6 rounds of four visits:

1. A first visit, with an empty cache.
2. Back: it followed a link away, then pressed the Back button.
3. Reload: it reloaded the page.
4. A later visit: it left again, then went to the page's address again, as a new visit.

How can a beacon tell these visits apart? Each page's navigation entry has a type: its navigation type. Navigation Timing names three:

• "navigate": a normal visit, such as following a link or typing the address.
• "reload": the page was reloaded.
• "back_forward": "Navigation that's applied from history", such as the Back button.

The web-vitals library copies the type onto every metric, as navigationType. It writes back_forward with a hyphen: back-forward. It also has a few types of its own. You met "back-forward-cache" there: a page restored whole from the back/forward cache.

Here, going Back loaded the page again. Chrome didn't restore it from the back/forward cache, so web-vitals said "back-forward". Here is how the first three kinds of visit compare:

1// p75 LCP from 6 visits of each kind, on a phone and a slow network, in ms.2const firstVisit = 1224;3const back = 368;4const reload = 364;5 6console.log(`Back was ${(firstVisit / back).toFixed(1)} times faster than a first visit.`);7console.log(`Reload was ${(firstVisit / reload).toFixed(1)} times faster than a first visit.`);

• Lines 2 to 4 are recorded: the p75 LCP of each kind of visit, from 6 of each.
• Line 6 divides the first visit's time by Back's. It prints Back was 3.3 times faster than a first visit.
• Line 7 does the same for Reload. It prints Reload was 3.4 times faster than a first visit.

Why so much faster? The photo came from the cache. Only the page itself came over the slow network, and each request there waited at least 300 ms. That wait was most of each of these visits' LCP.

Now the fourth kind, the later visit:

1// Two kinds of visit, 6 of each, on a phone and a slow network. LCP in ms.2const firstVisit = { navigationType: "navigate", p75Lcp: 1224 };3const laterVisit = { navigationType: "navigate", p75Lcp: 360 };4 5console.log(`Same navigation type: ${firstVisit.navigationType === laterVisit.navigationType}`);6console.log(`The later visit was ${(firstVisit.p75Lcp / laterVisit.p75Lcp).toFixed(1)} times faster.`);

• Lines 2 and 3 are recorded. Both kinds of visit had the navigation type "navigate".
• Line 5 prints Same navigation type: true.
• Line 6 prints The later visit was 3.4 times faster.

The later visit was a normal visit too. Only the cache was different. So the navigation type alone can't tell a first visit from a later one.

Your beacons can do the same. The page can keep a small note in localStorage: storage the browser keeps for each site. MDN says its data "is saved across browser sessions". If the note isn't there, send firstVisit: true, then write the note.

Why it matters: a site with many returning visitors can look fast overall while first visits stay slow. And a first visit is a new customer's first look at the shop.

Logged-in vs anonymous users

Some visitors log in. Then the site knows who they are, and can build the page for them: their name, their cart, their usual tea. A visitor who hasn't logged in is anonymous. Every anonymous visitor can get the same page.

Think of a café that makes each member's tea fresh, to their own recipe, while everyone else gets tea from a pot that's already brewed. The member's tea is more personal. It also takes longer.

The course's server played this out. A cookie is a small piece of text a site stores in the browser. The browser sends it back with each request to that site. When a request carried a session cookie, the server waited 250 ms before sending the page. That wait stands in for building a page for one person.

The course loaded the page 8 times each way, on the desktop and the fast network. The metric to watch is TTFB, time to first byte: how long the browser waited for the first byte of the page.

1// TTFB from 8 visits each, on a desktop and a fast network, in ms.2const anonymous = [2.3, 2, 1.4, 2, 1.3, 1.3, 1.7, 4.7];3const loggedIn = [252.8, 253.7, 254.7, 252.9, 253.5, 253.8, 263, 253.9];4 5const p75 = (values) => [...values].sort((x, y) => x - y)[Math.ceil(values.length * 0.75) - 1];6console.log(`Anonymous: p75 TTFB ${p75(anonymous)} ms`);7console.log(`Logged in: p75 TTFB ${p75(loggedIn)} ms`);

• Lines 2 and 3 are recorded: the TTFB of every visit.
• Line 6 prints Anonymous: p75 TTFB 2 ms.
• Line 7 prints Logged in: p75 TTFB 253.9 ms.

The gap is the server's 250 ms, and a little more. The device was the same, and so was the network. Only the server's work changed.

Why it matters: the visitors who log in can get the slowest pages, and only a split by login state shows it. So put a flag in each beacon: loggedIn: true or false. Never put in who they are. Sending performance data showed how much a beacon can carry without being asked.

Avoiding Simpson's paradox

Back to the team's question. Release a's p75 LCP was 1224 ms. Release b's was 144 ms. Is release b faster?

The course knew the answer before it started, because it made release b slower on purpose. Its server held release b's photo back 100 ms longer than release a's. Then each release got 16 visits, in a different mix:

• Release a: 12 phone visits and 4 desktop visits.
• Release b: 4 phone visits and 12 desktop visits.

The phones were on the slow network, and the desktops on the fast one. Here are the totals the dashboard showed:

1// LCP of every visit, in ms. Release a went mostly to phones, release b mostly to desktops.2const releaseA = { phones: [1224, 1216, 1232, 1228, 1216, 1224, 1216, 1224, 1224, 1220, 1224, 1228], desktops: [40, 36, 44, 40] };3const releaseB = { phones: [1228, 1228, 1224, 1232], desktops: [132, 124, 120, 140, 128, 144, 128, 140, 124, 128, 132, 132] };4 5const p75 = (values) => [...values].sort((x, y) => x - y)[Math.ceil(values.length * 0.75) - 1];6const everyone = (release) => [...release.phones, ...release.desktops];7console.log(`Release a, everyone: p75 LCP ${p75(everyone(releaseA))} ms`);8console.log(`Release b, everyone: p75 LCP ${p75(everyone(releaseB))} ms`);

• Lines 2 and 3 are recorded: every visit's LCP, by release and device.
• Line 6 puts each release's phones and desktops back together.
• Line 7 prints Release a, everyone: p75 LCP 1224 ms. Line 8 prints Release b, everyone: p75 LCP 144 ms.

Now compare like with like: phones with phones, and desktops with desktops.

1// LCP of every visit, in ms. Release a went mostly to phones, release b mostly to desktops.2const releaseA = { phones: [1224, 1216, 1232, 1228, 1216, 1224, 1216, 1224, 1224, 1220, 1224, 1228], desktops: [40, 36, 44, 40] };3const releaseB = { phones: [1228, 1228, 1224, 1232], desktops: [132, 124, 120, 140, 128, 144, 128, 140, 124, 128, 132, 132] };4 5const p75 = (values) => [...values].sort((x, y) => x - y)[Math.ceil(values.length * 0.75) - 1];6console.log(`Phones: a ${p75(releaseA.phones)} ms, b ${p75(releaseB.phones)} ms`);7console.log(`Desktops: a ${p75(releaseA.desktops)} ms, b ${p75(releaseB.desktops)} ms`);

• Lines 1 to 5 are the same as in the last example.
• Line 6 prints Phones: a 1224 ms, b 1228 ms.
• Line 7 prints Desktops: a 40 ms, b 132 ms.

On desktops, release b was 92 ms slower. On phones, it was no faster. Yet its total was more than eight times better. How?

Look at who visited. Release b's visits were mostly desktops on the fast network, and those are quick. Release a's were mostly phones on the slow network. The totals mostly measured who visited, not how fast each release was.

This trap has a name: Simpson's paradox. The Stanford Encyclopedia of Philosophy puts it this way: "the type of association at the population level (positive, negative, independent) changes at the level of subpopulations." In plain words, what's true of everyone together can turn around inside each group. Here, release b looked better for everyone together. It was better for no group at all.

Averages lie: reading distributions showed the same trap with a monthly total. Now you've seen it in real visits.

Why it matters: when a total moves, check the mix before you check the speed. Show each segment's share of the visits next to its p75. If the shares changed, compare each segment with itself.

Four visits are too few to judge

One more look at those phones. Release b's phone p75 was 4 ms slower than release a's. Is that real?

Visits work the same way. Here are release a's own 12 phone visits, four at a time, in the order they happened:

1// Release a's 12 phone visits, in the order they happened: LCP in ms.2const phones = [1224, 1216, 1232, 1228, 1216, 1224, 1216, 1224, 1224, 1220, 1224, 1228];3 4const p75 = (values) => [...values].sort((x, y) => x - y)[Math.ceil(values.length * 0.75) - 1];5for (let start = 0; start < phones.length; start += 4) {6  console.log(`Visits ${start + 1} to ${start + 4}: p75 LCP ${p75(phones.slice(start, start + 4))} ms`);7}

• Line 2 is recorded: release a's phone visits.
• Line 5 steps through them four at a time. Line 6 prints the p75 of each group of four.
• It prints Visits 1 to 4: p75 LCP 1228 ms, Visits 5 to 8: p75 LCP 1224 ms and Visits 9 to 12: p75 LCP 1224 ms.

Same release, same page, and nothing changed. Yet the p75 of four visits moved by 4 ms. Release a's first four phone visits even had the same p75 as release b's four: 1228 ms. So a 4 ms gap from four visits tells you nothing either way. Building a RUM pipeline saw a p75 of four visits jump around in the same way.

And the more ways you split, the smaller each segment gets. Phones, on a slow network, on the product page, logged in, in one country: few visits fit all five. Two habits help:

• Show the count next to every p75. The playground does: it's the second number in each row.
• Set a smallest size, and don't judge a segment below it. CrUX does this too. It publishes a country's numbers only with "the standard eligibility requirements applied at a country level". So each country needs enough page loads on its own.

How many visits are enough? Reading performance data works that out, with error bars.

Which split explains it?

Here is every split from this lesson in one table. Read each row across: the split, how a beacon can carry it, and what it changed in the recording.

Now put them to work. Each item below is a pair of recorded numbers. Sort it by what was different between the two.

Common mix-ups

• "One p75 for everyone shows who is slow." No. Release b's p75 of 144 ms hid four phone visits that took more than 1200 ms.
• "Phones are slow because of their networks." No. On the same fast network, the phone's slower CPU added 56 ms to INP. The slow network added nothing to INP.
• "Every route needs its own row." No. Group routes into templates. One route alone has few visits, and the fix lives in the template.
• "The navigation type tells first visits from later ones." No. Both were "navigate", and the later visit was 3.4 times faster. Send a first-visit flag.
• "A better total means a better release." No. Release b's total was more than eight times better, yet it was better for no group. Its mix of visits changed.
• "Any segment can be trusted." No. Four visits are too few. Show the count beside each p75, and set a smallest size.
• "CrUX splits by connection type." Not any more. It removed that dimension in February 2025. It still splits by form factor and country.

Practice exercises

Answer first, then use the hints. Exercises 3 and 4 use the playground in the first section. Exercises 5 and 6 send you to the docs.

// LCP of every visit, in ms. Release a went mostly to phones, release b mostly to desktops.
const releaseA = { phones: [1224, 1216, 1232, 1228, 1216, 1224, 1216, 1224, 1224, 1220, 1224, 1228], desktops: [40, 36, 44, 40] };
const releaseB = { phones: [1228, 1228, 1224, 1232], desktops: [] };

const p75 = (values) => [...values].sort((x, y) => x - y)[Math.ceil(values.length * 0.75) - 1];
const everyone = (release) => [...release.phones, ...release.desktops];
console.log(`Release a, everyone: p75 LCP ${p75(everyone(releaseA))} ms`);
console.log(`Release b, everyone: p75 LCP ${p75(everyone(releaseB))} ms`);

// FCP from 8 visits to each page, on a phone and a slow network, in ms.
const home = [804, 792, 800, 808, 788, 812, 808, 796];
const product = [832, 840, 856, 840, 852, 840, 836, 840];

const p75 = (values) => [...values].sort((x, y) => x - y)[Math.ceil(values.length * 0.75) - 1];
console.log(`Home page: p75 FCP ${p75(home)} ms`);
console.log(`Product page: p75 FCP ${p75(product)} ms`);

Check your understanding

For each question, ask: which segment is different, what changed between the two, and did the mix change too?

Key takeaways

• One p75 for everyone can hide a slow group of about a quarter of the visits or fewer. Release b's 144 ms hid its phones.
• Split one thing at a time: device, network, page template, navigation type, first visit, login state and country.
• The slow network moved loading: it added 1168 ms to LCP. The phone's CPU moved the click: it added 56 ms to INP.
• Group routes into templates, so each segment has enough visits and points at one fix.
• The navigation type can't tell a first visit from a later one. Send a first-visit flag, and a logged-in flag, but never who.
• When a total moves, check the mix before the speed. And show the count beside every p75.

Sources

The facts on this page come from these docs:

• Emulation.setCPUThrottlingRate, Chrome DevTools Protocol.
• Network.emulateNetworkConditions, Chrome DevTools Protocol.
• Browser detection using the user agent, MDN.
• NetworkInformation: effectiveType property, MDN.
• CrUX dimensions, Chrome for Developers.
• Navigation Timing Level 2, W3C.
• web-vitals 6.2.3 README, Google Chrome.
• Window: localStorage property, MDN.
• Simpson's Paradox, Stanford Encyclopedia of Philosophy.

Coming next: Choosing a RUM tool. What commercial and open-source RUM tools offer, and the questions to ask before you pick one.