A/B testing

The problem: did the change cause it?

Your team believes a bigger "Add to cart" button will sell more. You ship it on a Tuesday, and by Friday orders are up 4%. The team celebrates.

Then someone notices that Friday was payday, a competitor had an outage on Wednesday, and marketing sent a coupon email on Tuesday. Was it the button? You can't tell.

This is the problem with before and after comparisons. Many things change at the same time as your release, and any of them can move your numbers more than your change did, or hide a change that hurts.

An A/B test (also called a controlled experiment or a split test) fixes this by comparing at the same time instead of before and after. Each visitor is assigned at random to one of two groups. Group A, the control, sees the current page. Group B, the treatment, sees the new button.

Both groups live through the same payday, the same outage, and the same coupon email. The only systematic difference between them is the button.

Why random assignment is the whole trick

Random assignment means every kind of user (mobile and desktop, new and loyal, rich and careful) ends up in both groups in about the same proportions. So when the treatment group converts better, the only explanations left are the change itself or plain luck. Statistics tells you how likely luck is. That is the full logic of an experiment, and it can break at any of four steps.

• Assignment must be truly random and stable, or the groups differ in more than the change.
• Exposure must match assignment: a user in B must really see B, without a flash of A first.
• Logging must record who saw what, exactly once, or the counts you compare are wrong.
• Analysis must account for luck honestly, or you will ship noise and call it a win.

Why this is a frontend topic

Data scientists own the statistics, but the frontend owns most of the places where an experiment silently breaks. The browser decides when the variant renders, so it decides whether users see a flicker. The client fires the exposure event, so it decides whether the counts are right. The client adds code for both variants, so it decides whether the treatment is slower for reasons that have nothing to do with the idea being tested.

A broken experiment is worse than no experiment. With no experiment, the team knows it is guessing. With a broken one, the team is confident and wrong.

This lesson builds on the feature flags lesson: an experiment is a flag with a random split and a measurement attached.

The mental model: a coin flip, a stamp, and a ledger

Picture a clinic testing a new medicine. At the door, each patient flips a coin that decides their group, and a stamp on their hand records the result, so they get the same treatment on every visit. A nurse writes in a ledger the moment they actually take a pill, not the moment they walk in. At the end, a statistician compares the two groups in the ledger.

A web experiment has the same parts. The coin flip is assignment, and the stamp is stickiness: the user keeps their variant on every page and every visit. The pill is exposure: the moment the user actually sees something that differs between variants. The ledger is the exposure log, and the statistician is the analysis.

The words you need

Choosing the unit

The unit of randomization is the thing you flip the coin for. For most product tests it is the user: one person should see one version, or the experience feels broken and their behavior is a mix of both. For a logged-out site, the user is approximated by a random id in a first-party cookie, which really means one browser.

Smaller units, such as a session or a page view, give more samples but leak between groups: a Tuesday purchase may be caused by the button seen on Monday. Use them only for changes users cannot notice across views, such as a faster image server.

What an experiment looks like as data

Experiment platforms (Optimizely, LaunchDarkly, Statsig, GrowthBook, Eppo, or an in-house service) all store something like the definition below. The SDK in the server or browser reads it and computes the variant locally, so no network call is needed per user.

{
  "key": "checkout-button-size",
  "unit": "userId",
  "allocation": 0.2,
  "variants": [
    { "key": "control", "weight": 0.5 },
    { "key": "big-button", "weight": 0.5 }
  ],
  "layer": "checkout",
  "primaryMetric": "orders_per_user",
  "guardrails": ["lcp_p75", "js_error_rate", "refund_rate"]
}

Assignment and bucketing: the fair, sticky coin

Assignment has two jobs that pull against each other. It must be random, so the groups are alike. It must also be sticky, so the same user gets the same answer on every request, on every server, today and next week.

A call to Math.random() is random but not sticky. Storing every user's draw in a database is sticky but needs a lookup on every page.

The standard answer, the same one the feature flags lesson uses for rollouts, is a deterministic hash. Feed the experiment key and the unit id into a fast hash, for example checkout-button-size:u_42. The output looks random across users, but it is always the same for the same input. Every server and every browser computes the same bucket without talking to anyone.

Two hashes: one for allocation, one for the variant

Map the hash to a fine range, such as 0 to 9,999, so you can allocate 0.5% of traffic. Then decide two things: is this user in the experiment at all (allocation), and which variant do they get (split)? A subtle bug appears if one bucket number decides both.

Say allocation is 20% and the split is 50/50 on one bucket: buckets 0 to 999 are control and 1,000 to 1,999 are treatment. Now you grow allocation to 40%, so control becomes 0 to 1,999. Every user in buckets 1,000 to 1,999 silently moves from treatment to control, and their history now pollutes both groups.

The fix is to hash twice with different salts (a fixed extra string added to the hash input, such as alloc or variant), so growing the allocation only adds users and never moves anyone.

type Variant = { key: string; weight: number };

function bucket(input: string): number {
  return hash32(input) % 10_000; // 0..9999
}

function assign(exp: { key: string; allocation: number; variants: Variant[] }, unitId: string) {
  // 1. In the experiment at all? Growing allocation only adds users.
  if (bucket(`${exp.key}:alloc:${unitId}`) >= exp.allocation * 10_000) return null;

  // 2. Which variant? Independent of allocation, so nobody switches.
  const b = bucket(`${exp.key}:variant:${unitId}`);
  let edge = 0;
  for (const v of exp.variants) {
    edge += v.weight * 10_000;
    if (b < edge) return v.key;
  }
  return exp.variants[exp.variants.length - 1].key;
}

Include the experiment key in the hash input. Otherwise the same slice of users lands in treatment for every experiment, and their combined exposure biases all of them.

Sticky variants: when the hash is not enough

A pure hash is sticky only while the unit id and the targeting rules stay the same. If the experiment targets the free plan and the user upgrades, the rule now excludes them, and the next page shows control.

The fix is persisted assignment, which GrowthBook calls sticky bucketing and Optimizely provides through its User Profile Service. On first assignment, store experimentKey: variant in a cookie or on the user record, and read it before running the hash and rules. Version it, so a restarted experiment does not reuse stale answers.

Logged out, then logged in: identity stitching

Most sites have visitors who arrive anonymous and log in later. Before login, the unit is a random anonymous id stored in a first-party cookie, such as cid_9f3a. After login, the natural unit is the account id, u_42. Hash the new id and you get a new bucket, so about half of users switch variant at the moment they log in, which is exactly the moment many funnels start.

• Keep the anonymous id as the unit for the whole experiment. Stable on one browser, but a phone and a laptop may see two variants.
• Persist the first assignment and copy it onto the account at login, so other devices read the same variant.
• Use the account id only, on pages behind login. Clean and cross-device, but it cannot test the signup flow.

Whatever you choose, log both ids with every event and keep an alias record (cid_9f3a became u_42) so analysis can join anonymous browsing to later purchases. That join is the job of the analytics pipeline, covered in the analytics tracking lesson.

Exposure logging: write the ledger at the right moment

Assigning a user is not the same as exposing them. A checkout test may assign every visitor on the home page, but only 8% reach checkout. The other 92% never saw a difference, and their zero effect dilutes the real one. This dilution can turn a clear 5% lift into an unreadable 0.4%.

So log an exposure event at the moment the variant is actually used: when the button renders, or when the code path runs. The analysis then compares only exposed users, which is called triggered analysis. The rule must be the same in both groups: control users log an exposure at the same place, even though nothing visible changed for them.

{
  "event": "experiment_exposure",
  "experiment": "checkout-button-size",
  "variant": "big-button",
  "anonymousId": "cid_9f3a",
  "userId": "u_42",
  "ts": "2026-10-19T10:15:02.120Z",
  "page": "/checkout"
}

Trust the counts first: sample ratio mismatch and A/A tests

Before you look at any result, look at how many users each group has. If you asked for a 50/50 split, you should get close to 50/50. Random chance makes the groups differ a little, but only a little. When the gap is too big to be chance, the experiment has a sample ratio mismatch (SRM), and every metric in it is suspect.

Why so strict? A mismatch means some users went missing from one group, and they almost never go missing at random. If the treatment page is slower and 2% of its users leave before the exposure event fires, the users who remain are the patient ones, who also buy more. The treatment then looks better precisely because it was worse.

The check: a chi-square test on the counts

The test asks: if the split were truly 50/50, how likely is a gap at least this big? With 100,000 exposed users, you expect 50,000 in each group. A result of 50,200 and 49,800 is normal noise (p is about 0.21).

A result of 50,600 and 49,400 is not: the chi-square statistic is 14.4 and p is about 0.00015. Many teams raise an SRM alarm when p is below 0.001 and hide the results until someone explains it.

function srmPValue(control: number, treatment: number, controlShare = 0.5) {
  const n = control + treatment;
  const expC = n * controlShare;
  const expT = n - expC;
  const chi2 = (control - expC) ** 2 / expC + (treatment - expT) ** 2 / expT;
  return erfc(Math.sqrt(chi2 / 2)); // p-value for 1 degree of freedom
}

srmPValue(50_200, 49_800); // about 0.21: fine
srmPValue(50_600, 49_400); // about 0.00015: investigate

Try the calculator. Notice that the same 1.2% gap is harmless with 5,000 users and alarming with 500,000. Large experiments detect smaller leaks, which is good: a leak of 0.5% of users is enough to fake a 0.5% lift.

Where the missing users go

Almost every SRM traces back to the frontend or the logging pipeline.

• The exposure event is lost more in one variant. A slower treatment loses users before the event sends. Fire it early, with navigator.sendBeacon or fetch with keepalive.
• A redirect in one arm. Redirect tests lose the users whose redirect fails or who press back.
• Cookies that do not last. Safari's tracking prevention caps cookies set by script (document.cookie) at 7 days. A user who returns after 8 days gets a new anonymous id, a new coin flip, and counts twice. Set the id cookie from the server with Set-Cookie instead.
• Bot filtering that differs by variant, because the treatment changes page structure.
• Caching. A CDN that caches one variant's HTML without the variant in the cache key serves it to both groups.
• Exposure logged in different places, such as on render in one arm and on click in the other.

A/A tests: test the test

An A/A test splits users into two groups that see exactly the same thing. Nothing should differ. If your platform reports a significant difference in an A/A test far more often than 5% of the time (at a 5% threshold), something in assignment, logging, or analysis is broken. Run one whenever you build or change the pipeline, and keep a small A/A test running all the time as a smoke alarm.

Avoiding flicker: show the right variant on the first paint

Flicker (also called the flash of original content) is when a user in the treatment sees the control version for a moment, then watches it change. It happens whenever the variant is applied after the first paint: the HTML arrives with the old headline, the browser paints it, then an experiment script downloads, decides the variant, and rewrites the DOM.

Flicker is not only ugly; it breaks the experiment. Treatment users saw both versions, so you are no longer comparing A with B but A with A-then-B. The change itself draws attention (movement catches the eye), and layout shift from the swap hurts CLS (Cumulative Layout Shift), the Core Web Vitals score for visual stability. Some users click the old button just before it moves.

The anti-flicker snippet and what it costs

Client-side testing tools that edit the page visually (VWO, Optimizely Web, and Google Optimize before it shut down in 2023) offer an anti-flicker snippet: a small inline script in the head that hides the page, usually by setting opacity: 0 on the html element, until the experiment script has run or a timeout has passed. Google Optimize's default timeout was 4,000 ms.

<style>.ab-hide { opacity: 0 !important }</style>
<script>
  document.documentElement.classList.add("ab-hide");
  window.abReveal = function () {
    document.documentElement.classList.remove("ab-hide");
  };
  setTimeout(window.abReveal, 4000); // fallback if the tool never loads
</script>
<script src="https://cdn.example-ab.com/project-123.js" async></script>

This trades flicker for a blank page. The page may be ready to paint at 600 ms, but nothing paints until the tool's script has run, say at 1,600 ms. LCP (Largest Contentful Paint) moves out by the whole script time for every user, including control, and users whose browser blocks the script stare at a white page until the timeout.

The real fixes: decide before the HTML is built

Flicker disappears when the variant is known before the first byte of HTML is generated. There are three places to make that decision, and they are the same three places the feature flags lesson uses for flag evaluation.

export default async function middleware(req: Request) {
  const url = new URL(req.url);
  if (url.pathname !== "/") return fetch(req);

  let cid = readCookie(req, "cid");
  const isNew = !cid;
  if (!cid) cid = crypto.randomUUID();

  const variant = assign(homeHeroExperiment, cid) ?? "control";
  url.pathname = `/variants/home/${variant}`; // cached per variant

  const res = await fetch(new Request(url, req));
  const out = new Response(res.body, res);
  if (isNew) out.headers.append("Set-Cookie", `cid=${cid}; Path=/; Max-Age=31536000; Secure; SameSite=Lax`);
  return out;
}

Two details make the edge approach safe. The cache key is the rewritten path, not the cookie, so the CDN keeps exactly one copy per variant instead of one per user. And the cookie is set by the server with Set-Cookie, so browser limits on script-written cookies do not shorten it.

Metrics: one to decide, some to explain, guardrails to protect

An experiment is only as good as the question it measures. Pick metrics before the test starts and write them down. If you choose after seeing the data, you will find something that moved: with 40 metrics at a 5% threshold, you expect 2 false wins from noise alone.

The primary metric: the one that decides

The primary metric is the single number the decision rests on, such as orders per user or the share of users who finish signup. Some companies combine a few into an overall evaluation criterion, but the idea is the same: one agreed-upon definition of better, written down before launch.

A good primary metric is close to the change (a checkout button should move checkout completion, not monthly active users), sensitive enough to move within a few weeks, and aligned with long-term value. Clicks are easy to move and often misleading: a bright, vague button gets more clicks and fewer purchases.

Secondary metrics: the why

Secondary metrics explain the primary one. If orders went up, did more people add to cart, or did the same people finish more often? They help you learn, but they don't decide. Treat a lone secondary win with suspicion: it is one of many numbers, and some will move by chance.

Guardrail metrics: what must not get worse

Guardrail metrics are the things a test is not trying to change but must not harm. A new hero carousel may lift clicks while making the page 400 ms slower and adding errors on old Android phones. Guardrails catch that. A good set has four groups.

Frontend engineers should own the first two rows. Tag every real-user monitoring (RUM) beacon and error report with the active experiment variants, as the real-user monitoring and frontend logging lessons describe, so you can compare p75 LCP and error rate per variant on the same dashboard as the business metrics.

Metric shapes and their traps

The last row matters for frontend work. If the treatment makes the site usable on cheap phones, more cheap-phone users stay, and the variant's p75 LCP rises even though every page got faster. Split by device class before calling a performance guardrail breached.

Making metrics more sensitive

The noisier a metric, the more users you need. Three ways to reduce noise without more traffic: triggered analysis (only exposed users), capping outliers, and CUPED (Controlled-experiment Using Pre-Experiment Data), which subtracts the part of each user's metric that was predictable from their behavior before the test.

Statistics you need: p-values, intervals, sample size, and peeking

You don't need to be a statistician to run experiments, but you do need to read results without fooling yourself. This chapter covers the five ideas that come up in every review and every interview: the p-value, the confidence interval, the minimum detectable effect, the sample size, and the peeking problem.

The p-value: how surprising is this if nothing changed?

Start from the boring assumption that the change does nothing, called the null hypothesis. The p-value answers one question: if that were true, how often would random assignment alone produce a difference at least as big as the one you saw? A p-value of 0.03 means 3% of the time.

Teams pick a threshold in advance, usually 0.05, and call a result statistically significant when p is below it. That threshold is a contract: across many experiments where the change truly does nothing, about 5% will still look significant. Those are false positives. The p-value is not the chance the change works, and it says nothing about how big the effect is.

Confidence intervals: the range of believable effects

A confidence interval is more useful than a p-value because it shows size and uncertainty together. Worked example: 122,000 users per group, control converts at 5.00%, treatment at 5.25%. The difference is +0.25 percentage points, and the 95% confidence interval is about +0.08 to +0.42 points, or a relative lift of roughly +1.5% to +8.5%.

• The interval excludes zero, so the result is significant at the 5% level. The two statements are the same test viewed two ways.
• The interval is wide. The true lift could be a small +1.5% or a large +8.5%. Report the range, not just the point estimate of +5%.
• An interval that straddles zero (say -0.6% to +4.1%) does not prove no effect. It means this test could not tell.

Minimum detectable effect, power, and sample size

Plan the size of a test before you start. Power is the chance of detecting a real effect of a given size; 80% is the usual target. The minimum detectable effect (MDE) is the smallest true effect the test has that 80% chance of catching. Smaller effects need many more users, because the noise shrinks only with the square root of the sample.

function usersPerGroup(baseline: number, relativeMde: number) {
  const p1 = baseline;
  const p2 = baseline * (1 + relativeMde);
  const z = 1.96 + 0.84; // 5% two-sided threshold + 80% power
  return Math.ceil((z ** 2 * (p1 * (1 - p1) + p2 * (1 - p2))) / (p2 - p1) ** 2);
}

usersPerGroup(0.05, 0.10); // about  31,000
usersPerGroup(0.05, 0.05); // about 122,000
usersPerGroup(0.05, 0.02); // about 752,000

Halve the effect and you need four times the users. With 20,000 exposed users a day split 50/50, the 5% lift on a 5% baseline needs a little over 12 days, which teams round up to two full weeks. The 2% lift needs 75 days, which usually means the test is not worth running as designed: find a more sensitive metric, trigger closer to the change, or accept a larger MDE.

The peeking problem

Dashboards update live, so it is tempting to check every day and stop as soon as the result turns significant. This is called peeking, and it breaks the 5% contract. Each look is another chance for noise to cross the line, and you stop on the lucky one.

How bad is it? We simulated 20,000 A/A tests (no real difference) for each duration, with 10,000 users a day at a 5% baseline. Looking once at the end produced a false positive about 5% of the time, as promised.

Looking every day and stopping at the first p below 0.05 produced a false positive about 17% of the time over 7 days, 22% over 14 days, and 28% over 28 days. The longer you keep peeking, the more tests with no real effect end up looking like winners.

Use the lab: choose "no effect" and turn on "stop at first p below 0.05". Press "Run again" a few times and watch how often the early-stop rule declares a winner that does not exist. Then set a real effect and see how traffic and duration change the width of the interval and the users needed.

Sequential testing: peeking done honestly

Teams want to look early for good reasons: to stop a harmful test fast, or to ship a big win sooner. Sequential testing methods allow that by making each early look stricter. Group sequential designs plan a few looks and spend the 5% budget across them; always-valid methods let you look at any time.

The price is width: a sequential interval is wider than a fixed one on the same data. Methods vary by vendor, so read your platform's documentation before trusting its "stop early" button. Without sequential statistics, fix the duration in advance and read the result once.

More variants, more metrics, more false wins

Every extra comparison is another lottery ticket for noise. Test four variants against control on ten metrics and you make 40 comparisons; at 5% each, about two will look significant by chance. Name one primary metric in advance, limit variants to what you can power, and when you must compare many, use a correction such as Bonferroni (divide the threshold by the number of comparisons).

Time effects and running many experiments at once

Two things can mislead an experiment even when assignment, logging, and statistics are all correct: time and other experiments. Users react to change itself, traffic changes through the week, and your test is rarely the only one running.

Novelty and change aversion

A novelty effect is when users engage with something because it is new, not because it is better. A redesigned menu gets extra clicks in week one as people explore it, then settles back. The opposite is change aversion (sometimes called the primacy effect): loyal users stumble on a moved button and do worse at first, then adapt and maybe do better than before.

• Plot the effect by day or week, not only the total. A lift that shrinks steadily is a sign of novelty.
• Split new and returning users. New users have no old habit to compare against, so their result is closer to the long-term effect.
• Keep a holdout for big changes: a small group (say 5%) stays on the old experience for months after launch, so you can measure the lasting effect.

Seasonality: run whole weeks

Traffic on Tuesday is not traffic on Saturday. Weekday visitors may be office workers on desktops; weekend visitors browse on phones and buy differently. A test that runs Monday to Thursday describes weekday users only. Run experiments in whole weeks (7, 14, or 28 days), even if the sample size is reached sooner.

Watch the calendar too: a result measured during a big sale or a holiday may not hold the rest of the year.

When experiments interact

Large products run hundreds of tests at once. Usually that is fine: because each experiment hashes with its own key, other tests' variants spread evenly across both of your groups and add noise, not bias.

Trouble starts when two tests change the same thing. Test 1 makes the checkout button green; test 2 makes the button's text green. A user in both treatments gets green text on a green button that nobody can read. Neither test alone shows the problem, and both results are muddled.

This is an interaction effect. Published experience from large platforms suggests strong interactions are rare, but when they happen they are usually between tests on the same surface.

Layers: exclusive inside, overlapping across

The common fix, described in Google's 2010 paper on overlapping experiments, is layers (also called namespaces or universes). Each layer owns one area of the product, such as checkout, search ranking, or the home page. Inside a layer, experiments are mutually exclusive: the layer's hash splits users into slots, and each experiment gets its own slots, so no user is in two checkout tests. Across layers, experiments overlap freely, because a different layer hash shuffles users again.

const checkoutLayer = {
  key: "layer-checkout",
  experiments: [
    { key: "button-color", slots: [0, 4_999] },   // 50% of the layer
    { key: "express-pay", slots: [5_000, 7_999] }, // 30% of the layer
    // slots 8,000 to 9,999 are free for the next test
  ],
};

function experimentInLayer(layer: typeof checkoutLayer, unitId: string) {
  const slot = bucket(`${layer.key}:${unitId}`);
  return layer.experiments.find((e) => slot >= e.slots[0] && slot <= e.slots[1]) ?? null;
}

Layers cost traffic: exclusive tests each get only part of the layer. Use them for tests that touch the same element or decision, and let unrelated tests overlap.

Server or client experiments, and the ethics of testing people

The flicker chapter showed one cost of deciding in the browser. This chapter covers the rest of the trade-off, then a part teams often forget: the people inside the test.

Comparing the two

Pitfalls specific to client experiments

• Hydration mismatch. If the server renders control but the client SDK computes treatment, React throws away the server HTML or reports a hydration error. Pass the server's assignments to the client and never re-evaluate them during hydration.
• Both variants in the bundle. A redesigned component imported statically ships to control users too. Load variant code with a dynamic import() only for the users who need it, and remove the losing branch quickly after the test.
• A slower treatment for the wrong reason. If the treatment lazy-loads its code and control does not, the treatment is slower because of how it was built, not because of the idea. Make the cost equal, or measure it with the guardrails.
• Leaks. Variant names in the client config can reveal unreleased plans, such as pricing-v3-annual-discount. Use opaque keys for sensitive tests and keep their logic on the server.

Consent and privacy

Assignment for logged-out users usually relies on an identifier stored in the browser. In the EU, storing or reading such an identifier for purposes that are not strictly necessary generally requires the user's consent under ePrivacy rules, and many regulators place A/B testing in that category. The UK has similar rules, recently relaxed for some analytics uses. The details differ by country and change over time, so treat this as a question for your legal team, not a guess.

• Before consent, show control and don't store an id or count the user. Accept that results then describe consenting users only, who may differ from the rest.
• Send the minimum to vendors: an opaque id and the attributes targeting needs, not email addresses or names.
• Respect opt-outs everywhere: a user who declines analytics should not be bucketed and tracked through a different script.

Ethics: what not to test

Users don't know they are in an experiment, so the team carries the responsibility. A useful rule: only test variants you would be comfortable shipping to everyone. Hiding the cancel button, fake urgency, or different prices for the same product can raise legal questions and break trust.

In 2014, a published Facebook study that adjusted the emotional tone of news feeds drew wide criticism because people felt manipulated. Tests that touch money, health, minors, or emotions deserve review by someone outside the team. And both variants must meet your accessibility standards.

In real products and in the system design interview

Companies that run experiments at scale have published a lot about it. Microsoft, Booking.com, Netflix, Airbnb, and LinkedIn each describe running thousands of experiments a year on in-house platforms, with automatic SRM checks, guardrail dashboards, and layers.

A widely cited example comes from Microsoft's Bing in 2012. A small change to how ad headlines were displayed sat in the backlog for more than six months; when finally tested, it raised revenue by about 12%. The lesson the authors draw is humbling: most ideas, even ones experts are sure of, do not improve the metric they target.

When to bring it up

Bring up experimentation when the prompt involves growth, conversion, ranking, or a redesign of a high-traffic page: a checkout, a feed, a search results page, a landing page. One or two sentences are enough at first: "We'd ship this behind an experiment flag and measure orders per user, with LCP and error rate as guardrails." Go deeper if the interviewer follows up.

What to draw

1. An experiment config service that publishes definitions to SDKs and the edge.
2. Assignment at the edge or server: a hash of experiment key and unit id, sticky through a server-set cookie.
3. Rendering of the variant on the server, with assignments inlined for hydration.
4. An exposure event from the client when the element is visible, sent with sendBeacon.
5. An event pipeline into a warehouse, where the analysis joins exposures with metric events per user.
6. A results page with the SRM check first, then the primary metric with its interval, then guardrails.