CDN caching

Why a shared cache at the edge

Picture a sneaker shop that runs a limited drop every Friday at 18:00. The servers live in one data center in Virginia. Most days the site gets about 200 requests per second. At 18:00 on a drop day, 400,000 people open the product page within the same minute, and each page load asks for about 40 files: the HTML, two bundles, a stylesheet, fonts, and a dozen product images.

That is about 16 million requests in one minute, roughly 270,000 per second. The origin, your own servers where responses are created, was sized for 200. It falls over at 18:00:04, and the few who get through wait over 200 ms for every round trip from Mumbai or Sydney to Virginia.

Now the same drop with a CDN (content delivery network) in front. A CDN is a fleet of servers in hundreds of cities, called points of presence or PoPs. Every user talks to a PoP near them. When the PoP already holds a copy of the response, it answers by itself, and the origin never hears about the request.

The browser cache is not enough

The HTTP caching lesson covered the browser cache: Cache-Control, ETag, 304 Not Modified. That cache is private: it belongs to one person on one device. It makes your second visit fast. It does nothing for the first visit, and nothing for the 399,999 other people arriving at 18:00.

A CDN cache is shared. One copy of the product page in the Mumbai PoP serves every user in Mumbai. The first user pays for a trip to the origin; the next ten thousand get the stored copy in about 20 ms. The origin sees roughly one request per object per PoP per lifetime, instead of one per user.

What a CDN cache buys you

• Latency. Round trips drop from 150 to 300 ms across an ocean to 10 to 30 ms, and TTFB (the time to the first byte of the response) drops with them.
• Origin protection. At 98% hits, the origin handles 2% of the traffic. A 100x spike becomes a 2x spike at the origin.
• Cost. Bytes served from the CDN usually cost less than bytes leaving a cloud region, and the origin needs fewer machines.
• Availability. A PoP can keep serving a stale copy while the origin is down.

A shared cache also brings three questions the browser cache never had. Which requests count as the same request? How do you remove a copy from 300 cities when content changes? And how do you make sure one user's personal page never reaches another?

This lesson answers them in that order, then measures the result with the cache hit ratio.

The mental model: a lookup table in every city

Keep one picture in your head: every PoP holds a big lookup table. Each row has a key and a stored response. The key is built from the request, usually the host, the path, and the query string. The value is the full response (status, headers, body) plus two facts the CDN needs: when it was stored and how long it may be served without asking the origin.

When a request arrives, the PoP builds the key, looks it up, and checks the age of the row. That is the whole job. Every feature in this lesson either changes how the key is built, changes how long a row lives, or removes rows on purpose.

Think of a chain of libraries. Each branch keeps its own copies of popular books, orders a missing one from the central warehouse, and keeps it for the next reader. The Paris branch knows nothing about Mumbai's shelves. That last part matters: each PoP has its own cache, so a file can be a hit in Mumbai and a miss in Paris at the same moment.

One request, step by step

Follow a product page, GET /shoes/air-1, through the Mumbai PoP. The origin sends Cache-Control: public, max-age=0, s-maxage=300. In plain words: browsers must check every time, shared caches may keep it for 300 seconds. (The HTTP caching lesson explains s-maxage in detail.)

1. 08:00:00, first user. No row for the key shop.com/shoes/air-1, so the PoP asks the origin, which renders the page in 180 ms and answers 200 OK with ETag: "v41". The PoP stores it and answers with Age: 0. A miss.
2. 08:00:37, second user. Row found, stored 37 seconds ago, allowed 300. The PoP answers at once with Age: 37. A hit.
3. 08:05:10, a later user. The row is 310 seconds old: stale. The PoP asks the origin with If-None-Match: "v41", gets 304 Not Modified with no body, resets the row's clock, and serves its copy. A revalidation.
4. 08:05:10, 400 more users in the same second. A good CDN sends one request to the origin and holds the other 400 until it returns. This is request collapsing. Without it, a popular page that expires sends a burst of identical requests to the origin: the thundering herd.

Reading what the CDN did

You can see each of these outcomes in the response headers, which makes debugging a CDN much less mysterious. Open DevTools, click a request, and look for these.

When a page is slow, the first question is always: was it a hit? If Age stays at 0 on every reload, the CDN is not caching that URL at all, and no amount of origin tuning will make it feel like a cached page.

Edge caching rules: what gets stored, and for how long

A CDN decides whether to store a response from two places: the headers your origin sends and the rules you configure at the CDN. Headers travel with each response, so the team that writes the page controls them. Rules can override headers for whole groups of URLs, such as everything under /static/.

Most teams use both: headers carry the per-response truth (this page is public, this one is private), rules carry the policy (strip tracking parameters, never cache /api/cart). When they disagree, the rule usually wins, which is both its power and its danger.

Headers that steer the edge

You often want different lifetimes for the browser and the CDN. HTML is the classic case: the CDN may keep it for five minutes because you can purge it, but the browser should check every time because you cannot delete a copy in a user's browser. The HTTP caching lesson covers these headers in depth; here is what each means at the edge.

HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
Cache-Control: public, max-age=0, must-revalidate
CDN-Cache-Control: max-age=300, stale-while-revalidate=60, stale-if-error=86400
ETag: "v41"
Cache-Tag: product-air-1,category-running

What a CDN refuses to cache by default

Even with friendly headers, CDNs are careful by default. The exact defaults differ between vendors, so always check yours, but these are common:

• Methods other than GET and HEAD. A POST changes things on the server, so it always goes to the origin.
• Responses with Set-Cookie. Such a response might be starting a session; storing it would hand that cookie to the next user. Many CDNs, Fastly and Cloudflare among them, skip these unless a rule forces caching.
• Requests with an Authorization header. A shared cache must not reuse the response unless the response explicitly allows it, for example with public or s-maxage.
• HTML, on some CDNs. Cloudflare, for example, decides by file extension: it caches .js, .css, images, and fonts by default, but not HTML or JSON until you add a rule. Others follow your headers and apply a default lifetime when there are none.
• Most errors. Server errors such as 500 are usually not stored, while a 404 often is, for a few minutes. That negative caching protects the origin from bots asking for missing pages, but a long lifetime on a 404 can hide a page that was just published.

Cache keys and Vary: which requests are the same request

The cache key is the string the PoP looks up in its table. Two requests with the same key get the same stored response; two with different keys are strangers, even if the origin would send identical bytes. So the key decides correctness (does each user get the right response?) and efficiency (how many users share each copy?).

By default, most CDNs build the key from the host, the path, and the full query string. shop.com/shoes?page=2 is one key. Headers and cookies are not in the default key. (CloudFront's recommended cache policy leaves out the query string too.) That default is a reasonable start, and it is wrong in both directions for many real sites.

Two ways to get the key wrong

• Too narrow: something that changes the response is missing. The page at /shoes shows prices in the currency from a currency cookie, but the cookie is not in the key. The first visitor from India stores a page in rupees, and every visitor after them gets rupees. A correctness bug, and often a privacy bug.
• Too wide: something that does not change the response is included. Newsletter links add ?utm_source=mail; ads add a unique gclid= or fbclid= per click. Each is a new key, so each click is a miss. One page splits into thousands of copies that each get one hit, and your hit ratio quietly collapses.

The rule fits in one sentence: the key must contain every input that changes the response, and nothing else. Everything in this chapter is a tool for getting closer to that rule.

Normalizing the query string

Query strings are the biggest source of accidental keys. Every major CDN can apply these normalizations in a rule before the lookup:

1. Drop tracking parameters: utm_*, gclid, fbclid, and similar. Analytics scripts still read them from the URL bar; only the key changes.
2. Sort the remaining parameters, so ?color=red&size=9 and ?size=9&color=red share one key.
3. Prefer an allowlist for pages you know well. The listing reads only page, sort, and color, so the key keeps exactly those and ignores whatever a partner invents next month.
4. Normalize values the origin treats the same, such as lowercasing ?Color=Red.

Be careful with allowlists: if the origin later reads a new parameter, say ?view=grid, and nobody adds it to the key, every user gets whichever view was cached first. Keep the key rule next to the code that reads the parameters.

The Vary header

Sometimes the origin chooses the response from a request header, not the URL. The Vary response header says which ones. Vary: Accept-Encoding means: I chose this body based on Accept-Encoding, so store a separate copy for each value. A cache that honors Vary adds those header values to the key for that URL.

Vendors treat Vary differently. Fastly honors any header listed in it. Cloudflare honors Vary: Accept-Encoding, plus Vary: Accept for images when you turn that on, and ignores the rest, so Vary: Cookie there separates nobody and you use custom cache keys instead. Check your CDN's documentation, and test with two requests that differ only in that header.

Custom keys: put exactly what matters in

The cleaner tool is a custom cache key: a CDN rule listing which query parameters, headers, and cookies form the key, often after transforming them. Good additions are small, bounded values:

• A device class (mobile, tablet, desktop) worked out at the edge from User-Agent, if the HTML truly differs. Three values instead of thousands.
• A country from the CDN's geolocation, if prices or legal banners differ by country.
• One cookie value with few possible values, such as ab_bucket=b or currency=INR. Never the whole Cookie header, never a session id.

Each value multiplies the copies. A page keyed on 3 device classes, 4 currencies, and 2 experiment buckets has 24 copies per PoP, each with its own first miss: fine for a busy page, terrible for one with 10 views a day. Try it in the lab: change the rules and watch which requests share a copy.

Caching HTML vs hashed assets

A web app sends two very different kinds of files. Hashed assets have a fingerprint of their content in the name, like app.3f9a1c.js. HTML documents live at stable URLs like /shoes/air-1 that keep their name when their content changes.

This decides everything. A hashed file never changes: new content gets a new name, so every cache can keep it forever. An HTML URL can change at any moment, so every cache that holds it needs a way to find out.

Hashed assets: cache forever, never purge

Serve hashed files with Cache-Control: public, max-age=31536000, immutable. At the CDN they become near-permanent rows that you never purge. A deploy that changes the bundle produces app.8b20e4.js, a brand-new key, and the new HTML points to it.

The mistake to avoid is deleting old assets on deploy. A user who loaded the old HTML five minutes ago will lazy-load checkout.3f9a1c.js when they click Buy. If that file is gone, the click fails with a chunk-load error.

Keep old assets for days, longer than your longest HTML lifetime plus a typical session. Some hosts call this skew protection.

HTML: short lifetimes, or long lifetimes with purging

Rendering a product page might cost 150 ms of origin time. Caching it for even 60 seconds means a page viewed 1,000 times a minute is rendered once a minute per PoP. But the page must also update when the price changes. There are three common answers.

The order of a safe deploy

Together these rules give a deploy sequence that never shows a broken page: new assets exist before any HTML points at them, and old assets outlive any HTML that still points at them.

1. Upload the new hashed assets next to the old ones. Nothing references them yet, so nothing changes for users.
2. Switch the origin to serve the new HTML.
3. Purge the HTML at the CDN (by tag, or everything tagged html). PoPs fetch the new HTML on the next request.
4. Days later, delete assets that no live HTML can reference anymore.

Purging: removing copies from 300 cities

A lifetime says how long a copy may live. A purge (or invalidation) removes copies early. Purging is what lets HTML have a long CDN lifetime and still show a price change within seconds. It is also where most CDN incidents start: a purge that misses one URL leaves stale content up for the full lifetime.

CDNs offer purges at several sizes, and picking the right size is the skill.

• By URL. Remove one key. Precise, but you must know every URL that shows the changed data.
• By prefix or hostname. Remove everything under /blog/. Coarse, useful after a section redesign.
• Everything. Simple, and the most expensive: every PoP misses on everything at once.
• By tag. Remove every response labelled with a tag, wherever and however it is cached. This is the one to learn.

Why purging by URL falls short

Say the Air 1 drops from 120 to 99. Which cached responses show that price? The product page; the running category, pages 1 to 3, each with ?sort=price and ?sort=new versions; the home page carousel; search results for "air"; and the JSON at /api/products/air-1.

That is dozens of keys, the list changes whenever a designer adds a carousel, and the code that changes the price cannot know them all.

Surrogate keys and cache tags

Tags flip the problem around. Instead of the purger working out which URLs to remove, each response says what data it contains when it is rendered. The template knows it printed product air-1 and category running, so it adds those tags to its own response.

# Fastly reads a space-separated list
Surrogate-Key: product-air-1 category-running page-home

# Cloudflare reads a comma-separated list
Cache-Tag: product-air-1,category-running,page-home

# Price of air-1 changed: purge the tag everywhere
POST https://api.fastly.com/service/SERVICE_ID/purge/product-air-1
Fastly-Key: API_TOKEN

POST https://api.cloudflare.com/client/v4/zones/ZONE_ID/purge_cache
{ "tags": ["product-air-1"] }

The CDN keeps an index from each tag to the rows that carry it, and removes the tag header before the response reaches the browser. When the price changes, the backend sends one purge for product-air-1, and every page, variant, and JSON response that printed that product is gone from every PoP, usually within a second.

Good tags name data, not pages: product-air-1, category-running, author-42. A broad tag like html on every page lets a deploy purge all HTML without touching assets. Vendors cap the number and size of tags per response, so a listing of 200 products may need category-running rather than 200 product tags.

Purge pitfalls

• The purge race. The backend writes the new price and purges, but the CDN's refill reaches a database replica that has not caught up, gets the old price, and caches it for a day. Purge only after the write is visible to the origin's reads, or purge again a few seconds later.
• The purge storm. Purging everything on every deploy makes every PoP miss every popular page at once, just as new code starts. Purge narrowly by tag, and use shielding (covered later).
• Hard vs soft purge. A hard purge deletes the copy, so the next request waits for the origin. A soft purge (Fastly's term) marks it stale instead, so stale-while-revalidate and stale-if-error can still use it. Use soft purges for routine edits, hard purges for content that must vanish now, such as a legal takedown.
• The browser still has a copy. A purge only clears the CDN. HTML sent with max-age=3600 stays in users' browsers for an hour, which is why HTML gets a browser max-age of 0.

Personalized pages: caching what is shared

Look at a product page for a signed-in user. The header says "Hi, Asha" with 3 items in the cart, and a row shows "Recommended for you". Everything else (photos, title, price, reviews) is the same for every visitor: often more than 90% of the bytes.

If the server renders the greeting into the HTML, the whole page becomes personal and must be private, and you lose the CDN on your busiest pages for a few dozen bytes. Every approach below uses one idea: separate the shared part from the personal part, cache the shared part at the edge, and deliver the personal part another way.

Four ways to split the page

• Client-side fragments. The CDN caches one public page with empty slots; the browser then calls /api/me (sent private) and fills them. Simple and common, at the cost of a second request and a slot that pops in late, so reserve its space to avoid layout shift.
• Edge-side includes (ESI). The cached HTML contains include tags pointing at fragment URLs. The CDN fetches each fragment, stitches them in, and sends one complete document. Supported by Akamai, by Varnish, and by Fastly in a basic form.
• Edge compute. A small program at the PoP (Cloudflare Workers, Fastly Compute, Netlify Edge Functions) reads the cached shell, fetches the personal piece, and streams the result. ESI's idea as ordinary code, with full control over timeouts.
• Segments in the key. Show per-segment content instead of per-user: logged in or not, country, experiment bucket. Put that one bounded value in the key. Ten segments means ten copies per page, each shared by thousands.

<header>
  <a href="/">Shop</a>
  <esi:include src="/fragments/user-menu" />
</header>
<main>
  <!-- public product content, cached for a day -->
</main>

Cookies and the cache key

Cookies are where personalization meets the key, and where most leaks happen. The Cookie header holds every cookie for the site: session id, analytics ids, consent, experiment buckets. Keying on the whole header (or sending Vary: Cookie) gives each user their own copy, the same as not caching. Ignoring cookies is fine only if the response truly depends on none of them.

The safe pattern: make responses that render personal data private at the origin (or bypass the cache when a session cookie is present), and if a page varies by something bounded, key on only that cookie's value. Many sites split traffic in two: anonymous visitors, often the majority, get fully cached pages; signed-in users get fragment-based or uncached ones.

Cache hit ratio, tiered caches, and origin shielding

The cache hit ratio is the share of requests the CDN answered from its own copies. It tells you whether your CDN setup works. A static site might sit at 99%, a store with server-rendered pages and good keys at 90 to 95%, and a site with fragmented keys at 30% without knowing it.

What matters for the origin is the other side: origin requests = total requests x (1 - hit ratio). So small changes at the top are large at the bottom. Going from 98% to 99% halves origin traffic: during the sneaker drop, 5,400 requests per second become 2,700.

Request hit ratio and byte hit ratio

CDNs report two versions. The request hit ratio counts requests and predicts origin CPU load. The byte hit ratio counts bytes and predicts your egress bill. A site whose tiny icons hit but whose 5 MB videos often miss might show 96% of requests as hits but only 70% of bytes.

Watch both, split by content type.

Where misses come from

• Cold misses. The first request for a key at each PoP. With 300 PoPs, a new file can cost up to 300 origin fetches.
• Expiry misses. The lifetime ran out, so the copy is refetched or revalidated.
• Eviction misses. Storage is limited, so a PoP drops rarely used rows. The long tail of your catalogue is evicted long before its lifetime ends.
• Fragmentation misses. The same content stored under many keys, each paying its own cold misses.
• Not cacheable at all. private, no-store, Set-Cookie, or no rule allowing storage. Often the largest group.

A small model you can reason with

Take one URL at one PoP, getting a steady r requests per second, with a lifetime of T seconds. After each miss, the copy serves about r x T hits before it expires, then the next request misses again. So the hit ratio for that URL at that PoP is about rT / (1 + rT).

A home page at 50 requests per second per PoP with a 60-second lifetime: rT = 3,000, hit ratio 99.97%. A product page seen once every 10 minutes per PoP, same lifetime: rT = 0.1, hit ratio 9%. Hit ratio depends on traffic per key per PoP. More PoPs, more keys, or shorter lifetimes spread traffic thinner and lower it.

Tiered caching and origin shields

The per-PoP problem has a structural fix: a second layer. With tiered caching, edge PoPs that miss ask a larger regional cache before the origin. With an origin shield, one chosen location sits in front of the origin and every edge miss goes through it. Fastly calls this shielding, CloudFront offers Origin Shield on top of its regional edge caches, and Cloudflare calls it Tiered Cache.

Now 300 PoPs share the shield's copy, and the origin sees roughly one fetch per object per lifetime. The long-tail page that missed 91% of the time at each PoP mostly hits at the shield, which collects all its traffic. With request collapsing at the shield, a purge reaches the origin as one request per object.

The costs: an edge miss takes an extra hop, so place the shield near the origin, not near users; and the shield is a concentration point, so allow failover to direct origin fetches. Use the calculator to see the model at work: raise the number of URLs to simulate a long tail, then turn the shield on.

When it goes wrong: poisoning, deception, and outages

A shared cache multiplies whatever it stores. A good response is served to a million people for the cost of one; so is a bad one. Most serious CDN incidents follow one pattern: something private, wrong, or attacker-controlled got stored under a key that ordinary users request.

Two attacks are worth knowing by name: cache poisoning and web cache deception. Both exploit a gap between what the cache thinks makes two requests the same and what the origin actually uses to build the response.

Cache poisoning

In cache poisoning, an attacker crafts a request whose harmful response the CDN stores under a normal key. The trick is an unkeyed input: a header or parameter the origin uses but the key ignores.

1. The origin builds absolute script URLs from the X-Forwarded-Host header when present, a leftover from an old proxy setup.
2. The attacker requests GET / with X-Forwarded-Host: evil.example. The key is still shop.com/.
3. The origin renders a page whose script tag points at https://evil.example/app.js, and the CDN stores it for 10 minutes.
4. Every visitor to the home page at that PoP now runs the attacker's script. The attacker repeats the request before expiry to keep it there.

A quieter variant is cache-poisoned denial of service: an oversized or malformed header makes the origin return a 400, and the CDN caches that error for the page's URL. The defences for both:

• Never build a response from an input that is not in the key. Add it to the key, or strip it at the edge.
• Strip forwarding headers such as X-Forwarded-Host from client requests, so only the CDN sets them.
• Do not cache error responses, or only for a few seconds.
• Parse URLs the same way at the edge and the origin. Differences in how each reads ;, encoded characters, or repeated parameters let an attacker hide input from the key.

Web cache deception

Web cache deception goes the other way: the attacker tricks the cache into storing a victim's private page. They send the victim a link to https://shop.com/account/orders/x.css. Many frameworks ignore the unknown tail and render the account page.

The CDN sees .css, treats it as a static file, and caches it. The attacker opens the same URL and reads the victim's orders.

The defences: return 404 for unknown paths instead of falling back to a real page, send private or no-store on every personal response, and set the CDN to respect origin headers rather than caching by file extension. Some CDNs also check that the Content-Type matches the extension before caching.

Outages and stale content

The CDN can also save you during an outage. With stale-if-error, a PoP that cannot reach the origin keeps serving its last good copy, so public pages stay up. Set it on HTML and public JSON, with a window you are comfortable with: often an hour to a day.

The opposite failure is content that will not go away: a wrong price that survives the fix because a purge failed or raced the database. Log every purge and its result, alert when a purge call fails, and keep a runbook: purge by tag, then by URL, then everything.

In real products and in a system design interview

Frameworks now write most of these headers for you, which makes it more important to know what they mean. Translate each caching feature back into this lesson's pieces: a lifetime, a key, a tag, or a purge.

• Next.js sends s-maxage and stale-while-revalidate for static and revalidated pages, and offers revalidateTag and revalidatePath: tag and URL purges for its own cache, the same idea as surrogate keys.
• Remix and React Router let each route export a headers function, so the public-or-private decision sits next to the route's data loading.
• Fastly users tag responses with Surrogate-Key and purge by key from their CMS, often with soft purges.
• Cloudflare uses Cache Rules for lifetimes and custom keys, Cache-Tag for tag purges, and Tiered Cache for a shield layer.
• CloudFront puts the key in a cache policy (which headers, cookies, and query strings to include) and offers Origin Shield.

When to bring it up

Bring up CDN caching in any design that is read-heavy, global, or spiky: a news site, a product page, a marketing site with launches, a media-heavy feed. Draw the CDN between users and the origin in the high-level architecture, then come back to it for performance and scale.

For an app that is mostly personal data behind a login (an email client, a dashboard), keep it short: the CDN serves hashed assets and the app shell, and API responses are private. That shows judgement, not a gap.

What to draw and say

1. Draw the path: browser, edge PoP, shield, origin.
2. Give each response type a policy: hashed assets for a year; public HTML at the CDN for minutes with stale-while-revalidate, or longer with tag purges; personal data private.
3. Name the key: host, path, allowlisted query parameters, and at most one bounded segment such as country.
4. Explain freshness: when a price changes, the backend purges its tag after the write commits; browsers keep max-age=0 on HTML.
5. Estimate: traffic, hit ratio, origin load. "At 50,000 requests per second and a 97% hit ratio, the origin sees 1,500. A shield cuts the cold misses from our 300 PoPs."
6. Close with risk: personal pages never shared, unkeyed headers stripped, stale-if-error for outages.