CDNs and the edge

Serving from close to the visitor

Ana opens the Mint tea shop's page. The shop's own server is far away from her. Every round trip takes 200 ms. A round trip is a message from her browser to the server and the answer back. A millisecond (ms) is a thousandth of a second.

Her page starts to arrive after 806.2 ms. Then Ben opens the same page, in his own browser, and waits just as long. Every visitor makes the same long trip.

What if a copy of the page waited much nearer to them? That's the idea behind a CDN, a content delivery network. A CDN is a set of servers spread out near visitors. Each one keeps copies of a site's files and hands them out.

Here are both ways, with the times the course measured:

Look at the arrows. With no CDN, both trips go all the way to the shop's server. With a CDN, Ben's stops at a server near him, which kept a copy.

Latency vs bandwidth compared a CDN to a tea shop with a branch in every town. HTTP caching in depth called it a shared cache: the office fridge that everyone uses. This lesson opens one up and times it.

Two names first. The shop's own server is the origin: the place every copy comes from. A CDN's servers near the visitors are its points of presence, or PoPs. The course built a small CDN with two of them, PoP A and PoP B. Here is Ben's page two ways: straight from the origin, and from PoP A. PoP A had a copy of it:

1// Ben's time to first byte (TTFB), in ms: the median of 5 runs.2const straightToTheShop = 806.2;3const fromPopA = 65.5;4 5console.log(`Straight to the shop: ${straightToTheShop} ms.`);6console.log(`From PoP A, near Ben: ${fromPopA} ms.`);

• Line 1 says what the numbers are. Time to first byte, or TTFB, is when the first byte of the page arrived. It's counted from the start of the visit. The navigation timeline took it apart.
• Each time in this lesson is the median of 5 runs: the middle value, once they're sorted.
• Line 2 holds Ben's visit straight to the origin. Line 3 holds his visit to PoP A.
• Lines 5 and 6 print Straight to the shop: 806.2 ms. and From PoP A, near Ben: 65.5 ms.

From PoP A, his page started arriving in less than a tenth of the time. Why it matters: a faster server can't shorten the trip to it. Only a shorter distance can, as Faster server responses found.

How the course built its CDN

The course ran it all on one computer. nginx, a real web server, played every part of the CDN:

• The shop. Its page, /, links a stylesheet and a photo. The origin takes 200 ms to build the page. The stylesheet and the photo may be kept for a year, as in A caching strategy.
• Two PoPs, PoP A and PoP B, each with its own cache. Each is 20 ms from the visitors and 150 ms from the origin.
• A shield: one more cache, near the origin, used in some setups. The section on tiers explains it.
• Ana and Ben, two visitors, each with a new browser and an empty browser cache.
• A small program held every message for its round trip. Every setup ran 5 times, from empty caches.

Think of a chain of shops:

Those last two pairs are the words you'll see most. When a PoP has a copy and answers from it, that's a cache hit. When it has none and has to ask the origin, that's a cache miss. nginx writes them in its log as HIT and MISS.

PoPs sit at the edge of the network, nearest the visitors. web.dev calls them "edge caches".

Every visit in this lesson is in this playground. Pick a setup, then a visitor:

Want to see how a CDN is built inside? See CDN caching and DNS, CDNs, and the edge in the system design course.

Points of presence and anycast

Ben's 65.5 ms came from a hit. But the first visitor to a PoP always misses. Does a CDN help her at all?

Ana was that first visitor, with nothing cached anywhere. PoP A had to ask the origin for every file:

1// Ana's first visit, with nothing cached anywhere. Her TTFB in ms: the median of 5 runs.2const straightToTheShop = 806.2;3const throughPopA = 570.1;4 5console.log(`Straight to the shop: ${straightToTheShop} ms.`);6console.log(`Through PoP A, which had to ask the shop: ${throughPopA} ms.`);

• Line 1 says this is Ana's first visit. Every cache was empty.
• Line 2 holds her visit straight to the origin. Line 3 holds her visit through PoP A, which missed.
• Lines 5 and 6 print Straight to the shop: 806.2 ms. and Through PoP A, which had to ask the shop: 570.1 ms.

Ana was quicker through the CDN, without a single hit. How can a detour beat going straight there?

Counting the round trips

Before a browser can ask for a page, it needs two round trips. TCP connections and slow start showed the first: the TCP hello. It's like waiting for someone to pick up the phone. TLS without the wait showed the second: agreeing on a secret code. The request itself is a third round trip.

• Straight to the origin: three round trips of 200 ms take 600 ms. The shop builds the page in 200 ms. That's 800 ms, close to the recorded 806.2.
• Through PoP A: the three round trips take 20 ms each, so 60 ms. PoP A has no line open to the origin yet. It opens one, a 150 ms round trip, then asks, another 150 ms. The shop builds the page in 200 ms. That's 560 ms, close to the recorded 570.1.

The hellos made the difference. Through the CDN, they crossed the short distance. web.dev: "terminating the new connection as close to the user as possible eliminates unnecessary connection setup costs." Why it matters: a CDN speeds up even the files it can't keep.

The PoP stays on the line

Ben came next. In this setup, the CDN kept the stylesheet and the photo, but not the page. The page was sent with no-cache. That means a cache must check with the origin before each use. So PoP A had to ask the origin for it again:

1// Ben came right after Ana. The CDN kept the files but not the page, so PoP A asked the shop again.2// TTFB in ms: the median of 5 runs.3const ana = 570.1;4const ben = 418.8;5 6console.log(`Ana: ${ana} ms. Ben: ${ben} ms.`);7console.log(`Ben's page came ${(ana - ben).toFixed(1)} ms sooner.`);

• Lines 1 and 2 say what happened. Ben came right after Ana, and PoP A asked the origin for his page too.
• Lines 3 and 4 hold their times.
• Line 6 prints Ana: 570.1 ms. Ben: 418.8 ms. Line 7 subtracts, and prints Ben's page came 151.3 ms sooner.

Both visits missed. Yet Ben's page came about one 150 ms round trip sooner. That's the hello PoP A didn't say again. Like the phone call in TCP connections and slow start, PoP A stayed on the line after Ana's visit. Count it: 60 ms to reach PoP A, 150 ms to ask, and 200 ms to build make 410 ms, close to the recorded 418.8.

A line that's already open is called a warm connection. web.dev: a CDN's trip to the origin "often includes existing, persistent connections with the origin." Persistent means kept open.

When the CDN kept the page too, Ben's visit was the hit from the overview. Here are all three setups:

One address, many PoPs

How does Ana's browser reach a PoP near her? A CDN can give all its PoPs the same address. That's called anycast. A datagram is one packet of data. RFC 4786 defines anycast as making one address "available in multiple, discrete, autonomous locations, such that datagrams sent are routed to one of several available locations." The internet's routers pick the location. RFC 4786 says the choice depends partly on "the point in the network at which the request originates". That's where the visitor is.

It's like one phone number for a chain of taxi firms. Which office answers depends on where you call from. The course's small CDN didn't use anycast. It sent each visitor to a PoP it chose.

Cache hit ratio

Ana filled PoP A's cache, and Ben found it full. Every PoP starts out this way:

web.dev calls this "origin pull". The first time a file is asked for, "the CDN will request it from the origin server and cache the response." A cache fills up one miss at a time.

How often did visitors get the quick answer? That's the cache hit ratio. web.dev defines it as "the number of cache hits divided by the number of total requests during a given time interval." Here is PoP A's ratio, over Ana's visit and Ben's.

1// What PoP A did with each request when it kept the page too. The same in all 5 runs.2const ana = ["MISS", "MISS", "MISS"]; // the page, the stylesheet and the photo3const ben = ["HIT", "HIT", "HIT"];4 5const all = [...ana, ...ben];6const hits = all.filter((status) => status === "HIT").length;7console.log(`${hits} hits out of ${all.length} requests: a hit ratio of ${(hits / all.length) * 100}%.`);

• Line 1 says what the lists hold: PoP A's answers, from nginx's log. They were the same in all 5 runs.
• Line 2 holds what PoP A did with Ana's three requests: the page, the stylesheet and the photo. All three missed.
• Line 3 holds Ben's three. All three hit.
• Line 5 puts the six together. Line 6 counts the hits.
• Line 7 prints 3 hits out of 6 requests: a hit ratio of 50%.

Half of PoP A's answers were hits. With more visitors, the ratio climbs, because only the first one misses. Why it matters: every miss sends a visitor on the long trip, and gives the origin work.

Hits in your own field data

web.dev says your CDN should report its hit ratio to you. But what does a miss cost real visitors? For that, you need the hit or miss for each visit. Put it next to the visit's largest contentful paint (LCP). A Server-Timing header can carry it. MDN says it "communicates one or more performance metrics about the request-response cycle to the user agent", the browser.

Each metric in it has a name, and can have a description, called desc. The course's PoPs added one metric to every response: cdn-cache, with nginx's HIT or MISS as its description. When Ben's page came from PoP A's cache, his browser read cdn-cache HIT. A page can read it like this:

1const [page] = performance.getEntriesByType("navigation");2const cache = page.serverTiming.find((entry) => entry.name === "cdn-cache");3sendToAnalytics({ cdnCache: cache?.description ?? "unknown" });

• Line 1 gets the page's navigation entry, as in The navigation timeline.
• Line 2 looks through its serverTiming list for the CDN's metric, cdn-cache.
• Line 3 sends its description, HIT or MISS, with the rest of the visit's data. sendToAnalytics stands for your own code that sends it, as in Sending performance data.

Now you can split your field data into visits that hit and visits that missed. If the misses are much slower, the next two sections show two ways to raise the ratio.

Cache keys and query strings

The shop runs five campaigns, like a newsletter. Each link adds a tag to the address. That way the shop can count where visitors came from: /?utm_source=campaign1, /?utm_source=campaign2, and so on. The part after the ? is the query string. The page is the same whatever it says.

Does the CDN know that? A cache files each copy under a cache key, and looks it up by that key. A caching strategy showed this. By default, the key holds the whole address, query string and all. Here are two:

• Cloudflare's default key includes the "URI with query string - for example, /logo.jpg?utm_source=newsletter".
• nginx's default key ends in $request_uri. nginx's docs describe it as the "full original request URI (with arguments)". The arguments are the query string.

So to a CDN, every campaign link is a different page. web.dev: "By default, a CDN would cache example.com/blog and example.com/blog?referral_id=2zjk separately even though they are likely the same underlying resource."

A caching strategy had the opposite problem. Its key left out something that mattered: the visitor's cookie. So Ben got a cup with Ana's name on it. Here, the key holds something that doesn't matter. So the same page is stored again and again.

The course tried both keys at PoP A. Here are its cache settings. This one leaves the query string out:

1proxy_cache edge;2proxy_cache_key $scheme$host$uri;3proxy_pass http://origin;4proxy_http_version 1.1;5proxy_set_header Connection "";6proxy_set_header Host $host;7add_header Server-Timing "cdn-cache;desc=$upstream_cache_status";

• Line 1 turns on the cache, which the course named edge.
• Line 2 sets the key: the scheme, the host, and $uri, the path without the query string. For the key with the query string, the course used $request_uri here instead.
• Line 3 sends a miss on to origin. That's a group of servers, set up elsewhere in the file. It keeps idle lines to the origin open.
• Lines 4 and 5 let those lines stay open between requests, as nginx's docs ask. Line 6 passes on the visitor's host name.
• Line 7 adds the Server-Timing metric from the last section. $upstream_cache_status holds what the cache did, such as HIT or MISS.

Five visitors then came through PoP A, one after another. Each came from a different campaign link:

1// Five visitors came through PoP A, each from a different campaign link: ?utm_source=campaign1 to campaign5.2// How many of their 5 pages came from PoP A's cache. The same in all 5 runs.3const keyWithTheQuery = 0;4const keyWithoutIt = 4;5 6console.log(`Key with the query string: ${keyWithTheQuery} of 5 pages from the cache.`);7console.log(`Key without it: ${keyWithoutIt} of 5 pages from the cache.`);

• Lines 1 and 2 say what happened. The numbers count pages that came from PoP A's cache.
• Line 3 is the key with the query string. Line 4 is the key without it.
• Lines 6 and 7 print Key with the query string: 0 of 5 pages from the cache. and Key without it: 4 of 5 pages from the cache.

With the query string in the key, every visitor waited for the origin. Each link was new to PoP A. Without it, Visitor 1 filled the cache, and the other four got a hit. Here is each visitor's page:

The stylesheet and the photo have no query string. So they hit from Visitor 2 on, with either key. Counting them too, here is PoP A's hit ratio:

1// All 15 requests PoP A got from the five visitors: 5 pages, 5 stylesheets and 5 photos.2const requests = 15;3const hitsWithTheQuery = 8;4const hitsWithoutIt = 12;5 6const ratio = (hits) => `${((hits / requests) * 100).toFixed(1)}%`;7console.log(`Key with the query string: a hit ratio of ${ratio(hitsWithTheQuery)}.`);8console.log(`Key without it: a hit ratio of ${ratio(hitsWithoutIt)}.`);

• Line 2 counts PoP A's requests: five visitors, three files each.
• Lines 3 and 4 hold the hits with each key, from nginx's log.
• Line 6 turns a number of hits into a percentage.
• Lines 7 and 8 print Key with the query string: a hit ratio of 53.3%. and Key without it: a hit ratio of 80.0%.

One line of settings took PoP A from 53.3% to 80.0%. And the four extra hits were all pages: the one response the origin takes 200 ms to build.

Which parameters to leave out

Leave out only the parameters that don't change the page, like utm_source. On Cloudflare, a cache rule can exclude them from the key.

Keep every parameter that does change the page, like ?page=2 on a list. Leave that one out, and visitors get the wrong page: Ben's cup again. web.dev warns about caching that isn't precise enough. It "results in incorrect responses being served to users."

Tiered caching and origin shields

So far, Ana and Ben both went to PoP A. A real CDN can have many PoPs, so that more visitors have one nearby. But each PoP has its own cache. What happens when Ben goes to a different one?

In the next setup, Ana visited through PoP A, as before. Then Ben visited through PoP B. Both PoPs could keep the page:

1// Ana visited through PoP A. Then Ben visited through PoP B, whose cache was empty.2// Ben's TTFB in ms: the median of 5 runs. The requests that reached the shop: the same in every run.3const ttfb = 417.8;4const toTheShop = 3;5 6console.log(`Ben's page started arriving at ${ttfb} ms.`);7console.log(`The shop got ${toTheShop} of his requests.`);

• Lines 1 and 2 say what happened, and what the numbers are.
• Line 3 holds Ben's time. Line 4 counts his requests that went on to the origin.
• Lines 6 and 7 print Ben's page started arriving at 417.8 ms. and The shop got 3 of his requests.

PoP A had Ana's copies. But PoP B didn't, so all three of Ben's files came from the origin. His wait was close to his miss at PoP A, 418.8 ms. In the course's CDN, both PoPs ran inside one nginx. So PoP B could use the line PoP A had opened, and skip the hello.

Latency is the time a message takes to arrive. web.dev says CDNs with many PoPs "can deliver lower latency but may experience lower cache hit ratios as a result of traffic being split across more caches." More PoPs put a shop nearer every visitor. They also give the CDN more shelves to fill.

That depot is an origin shield. CloudFront, Amazon's CDN, describes its Origin Shield as "an additional layer in the CloudFront caching infrastructure".

Cloudflare calls the idea Tiered Cache. Its lower tiers are generally the data centres nearest the visitors. Its upper tiers sit behind them. When a lower tier misses, its docs say: "the lower-tier must ask an upper-tier to see if it has the content. If the upper-tier does not have the content, only the upper-tier can ask the origin for content." That's the shield.

The course added a shield 30 ms from the origin, and 120 ms from each PoP. Then it ran the same two visits again:

1// The same two visits, with a shield between the PoPs and the shop.2const ttfb = 188.7;3const toTheShop = 0;4 5console.log(`Ben's page started arriving at ${ttfb} ms.`);6console.log(`The shop got ${toTheShop} of his requests.`);

• Line 1 says what changed: a shield between the PoPs and the origin.
• Lines 2 and 3 hold Ben's time, and his requests that reached the origin.
• Lines 5 and 6 print Ben's page started arriving at 188.7 ms. and The shop got 0 of his requests.

PoP B still missed. But this time it asked the shield. And the shield had Ana's copies of all three files. Ben's page came in less than half the time, and the origin heard nothing.

Two things made it quick. The shield was nearer than the origin: 120 ms away, not 150. And it already had the page, so nobody waited 200 ms for the shop to build it.

A shield costs one more stop. For Ana, PoP A asked the shield, and the shield asked the origin. Her page started arriving at 573.9 ms, against 571.2 ms with no shield. The stop cost little, because 120 ms plus 30 ms makes 150. web.dev calls tiers a trade of "slightly greater latency for a higher likelihood that the resource can be served from a CDN cache."

The origin gains too. Over both visits, it got six requests with no shield, and three with one. Cloudflare says tiers reduce "origin load" by "limiting the number of data centers that can ask the origin for content".

Every path a page can take

Here is every path the page took through the CDN. Read it from the top, quickest first. Each row is one path:

Lower rows lost more time on long trips, and on the shop building the page.

Caching HTML at the edge

Ben's quickest visit, 65.5 ms, came when PoP A kept the page itself. That's harder than keeping a file with a hash in its name. A hash is a short code worked out from a file's content, so the name changes when the file does. The page's address never changes, but its content can.

A caching strategy checked the page on every visit, with no-cache. For a page that's the same for everyone, there's a middle way: s-maxage. It's the office fridge's own rule from HTTP caching in depth. The shop sent Cache-Control: max-age=0, s-maxage=60:

• max-age=0: a browser's copy is stale at once, too old to use without checking. So the browser checks on every visit.
• s-maxage=60: a shared cache, like a PoP, may keep the page for 60 seconds. MDN says browsers' own caches ignore it.

1// Ben's visit to PoP A, right after Ana's. What PoP A told his browser, the same in all 5 runs,2// and his first contentful paint (FCP) in ms, the median of 5 runs.3const noCache = { status: "MISS", fcp: 468 };4const sMaxage = { status: "HIT", fcp: 116 };5 6console.log(`no-cache: PoP A said ${noCache.status}, and the page painted at ${noCache.fcp} ms.`);7console.log(`s-maxage=60: PoP A said ${sMaxage.status}, and the page painted at ${sMaxage.fcp} ms.`);

• Lines 1 and 2 say what the numbers are. First contentful paint, or FCP, is when the first text or image appeared on Ben's screen.
• Line 3 is the page sent with no-cache. Line 4 is the page sent with max-age=0, s-maxage=60.
• Each holds what PoP A told Ben's browser in its cdn-cache metric, and his FCP.
• Lines 6 and 7 print no-cache: PoP A said MISS, and the page painted at 468 ms. and s-maxage=60: PoP A said HIT, and the page painted at 116 ms.

With the page at the edge, Ben saw the shop at 116 ms instead of 468. His whole visit stayed at PoP A: the page, the stylesheet and the photo were all hits. web.dev says a CDN improves LCP by "reducing TTFB in connection setup and in caching the document". Ben's visit had both.

When the page changes

A page kept at the edge can go out of date. A caching strategy showed the fix: a purge as a step of every deploy. It's like collecting old menus from the tables. A purge tells the CDN to drop its copies.

With purging, a changing page can even stay at the edge for a long time. A TTL, time to live, is how long a copy may be kept. web.dev: "cache dynamic content using a long TTL, then purge the resource whenever it is updated."

Personal pages are different. Never keep one at the edge. web.dev: private resources "should not be cached by a CDN." Mark them private. A caching strategy did that with its greeting.

Small jobs at the edge

A personal page can't be shared. But some work can still happen near the visitor. web.dev lists "edge computing" among the features CDNs commonly offer: running your own code at the PoPs.

Cloudflare calls its version Workers. Its docs say that rather than running "on a centralized server", "Workers functions run on Cloudflare's global network". Here is a small one, in the shape Cloudflare's docs use:

1export default {2  async fetch(request) {3    const name = new URL(request.url).searchParams.get("name") ?? "guest";4    return new Response(`Hello, ${name}!`);5  },6};

• Line 1 hands Cloudflare an object.
• Line 2 is its fetch function. Cloudflare's docs say when it runs. A request that reaches one of its data centres calls this function, with the request.
• Line 3 reads the name from the address, like ?name=Ana, or uses guest.
• Line 4 answers with a new Response, made right there. For https://shop.test/hello?name=Ana, it answers Hello, Ana!

The greeting is personal, so no cache could share it. But it's made at the PoP, so it never makes the long trip. If the function fetches something from the origin, that request does. The course didn't time an edge function, so this lesson has no number for one.

Which change fits?

Each case below needs one change more than any other. Ask these questions, in order:

1. Is it personal? Keep it out of the CDN's cache.
2. Is it the same page under many addresses? Take the extra part out of the cache key.
3. Do visitors spread over many PoPs, or does the origin struggle with misses? Add a shield.
4. Is it the same for everyone? Cache it at the edge.

Common mix-ups

• "A CDN only helps files it has cached." No. Ana's first visit missed everywhere, and her page still started arriving at 570.1 ms, not 806.2. Her hellos happened near her.
• "More PoPs always mean more hits." No. Each PoP has its own cache. Ben's first visit to PoP B missed, though PoP A had the page.
• "A longer max-age fixes a low hit ratio." No. When every campaign link has its own key, each one misses the first time, however long the copies are kept.
• "A CDN can't cache HTML." No. With s-maxage=60, Ben's page was a hit at PoP A, and painted at 116 ms.
• "A shield is one more stop, so it's slower." No. Ben's miss at PoP B took 188.7 ms with a shield, and 417.8 ms without one. Ana's first visit was only a little slower: 573.9 ms, against 571.2.
• "An edge function is a kind of cache." No. It builds a new answer for each request. It's quick because it runs near the visitor, not because it kept a copy.

Practice exercises

Answer first, then use the hints.

// Ben's time to first byte (TTFB), in ms: the median of 5 runs.
const straightToTheShop = 806.2;
const fromPopA = 65.5;
const fromPopB = 417.8;

console.log(`Straight to the shop: ${straightToTheShop} ms.`);
console.log(`From PoP A, near Ben: ${fromPopA} ms.`);
console.log(`From PoP B, whose cache was empty: ${fromPopB} ms.`);

// Ben's visit, right after Ana's. What his PoP told his browser, the same in all 5 runs,
// and his first contentful paint (FCP) in ms, the median of 5 runs.
const noCache = { status: "MISS", fcp: 468 };
const sMaxage = { status: "HIT", fcp: 116 };
const atPopBWithAShield = { status: "MISS", fcp: 360 };

console.log(`no-cache: PoP A said ${noCache.status}, and the page painted at ${noCache.fcp} ms.`);
console.log(`s-maxage=60: PoP A said ${sMaxage.status}, and the page painted at ${sMaxage.fcp} ms.`);
console.log(`s-maxage=60, at PoP B with a shield: PoP B said ${atPopBWithAShield.status}, and the page painted at ${atPopBWithAShield.fcp} ms.`);

Check your understanding

For each question, ask how far the request travelled, which cache could answer it, and what made it miss.

Key takeaways

• A CDN puts servers, called PoPs, near your visitors. On a hit, nobody waits for the far-away origin.
• Even a miss is quicker. The hellos happen near the visitor, and the PoP keeps its line to the origin open.
• The hit ratio is hits out of all requests. Read the CDN's cache status from Server-Timing to split your field data.
• Keep tracking parameters out of the cache key, and keep every parameter that changes the page in it.
• A shield turns a miss at a new PoP into a nearby hit, and keeps requests off the origin.
• Keep a page that's the same for everyone at the edge with s-maxage, and purge it when it changes.
• An edge function runs your own code at the PoP, so an answer made for one visitor skips the trip to the origin.

Sources

The facts on this page come from these standards and docs:

• Content delivery networks (CDNs), web.dev.
• RFC 4786: Operation of Anycast Services, the IETF.
• Server-Timing header, MDN.
• Module ngx_http_upstream_module, nginx.
• Cache keys, Cloudflare.
• Module ngx_http_proxy_module, nginx.
• Module ngx_http_core_module, nginx.
• Tiered Cache, Cloudflare.
• Use Amazon CloudFront Origin Shield, Amazon Web Services.
• Cache-Control header, MDN.
• How Workers works, Cloudflare.

Coming next: gzip, Brotli, and Zstandard, also in Fast servers, caches, and networks. Every byte that does travel can be made smaller, and web.dev notes that "CDNs can compress both cacheable and non-cacheable resources." The next lesson compares the formats and their levels.