completefrontend
Open lab My progress
THE WEB PERFORMANCE FIELD GUIDE

QUIC internals

How QUIC carries a page: stream IDs, flow control, and how lost packets are found and sent again. Then 0-RTT's replay risk, and reading it all in a qlog.

By the end, you can
  • 01
    Read streams and flow controlTell a stream's kind from its ID, and say what MAX_STREAM_DATA, MAX_DATA and the BLOCKED frames do.
  • 02
    Explain loss recoverySay why QUIC declared a packet lost: 3 later packets, 9/8 of a round trip, or nothing back at all.
  • 03
    Judge 0-RTTSay what Chrome sends before the handshake ends, and why a copy of it can run twice.
  • 04
    Debug with logsFind a lost packet and its resend in a server's qlog, Chrome's NetLog and qvis.

Asha's bus ride: lost packets, a slower page

Asha opens the Mint tea page on the bus. Her phone's signal comes and goes. Some packets from the server never reach her. Still, the page arrives whole, with every photo. Nothing is missing. It just takes much longer.

So how does the server know a packet went missing? When does it send it again? And while one photo waits, do the others wait too? This lesson answers all three, from inside QUIC, the transport under HTTP/3.

The course loaded the page on Asha's home line from Congestion control: 10 Mbit/s, 20 ms each way. It tried three versions of that line:

  • a clean line, where nothing is lost;
  • a line that drops 1 packet in 25 from the server;
  • a line with a 200 ms tunnel, where nothing gets through either way.
When the page finished loading, on each linePop out in the code editor (opens in a new tab)JavaScript
// When the Mint tea page finished loading on Asha's phone, in ms.// The middle of three visits on each line.const loadMs = { cleanLine: 661.8, dropsOneIn25: 1554.4, tunnel: 1010.2 }; console.log(`Clean line: ${loadMs.cleanLine} ms`);console.log(`Drops 1 packet in 25: ${loadMs.dropsOneIn25} ms`);console.log(`A 200 ms tunnel: ${loadMs.tunnel} ms`);
  • Lines 1 and 2 are comments. Each time is the middle of three visits.
  • Line 3 holds the three recorded load times.
  • Lines 5 to 7 print Clean line: 661.8 ms, Drops 1 packet in 25: 1554.4 ms and A 200 ms tunnel: 1010.2 ms.

Losing 1 packet in 25 is 4%. Yet the page took more than twice as long. Here is Asha's screen at the same three moments, on each line:

Asha's screen at 300, 600, 1000 ms after she opened the Mint tea page, on three lines. Clean line: frames drawn at 297 ms, 584 ms, 584 ms; the page finished loading at 661.8 ms. Drops 1 packet in 25: frames drawn at 151 ms, 590 ms, 940 ms; the page finished loading at 1554.4 ms. A 200 ms tunnel: frames drawn at 161 ms, 161 ms, 957 ms; the page finished loading at 1010.2 ms.300 ms600 ms1,000 msLoaded atClean lineClean line: the frame Chrome drew at 297 ms.Clean line: the frame Chrome drew at 584 ms.Clean line: the frame Chrome drew at 584 ms.661.8 msDrops 1 in 25Drops 1 packet in 25: the frame Chrome drew at 151 ms.Drops 1 packet in 25: the frame Chrome drew at 590 ms.Drops 1 packet in 25: the frame Chrome drew at 940 ms.1554.4 ms200 ms tunnelA 200 ms tunnel: the frame Chrome drew at 161 ms.A 200 ms tunnel: the frame Chrome drew at 161 ms.A 200 ms tunnel: the frame Chrome drew at 957 ms.1010.2 ms
The same page and browser on three lines. Each picture is the last frame Chrome had drawn by that moment. Lost packets delayed the photo, then the teas below it.

On the clean line, the photo is there at 300 ms. On the line that drops packets, it comes later. In the tunnel, it's still empty at 600 ms. Why it matters: a few lost packets can cost a page far more than their share of its bytes.

Real-life analogyA delivery note that lists the boxes

Leaf and Kettle ships a big order in numbered boxes. Every box gets a new number, even one packed a second time. Your delivery note lists the numbers you got: "boxes 1 to 26, 28 to 43". The shop sees that 27 is missing. It packs that tea again, in box 59. Meanwhile, you unpack the teapot from box 30. It doesn't wait for the tea.

In real life: Each numbered box
In QUIC: A QUIC packet, with its packet number
In real life: The delivery note: "boxes 1 to 26, 28 to 43"
In QUIC: An ACK frame, with its ranges
In real life: Packing the tea again, in box 59
In QUIC: Sending the lost bytes again, in a new packet
In real life: The teapot that doesn't wait for the tea
In QUIC: Other streams, which don't wait for a lost packet
The main idea

QUIC numbers every packet once. The receiver lists what arrived. The sender declares a packet lost after 3 later ones arrive, or after a little more than a round trip, and sends its bytes again in a new packet. Only the stream that lost bytes waits for them. Every other stream carries on.

How the course recorded it:

  • The server was the example HTTP/3 server of quiche 0.30.1, Cloudflare's QUIC library, as in the two lessons before. It sent with CUBIC, its default, and paced its packets.
  • Every packet went through the course's relay, which imitates Asha's line. The relay dropped packets on purpose. On the patchy line, it dropped every 25th packet from the server that carried the page. In the tunnel, it dropped every packet both ways, from 250 to 450 ms.
  • Asha's visits were in Chrome 157.0.8103.52, on a phone-sized screen, over HTTP/3. Chrome's network log, the NetLog, noted every packet and ACK frame Chrome got and sent.
  • The server wrote a qlog, a log of every packet it sent and got. Its own log named each packet it declared lost.
  • Each line got three visits, taking turns, after one that wasn't kept. The lesson shows the middle one.

Streams and the numbers that name them

One connection carries the whole page: the HTML and eight photos. Each response travels in its own stream, as HTTP prioritization internals showed. A stream is a lane of bytes inside the connection. Its number is its stream ID.

The ID is more than a counter. Its two lowest bits say what kind of stream it is. RFC 9000, the QUIC standard, sets them:

  • The lowest bit says who opened it: 0 for the client, 1 for the server.
  • The next bit says which way it goes: 0 for both ways, 1 for one way only.

In code, that's an even or odd check, twice:

What a stream ID saysPop out in the code editor (opens in a new tab)JavaScript
// What a stream ID says about its stream (RFC 9000, section 2.1).function kindOf(id) {  const opener = id % 2 === 0 ? "the client" : "the server";  const way = Math.floor(id / 2) % 2 === 0 ? "both ways" : "one way";  return `Stream ${id}: opened by ${opener}, ${way}`;} console.log(kindOf(0));console.log(kindOf(4));console.log(kindOf(2));console.log(kindOf(3));
  • Line 3 reads the lowest bit: an even ID was opened by the client.
  • Line 4 halves the ID and checks again. That reads the second bit.
  • Line 5 puts the answer into words.
  • Lines 8 and 9 print Stream 0: opened by the client, both ways and Stream 4: opened by the client, both ways.
  • Lines 10 and 11 print Stream 2: opened by the client, one way and Stream 3: opened by the server, one way.

So the client's two-way streams are 0, 4, 8, 12 and so on. That's where HTTP/3 puts requests: RFC 9114 sends the first request on stream 0, the next on 4, then 8. Here are the streams of Asha's clean visit, from Chrome's NetLog and the server's qlog. Read each row across:

Every stream in one visit to the Mint tea page
StreamWho opened it?Which way?What did it carry?
0ChromeBoth waysThe request and response for page.html.
2ChromeOne wayControl messages, like its settings.
3The serverOne wayControl messages, like its settings.
4ChromeBoth waysThe request and response for mint-tea.jpg.
7The serverOne wayQPACK, HTTP/3's header compression: the encoder's notes.
8ChromeBoth waysThe request and response for black-tea.jpg.
11The serverOne wayQPACK: the decoder's notes.
12ChromeBoth waysThe request and response for chamomile.jpg.
15The serverOne wayNothing. Its type is a reserved one, sent to check that Chrome ignores types it doesn't know.
16ChromeBoth waysThe request and response for green-tea.jpg.
20ChromeBoth waysThe request and response for hibiscus.jpg.
24ChromeBoth waysThe request and response for jasmine.jpg.
28ChromeBoth waysThe request and response for oolong.jpg.
32ChromeBoth waysThe request and response for rooibos.jpg.

Three things stand out:

  • Each request took the next free number: the page on 0, the Mint tea photo on 4, rooibos.jpg on 32. RFC 9000 says an ID is never used twice in a connection.
  • Each side opened a one-way control stream for its settings: Chrome's 2, the server's 3. The server also opened two for QPACK, the way HTTP/3 compresses headers.
  • Stream 15 carried nothing. RFC 9114 keeps some stream types with no meaning, to check that the other side ignores types it doesn't know. quiche opens one on purpose.

Why it matters: in a NetLog or a qlog, the stream ID tells you whose bytes a packet carried. Here, 4 is the photo.

Flow control: the receiver sets the limits

A server can send bytes faster than a phone uses them. The phone keeps them somewhere until the page reads them. If the server could send without limit, that store could overflow. Flow control stops that: the receiver says how much the sender may send.

Real-life analogyShelf space in the stockroom

Leaf and Kettle's stockroom has a shelf for each tea, and the room itself has a size. The warehouse may only send what fits. When the shop sells some mint tea, it phones: "you can send mint up to box 40 now". If the warehouse has more mint but no room for it, it says so, and waits for the next call.

In real life: The shelf space for one tea
In QUIC: One stream's limit
In real life: The space in the whole stockroom
In QUIC: The connection's limit
In real life: "You can send mint up to box 40 now"
In QUIC: A MAX_STREAM_DATA frame with a higher limit
In real life: "I have more mint, but no shelf space"
In QUIC: A STREAM_DATA_BLOCKED frame

QUIC has those two levels. RFC 9000 calls them:

  • Stream flow control: a limit for each stream. The receiver raises it with a MAX_STREAM_DATA frame.
  • Connection flow control: a limit for all streams together. The receiver raises it with MAX_DATA.

The first limits travel in the handshake, as transport parameters. A sender that reaches a limit, with more to send, should say so: STREAM_DATA_BLOCKED for a stream, DATA_BLOCKED for the connection.

Chrome is generous. Its NetLog shows the limits it sent the server: 15,728,640 bytes for the connection, and 6,291,456 for each stream. That's 15 MB and 6 MB, as Chromium's source sets them. quiche's server offered Chrome 10,000,000 and 1,000,000. A page of photos never comes close.

So when does a limit slow a page down? RFC 9000 answers in technical words. In plain ones: the limit has to stay above what the line carries in one round trip. Otherwise the sender stops and waits each round trip.

Is the window big enough for Asha's line?Pop out in the code editor (opens in a new tab)JavaScript
// Asha's line: how many bytes are on their way during one round trip.const bytesPerMs = 10_000_000 / 8 / 1000; // 10 Mbit/sconst roundTripMs = 40;const onTheWay = bytesPerMs * roundTripMs; for (const window of [16384, 131072]) {  const verdict = window < onTheWay ? "holds the sender back" : "lets it run at full speed";  console.log(`A ${window}-byte window ${verdict}`);}
  • Line 2 turns 10 Mbit/s into bytes per ms: 1,250.
  • Line 3 is the round trip on Asha's line: 40 ms.
  • Line 4 multiplies them: 50,000 bytes are on their way during one round trip.
  • Line 7 compares each window with that.
  • It prints A 16384-byte window holds the sender back and A 131072-byte window lets it run at full speed.

The course tested it. quiche's example client downloaded the Mint tea photo, 102,809 bytes, with five receive windows. Each window was fixed, so it couldn't grow. Pick a window:

Playground: the photo through a small window
When the server sent the photo0 to 600 ms
Bursts of packets
11
STREAM_DATA_BLOCKED frames
9
First time the window was full
88.1 ms
First MAX_STREAM_DATA
25,527 bytes, at 137.7 ms
Last packet of the photo sent
499.3 ms
Try it yourself
With a 16 KB window, the photo took 547.3 ms

The window filled 9 times. Each time, the server sent STREAM_DATA_BLOCKED and waited for the client's MAX_STREAM_DATA, about one round trip later.

quiche's example client downloading the 102,809-byte Mint tea photo from quiche's example server on Asha's emulated line, with a receive window that can't grow. Each dark stretch is a burst of packets from the server. The time is the middle of three downloads.

With 16 KB, the photo took 547.3 ms. From 128 KB up, about 270 ms. With 16 KB, the window was full at 88.1 ms. The server sent STREAM_DATA_BLOCKED and stopped. At 137.7 ms, about one round trip later, MAX_STREAM_DATA raised the limit to 25,527 bytes.

Why not a whole window more? quiche's client asks for more once less than half of its window is left. The new limit is what it has read, plus one window. So even 64 KB filled up 2 times. Why it matters: a server or proxy with small windows slows every big download, on any line.

A limit on how many streams

The receiver also limits how many streams the other side may open (RFC 9000). It raises the limit with MAX_STREAMS. A client that wants more should send STREAMS_BLOCKED. RFC 9114 asks servers to allow at least 100 requests at a time. quiche and Chrome both allow 100.

The course gave Chrome a server that allowed 3. Chrome's NetLog shows 12 STREAMS_BLOCKED frames in the kept visit. As responses finished, the server sent MAX_STREAMS, and Chrome sent more requests. green-tea.jpg's request left at 359.7 ms instead of 93.6. The page loaded at 735.7 ms instead of 655.4. Why it matters: a stream limit lower than a page's requests makes them wait in the browser until earlier responses finish.

How QUIC finds a lost packet

Back to Asha's bus. In the visit the lesson keeps, the relay dropped 21 of the server's packets. The server found every one and sent its bytes again. How?

It starts with numbers. Every QUIC packet carries a packet number. Each new packet gets a higher one, and no number is used twice in a connection. RFC 9002, QUIC's loss recovery standard, puts it plainly: "Packet numbers never repeat".

The receiver answers with ACK frames. An ACK frame doesn't list every number. It lists ranges, like "9 to 26, 28 to 43". A number between two ranges didn't arrive. Here is one ACK frame Chrome sent, at 236.4 ms:

Reading an ACK frame's rangesPop out in the code editor (opens in a new tab)JavaScript
// One ACK frame Chrome sent: the packet numbers it got, as [first, last] ranges.const ranges = [[9, 26], [28, 43], [45, 52], [54, 54]]; const missing = [];for (let i = 1; i < ranges.length; i++) {  const lastBefore = ranges[i - 1][1];  for (let n = lastBefore + 1; n < ranges[i][0]; n++) missing.push(n);}console.log(`Missing: ${missing.join(", ")}`);
  • Line 2 holds the ranges Chrome acknowledged, smallest first.
  • Line 5 walks through the ranges, from the second one on.
  • Line 6 takes the last number of the range before.
  • Line 7 collects every number between that and the start of the next range.
  • Line 9 prints Missing: 27, 44, 53.

Three numbers are missing, for two different reasons:

  • 27 and 53 were lost: the relay dropped them.
  • 44 was never sent. quiche skips a number now and then, on purpose. RFC 9000 allows it. A receiver that acknowledges a number never sent is lying, perhaps to make the server send faster. quiche skipped 9 numbers in this visit.

So a gap means "not here yet". It doesn't mean lost. Packets can arrive out of order. How long should the sender wait before it decides?

Real-life analogyTaking the register

A teacher calls the register in order. Ben doesn't answer. She doesn't mark him absent at once: he may just be slow. Once three more pupils after him have answered, she marks him absent. She does the same if a little longer than a usual pause has passed. And if nobody answers at all, she asks "Is anyone there?"

In real life: Ben not answering
In QUIC: A packet with no ACK
In real life: Three pupils after him answering
In QUIC: The packet threshold: 3 later packets acknowledged
In real life: A little longer than a usual pause
In QUIC: The time threshold: 9/8 of a round trip
In real life: "Is anyone there?"
In QUIC: A probe, when the probe timer fires

RFC 9002 gives two rules. Say a packet has no ACK, but a later one does. It's lost when:

  • the packet threshold is met: a packet 3 or more numbers later was acknowledged;
  • the time threshold is met: it was sent at least 9/8 of a round trip ago.

The round trip here is the larger of two: the server's smoothed average, and its latest measurement. quiche starts with exactly these: 3 packets and 9/8, and it checks both at once. If a packet it called lost turns up after all, it was only late. Then quiche waits longer from then on: more packets, and 5/4 of a round trip. Here are the rules as code, with two packets from Asha's visit:

RFC 9002's two rules, on two real packetsPop out in the code editor (opens in a new tab)JavaScript
// RFC 9002's two rules for declaring a packet lost, with its numbers.function isLost(packet, largestAcked, now, rttMs) {  if (largestAcked >= packet.number + 3) return "lost: 3 later packets arrived";  if (now - packet.sentAt >= (9 / 8) * rttMs) return "lost: it waited 9/8 of a round trip";  return "not yet";} console.log(isLost({ number: 53, sentAt: 211.3 }, 56, 262.9, 50.2));console.log(isLost({ number: 154, sentAt: 400.2 }, 155, 450.4, 44.1));
  • Line 3 is the packet threshold. Line 4 is the time threshold.
  • Line 8 is packet 53. At 262.9 ms, the server's largest acknowledged number was 56: three later. It prints lost: 3 later packets arrived.
  • Line 9 is packet 154. Only 155 had been acknowledged, one later. But it had waited 50.2 ms, more than 9/8 of a 44.1 ms round trip. It prints lost: it waited 9/8 of a round trip.

Packet 53 had waited only 51.6 ms, short of 56.5 ms. The packet threshold caught it first. Of the 21 losses, the packet threshold caught 12, the time threshold 8, and 1 met both at once. Why it matters: the time threshold catches losses near the end of a burst, where 3 later packets never come.

Packets 51 to 56 that the server sent. Packet 53 was lost on the way. Chrome's ACK listed 45–52, 54–56: 53 was missing, and 3 later packets had arrived. So the server declared 53 lost and sent its 1,324 bytes of the photo again, in a new packet, number 83.The server sent515253545556lost on the wayChrome's ACKgot 45–52, 54–5653 is missing3 later packets got throughSent again as83the same 1,324 bytes of the photoA number is never used twice. A resend gets a new one.
Packet 53 never arrived. Once Chrome acknowledged three packets after it, the server declared it lost, and put its bytes in a new packet with a new number.

Packet 53's bytes went out again in packet 83. QUIC doesn't send a lost packet again. It sends its bytes again, in a new packet (RFC 9000). Each STREAM frame carries an offset, the position of its bytes in the stream, so Chrome knows where they go.

RFC 9002 explains why this matters. TCP reuses the same numbers for a resend. When an ACK comes back, TCP can't always tell which copy it's for. QUIC always can, so it measures round trips more accurately.

A resend can be lost too. Packet 103's bytes went out again in 128, and the relay dropped that one as well. They went out a third time, in 153. Pick any lost packet to see why it was declared lost, and when its bytes came again:

Playground: why each packet was declared lost
Packet 27mint-tea.jpg
Packet threshold
largest acknowledged 30, needs 30: met
Time threshold
waited 51 ms, needs 52.1 ms: not met
Its bytes went out again in
packet 59, at 215.3 ms
Chrome had them at
241.4 ms
Try it yourself
Packet 27: declared lost by the packet threshold

It carried 2 bytes of mint-tea.jpg. The server sent it at 164.3 ms and declared it lost at 215.3 ms, when an ACK frame arrived.

Every packet quiche's server declared lost in the kept visit on the line that drops 1 packet in 25, from its log and its qlog. Times are on the page's clock. Chrome's times come from its NetLog.

What the losses cost

The page took 1554.4 ms instead of 661.8. Sending 21 packets again doesn't take 892.6 ms. The bigger cost is the sender slowing down. A loss tells CUBIC the line may be full, as Congestion control showed. At the first loss, its congestion window fell from 37,800 bytes to 26,460: 70% of it. On the clean line, the window grew to 246,330 bytes. On the patchy line, it never went past 37,800. Why it matters: on a lossy line, a page pays twice, once for the resends and again for a slower sender.

When nothing comes back: the probe timeout

Both rules need an ACK. What if no ACK comes at all? On the third line, Asha's bus goes through a tunnel. From 250 to 450 ms, the relay dropped every packet, both ways.

For that, the sender keeps a probe timer, the PTO (probe timeout). RFC 9002 says: if nothing is acknowledged in time, send one or two probe packets, to get an ACK back. Here is how long the timer runs:

How long the probe timer runsPop out in the code editor (opens in a new tab)JavaScript
// When the server's probe timer (PTO) fires (RFC 9002, section 6.2.1).const smoothedRtt = 49.5; // ms, the server's estimate before the tunnelconst rttVariation = 5;const maxAckDelay = 25; // Chrome sent none, so the default let pto = smoothedRtt + Math.max(4 * rttVariation, 1) + maxAckDelay;console.log(`First probe after ${pto} ms`);pto = pto * 2;console.log(`Second probe after another ${pto} ms`);
  • Line 2 is the server's smoothed round trip before the tunnel.
  • Line 3 is how much its round trips varied.
  • Line 4 is how long Chrome may hold an ACK back. Chrome didn't send a value, so it's the default, 25 ms.
  • Line 6 adds them up, as RFC 9002 does, and quiche too. It allows four times the variation. Line 7 prints First probe after 94.5 ms.
  • Line 8 doubles it, for the next time. Line 9 prints Second probe after another 189 ms.

In the recording, the server sent packet 130 at 269.1 ms. No ACKs came back, so it stopped sending. The timer fired 95.6 ms later, a little late, as timers are. Step through what happened next:

Playground: the bus goes through a tunnel
Step 1 of 7
  1. 250 msThe tunnel starts
  2. 269.1 msThe server stops sending
  3. 364.7 msThe timer fires
  4. 450 msThe tunnel ends
  5. 554.9 msThe timer fires again
  6. 598.8 msThe ACK comes back
  7. 618.9 msThe photo is complete
Try it yourself
250 ms: The tunnel starts

For the next 200 ms, nothing gets through, either way. 41 packets from the server and 13 of Chrome's are lost.

The kept visit on the line with a 200 ms tunnel: the relay dropped every packet, both ways, from 250 to 450 ms after the visit started. The other times come from quiche's qlog and Chrome's Resource Timing, on the page's clock.

Three rules from RFC 9002 show here:

  • A PTO declares nothing lost. quiche's probes don't mark packets lost. The 41 losses were declared only when an ACK showed the gap.
  • The timer doubles each time it fires, so a dead line isn't flooded.
  • The first PTO sent one probe, the second two. quiche sends at most two.

The tunnel ended at 450 ms. But the server sent no new bytes until its next probe, at 554.9 ms. Why it matters: after an outage, the page waits for the sender's next probe, not just for the line. The photo's Largest Contentful Paint came at 612 ms, against 288 ms on the clean line.

One lost packet holds up one stream

HTTP/3 and QUIC showed the idea with a model. Over TCP, a lost packet holds up every file. Over QUIC, it holds up only its own. Here it is in Asha's real visit.

Packet 205 carried 1,324 bytes of chamomile.jpg. The relay dropped it. From 567.4 ms, chamomile.jpg's later bytes reached Chrome, but had to wait. A stream is handed over in order, so the page can't use bytes after a gap. The missing bytes arrived at 770.4 ms: 203 ms later.

When the bytes of five photos reached Chrome, from 530 to 810 ms. chamomile.jpg lost packet 205: from 567.4 ms its later bytes waited, until the lost ones came again at 770.4 ms. Meanwhile 100,202 bytes of other photos arrived, 90,628 of oolong.jpg, 3,972 of black-tea.jpg, 2,954 of hibiscus.jpg, 2,648 of green-tea.jpg.Bytes reaching Chromechamomile.jpgoolong.jpgblack-tea.jpghibiscus.jpggreen-tea.jpgwaiting for packet 205's bytes550600650700750800ms after Asha opened the page
The shaded stretch is 203 ms in which chamomile.jpg waited for one lost packet. The other photos kept arriving: 100,202 bytes, 90,628 of them oolong.jpg's.

Meanwhile, 100,202 bytes of other photos arrived. Nothing before them was missing, so Chrome could use them at once. Most of them, 90,628 bytes, were oolong.jpg's. RFC 9000 says why: "QUIC does not provide any means of ensuring ordering between bytes on different streams." Each stream waits only for its own gaps.

One TCP connection keeps one order for every byte. If it had carried the same bytes at the same moments, all 100,202 would have waited until 770.4 ms too. RFC 9114 says a lost packet under HTTP/2 "causes all active transactions to experience a stall". Here are the two side by side. Read each row across:

TCP and QUIC after a loss
QuestionTCPQUIC
What gets a number?Each byte. A resend carries the same numbers again.Each packet. A resend goes in a new packet, with a new number.
Can the sender tell a resend's ACK from the first one's?Not always. RFC 9002 calls this the retransmission ambiguity problem.Yes. Every packet number is used once.
How many gaps can one ACK report?Three SACK ranges.Many ACK ranges.
What waits for a lost packet?Every byte after it, from every file.Only the bytes after it in the same stream.
What happens when no ACK comes at all?A retransmission timeout, and the window collapses.A probe timeout. The probes are new packets, and the window stays.

Why did chamomile.jpg wait so long? The server declared packet 205 lost at 589.5 ms. But quiche puts lost bytes back into their stream, and the stream waits its turn in the scheduler from HTTP prioritization internals. The server spent that time on oolong.jpg. Why it matters: a resend is scheduled like any other bytes, so priorities decide which photo recovers first.

0-RTT: an early request, and a copy of it

TLS without the wait met 0-RTT: a browser that has connected before sends its request in its very first flight, before the handshake ends. Bytes sent this way are called early data. RFC 9001 brings it to QUIC, using what the browser kept from its last connection. Here is what it does in Chrome, and what can go wrong.

The course started quiche's server with early data on. It's off unless you ask, with --early-data. Chrome loaded the page, waited for the server to close the idle connection, then loaded it again on a new one:

A first visit and a return visit in 0-RTTPop out in the code editor (opens in a new tab)JavaScript
// The page's request and its first byte, in ms, on a new connection.// The middle of three: a first visit, then a return visit in 0-RTT.const firstVisit = { requestStart: 44.8, responseStart: 89.4 };const returnVisit = { requestStart: 1.6, responseStart: 44.6 }; const asked = firstVisit.requestStart - returnVisit.requestStart;const answered = firstVisit.responseStart - returnVisit.responseStart;console.log(`The request left ${asked.toFixed(1)} ms sooner`);console.log(`The first byte came ${answered.toFixed(1)} ms sooner`);
  • Lines 3 and 4 hold when Chrome sent the page's request, and when its first byte came back, on each visit.
  • Lines 6 and 7 subtract the return visit from the first one.
  • Lines 8 and 9 print The request left 43.2 ms sooner and The first byte came 44.8 ms sooner.

That's one round trip. Chrome's NetLog says why: its 0-RTT state went from NotAttempted on the first visit to AttemptedAndSucceeded on the second. The server's qlog shows 2 packets of the type 0-RTT, carrying streams 2 and 0: the control stream and the page's GET request.

What Chrome sends early

Chrome puts a request in 0-RTT only if it's safe to repeat: a GET, HEAD, OPTIONS or TRACE, or one marked as idempotent. Every other request waits for the handshake. If a server answers 425 Too Early, Chrome sends the request again, without early data. That's what RFC 8470 asks of browsers: they "MUST NOT send unsafe methods" in early data.

The copy

Real-life analogyA photocopied order slip

At a busy counter, you hand in your order slip before you show your loyalty card. Someone behind you photocopies the slip and hands in the copy. The kitchen makes a second tea. It never sees the copy's card: it starts on a slip as soon as one arrives.

In real life: Your order slip
In QUIC: The request in 0-RTT packets
In real life: Showing your loyalty card
In QUIC: Finishing the handshake
In real life: The photocopy of your slip
In QUIC: A copy of the 0-RTT packets, sent again
In real life: The second tea
In QUIC: The server running the request again

The course did exactly that. After the server had closed the second connection, the relay sent its first 4 packets from Chrome again, from a new address. The server's log shows GET /page.html a third time. The copy's handshake never finished: the server gave up with "Failed to establish connection". But the request had already run.

That's the risk RFC 9001 describes: 0-RTT in QUIC "is similarly vulnerable to replay attack". It adds: "Disabling 0-RTT entirely is the most effective defense against replay attack." RFC 9114 makes HTTP/3 servers follow RFC 8470, which gives a server three choices:

  • refuse early data altogether;
  • wait for the handshake to finish before acting on a request;
  • answer 425 Too Early to the risky ones.

A CDN that passes early data on adds an Early-Data: 1 header, so your origin server can decide. Why it matters: for a GET of a page, a copy costs only work. For anything that changes data, it could happen twice. So keep GET requests free of side effects.

Reading it yourself: qlog and qvis

QUIC is encrypted, so a packet capture shows very little. Instead, each side writes its own log. Every number in this lesson came from two of them.

Real-life analogyA flight recorder

An airliner carries a flight recorder. It notes every moment of the flight, each with its time. After a bumpy landing, investigators read it, and draw charts to see the whole flight at a glance.

In real life: The flight recorder
In QUIC: The server's qlog, or Chrome's NetLog
In real life: One entry, with its time
In QUIC: One event, like quic:packet_sent
In real life: The investigators' charts
In QUIC: qvis's views

qlog is the standard format for QUIC's logs. The IETF's qlog drafts define the file and the QUIC events in it. quiche's server writes one per connection when the QLOGDIR setting names a folder. Each event is one JSON record on its own line. Here are four of the server's records around packet 53, shortened. Their times count from when the server's connection began:

The server's qlog, from packet 53 to its resendJSON
{"time":188.3,"name":"quic:packet_sent","data":{"header":{"packet_type":"1RTT","packet_number":53},"frames":[{"frame_type":"stream","stream_id":4,"offset":52928,"raw":{"payload_length":1324}}]}}{"time":239.9,"name":"quic:packet_received","data":{"header":{"packet_type":"1RTT"},"frames":[{"frame_type":"ack","acked_ranges":[[9,26],[28,43],[45,52],[54,56]]}]}}{"time":239.9,"name":"quic:recovery_metrics_updated","data":{"cf_lost_packets":{"total":2,"delta":1}}}{"time":239.9,"name":"quic:packet_sent","data":{"header":{"packet_type":"1RTT","packet_number":83},"frames":[{"frame_type":"stream","stream_id":4,"offset":52928,"raw":{"payload_length":1324}}]}}
  • Line 1: the server sends packet 53. Its STREAM frame holds 1,324 bytes of stream 4, from offset 52,928.
  • Line 2: an ACK frame arrives. Its last two ranges skip 53.
  • Line 3: quiche counts a lost packet. cf_lost_packets is quiche's own field. The drafts also define a packet_lost event that names the rule. quiche 0.30.1 counts losses in this field instead.
  • Line 4: packet 83 carries the same stream, offset and length.

The whole file from the patchy visit is published with the lesson. The exercises use it.

Charts with qvis

qvis draws charts from these logs. It runs in your browser: its upload page says "No data is transfered to the server." It reads Chrome's NetLog too, if the file's name ends in .netlog. To try it:

  1. Record a NetLog with chrome://net-export, as How Chromium loads resources showed.
  2. Rename the file so it ends in .netlog.
  3. In qvis, open Manage files, upload it, and press Import.
  4. Open Sequence, and pick the client's trace.

Here is the NetLog of Asha's patchy visit in qvis, around the first packet the relay dropped:

A screenshot of qvis's sequence view, with Chrome's NetLog from the visit on the line that drops packets. Chrome is the line on the left, the server the line on the right. Each arrow is a packet with its number and what it carried. Packets 26 and 28 arrive from the server, but not 27. Chrome's ACK frames list the ranges it got, with 27 left out.qvis.quictools.infoChromethe serverqvis's sequence view: Chrome on the left, the server on the right, with packets and Chrome's ACK frames around packet 27.
Chrome's NetLog in qvis. Packet 27 never arrives, so every ACK Chrome sends after it leaves 27 out of its ranges.

Each arrow is a packet, with its number and its frames. Chrome's ACKs read "ack 9-26,28": 27 is missing. qvis's Multiplexing view draws every stream's bytes in the order they came. It marks a frame that arrived after a gap in purple, and the one that filled a gap in black. So the lost packets and their resends stand out at a glance.

One snag: qvis's code reads the 0.3 layout of qlog. quiche 0.30.1 writes the layout of the current drafts, and qvis stops with "File did not start with the proper qlog header". For quiche's side, the text is the tool: search it for a packet number. Why it matters: the qlog shows what the server decided, and the NetLog what Chrome saw. Together they say whether a slow page was the line, the server or the browser.

What a frontend team can do

Loss recovery happens in the server and the browser. But your pages and your questions to the server team still count:

  • Keep GET requests free of side effects. They can travel in 0-RTT, and a copy can run twice. Put actions behind POST.
  • Ask about 0-RTT. Does your server or CDN accept early data? For which requests? Does it answer 425 Too Early, or pass Early-Data: 1 on?
  • Check the limits. At least 100 streams, as RFC 9114 asks, and stream windows well above one round trip of data.
  • Mark what matters. On quiche, resent bytes wait their turn by priority. So fetchpriority="high" on the hero image helps it recover first.
  • Send fewer bytes. On a lossy line, every packet is a chance to lose one, and each loss slows the sender.
  • Record both sides. A NetLog from Chrome and a qlog from the server. Open the NetLog in qvis.

Each QUIC message comes from one side of the transfer. Sort them:

Who sends it?
  • A MAX_STREAM_DATA frame
  • A MAX_DATA frame
  • A STREAM_DATA_BLOCKED frame
  • An ACK frame with ranges
  • Declaring a packet lost
  • Sending a probe when the PTO fires
  • Skipping a packet number on purpose
Try it yourself
0 of 7 correct

Sort each frame or action by the side that sends or does it.

Choose a category for every card. You can change an answer at any time; Reset clears them all.

Common mix-ups

  • "A lost packet is sent again as it was." No. Its bytes go out again in a new packet, with a new number. Packet 53's bytes went out in packet 83.
  • "A gap in Chrome's ACK always means a lost packet." No. quiche skipped 9 numbers on purpose in one visit. They were never sent.
  • "QUIC declares a packet lost as soon as a later one arrives." No. It waits for 3 later packets, or 9/8 of a round trip, because packets can arrive out of order.
  • "When the probe timer fires, the packets are lost." No. A PTO only sends probes. The losses are declared when an ACK shows the gap.
  • "QUIC has no head-of-line blocking." No. Inside one stream, bytes after a gap still wait. chamomile.jpg waited 203 ms.
  • "0-RTT is safe because it's encrypted." No. A copy of the encrypted packets made the server run the request again.
  • "Flow control and congestion control are the same." No. Flow control protects the receiver's memory. Congestion control protects the line.

Practice exercises

Answer first, then use the hints.

Exercise 1 · Warm-upA stream the server opens

What does the reader print for stream 9?

Starter codePop out in the code editor (opens in a new tab)JavaScript
// What a stream ID says about its stream (RFC 9000, section 2.1).
function kindOf(id) {
  const opener = id % 2 === 0 ? "the client" : "the server";
  const way = Math.floor(id / 2) % 2 === 0 ? "both ways" : "one way";
  return `Stream ${id}: opened by ${opener}, ${way}`;
}

console.log(kindOf(9));

Answer, then press Check. Spacing and letter case don’t matter.

    Exercise 2 · Warm-upA later ACK frame

    Chrome sent this ACK frame later in the same visit. What does the code print?

    Starter codePop out in the code editor (opens in a new tab)JavaScript
    // One ACK frame Chrome sent: the packet numbers it got, as [first, last] ranges.
    const ranges = [[131, 153], [155, 178], [181, 187]];
    
    const missing = [];
    for (let i = 1; i < ranges.length; i++) {
      const lastBefore = ranges[i - 1][1];
      for (let n = lastBefore + 1; n < ranges[i][0]; n++) missing.push(n);
    }
    console.log(`Missing: ${missing.join(", ")}`);

    Answer, then press Check. Spacing and letter case don’t matter.

      Exercise 3 · PracticeLost, or not yet?

      Packet 60 was sent at 100 ms. At 146 ms, the largest acknowledged packet is 62. The round trip is 40 ms. What does it print?

      Starter codePop out in the code editor (opens in a new tab)JavaScript
      // RFC 9002's two rules for declaring a packet lost, with its numbers.
      function isLost(packet, largestAcked, now, rttMs) {
        if (largestAcked >= packet.number + 3) return "lost: 3 later packets arrived";
        if (now - packet.sentAt >= (9 / 8) * rttMs) return "lost: it waited 9/8 of a round trip";
        return "not yet";
      }
      
      console.log(isLost({ number: 60, sentAt: 100 }, 62, 146, 40));

      Answer, then press Check. Spacing and letter case don’t matter.

        Exercise 4 · PracticeWhere chamomile.jpg's bytes went

        In the lost-packet playground, pick packet 205. In which packet did its bytes go out again?

        Answer, then press Check. Spacing and letter case don’t matter.

          Exercise 5 · ChallengeRead the real qlog

          Open the server's qlog from the patchy visit. How many bytes of the Mint tea photo did packet 27 carry?

          Answer, then press Check. Spacing and letter case don’t matter.

            Exercise 6 · ChallengeA loss detector that gives up too soon

            This detector calls packet 11 lost, though it arrived. Fix line 4 the RFC 9002 way. What does it print then?

            Starter codePop out in the code editor (opens in a new tab)JavaScript
            // A loss detector. Packets 10 to 13 left in order,
            // but 11 took a slower path and arrived after 12.
            function isLost(packet, largestAcked) {
              return largestAcked > packet.number;
            }
            
            console.log(isLost({ number: 11 }, 12));
            console.log(isLost({ number: 11 }, 13));

            Answer, then press Check. Spacing and letter case don’t matter.

              Check your understanding

              For each question, ask who knows what: the side that sent the bytes, or the side that got them.

              The QUIC internals quiz · 8 questionsScore: first tries count
              1. Question 1 of 8What does the stream ID reader print for stream 6?

                Read the code, then predictPop out in the code editor (opens in a new tab)JavaScript
                // What a stream ID says about its stream (RFC 9000, section 2.1).
                function kindOf(id) {
                  const opener = id % 2 === 0 ? "the client" : "the server";
                  const way = Math.floor(id / 2) % 2 === 0 ? "both ways" : "one way";
                  return `Stream ${id}: opened by ${opener}, ${way}`;
                }
                
                console.log(kindOf(6));

                Choose an answer to see the explanation.

              2. Question 2 of 8Packet 53 is missing. Chrome has acknowledged 54 and 55, and not much time has passed. Is 53 lost yet?

                Choose an answer to see the explanation.

              3. Question 3 of 8In the tunnel, the server's probe timer fired. What did it do with the packets still unacknowledged?

                Choose an answer to see the explanation.

              4. Question 4 of 8The server sends a lost packet's bytes again. What number does the new packet get?

                Choose an answer to see the explanation.

              5. Question 5 of 8While chamomile.jpg waited for its lost bytes, oolong.jpg's bytes kept arriving. What did Chrome do with them?

                Choose an answer to see the explanation.

              6. Question 6 of 8Why did a 16 KB receive window slow the photo down on Asha's line?

                Choose an answer to see the explanation.

              7. Question 7 of 8Which requests does Chrome 157 send in 0-RTT, before the handshake is confirmed?

                Choose an answer to see the explanation.

              8. Question 8 of 8The relay copied Chrome's first 0-RTT packets and sent them again from a new address. What did quiche's example server do?

                Choose an answer to see the explanation.

              Key takeaways

              • A stream ID's two lowest bits say who opened the stream and which way it goes. Requests use 0, 4, 8 and so on.
              • The receiver sets limits per stream and per connection. A window smaller than a round trip of data slows a download.
              • Packet numbers never repeat. ACK frames list ranges. Lost bytes go out again in a new packet.
              • A packet is lost after 3 later ones are acknowledged, or after 9/8 of a round trip. With no ACKs at all, the PTO probes, and doubles.
              • A loss holds up only its own stream. But each loss also makes CUBIC slow the whole connection.
              • 0-RTT saves a round trip, and a copy of it can run twice. Keep GET requests free of side effects.
              • The server's qlog and Chrome's NetLog show both sides. qvis draws them.

              Remember the one-liner.
              QUIC numbers every packet once, lists what arrived, and resends only the missing bytes, so each stream waits only for its own.

              Sources

              The facts on this page come from these standards, docs and source files:

              Coming next: How performance standards are made. You'll see how the APIs this course measures with are designed and shipped, and how to take part.

              CompleteFrontend Clear concepts. Working examples.