QUIC internals
How QUIC carries a page: stream IDs, flow control, and how lost packets are found and sent again. Then 0-RTT's replay risk, and reading it all in a qlog.
- 01Read streams and flow controlTell a stream's kind from its ID, and say what MAX_STREAM_DATA, MAX_DATA and the BLOCKED frames do.
- 02Explain loss recoverySay why QUIC declared a packet lost: 3 later packets, 9/8 of a round trip, or nothing back at all.
- 03Judge 0-RTTSay what Chrome sends before the handshake ends, and why a copy of it can run twice.
- 04Debug with logsFind a lost packet and its resend in a server's qlog, Chrome's NetLog and qvis.
Asha's bus ride: lost packets, a slower page
Asha opens the Mint tea page on the bus. Her phone's signal comes and goes. Some packets from the server never reach her. Still, the page arrives whole, with every photo. Nothing is missing. It just takes much longer.
So how does the server know a packet went missing? When does it send it again? And while one photo waits, do the others wait too? This lesson answers all three, from inside QUIC, the transport under HTTP/3.
The course loaded the page on Asha's home line from Congestion control: 10 Mbit/s, 20 ms each way. It tried three versions of that line:
- a clean line, where nothing is lost;
- a line that drops 1 packet in 25 from the server;
- a line with a 200 ms tunnel, where nothing gets through either way.
// When the Mint tea page finished loading on Asha's phone, in ms.// The middle of three visits on each line.const loadMs = { cleanLine: 661.8, dropsOneIn25: 1554.4, tunnel: 1010.2 }; console.log(`Clean line: ${loadMs.cleanLine} ms`);console.log(`Drops 1 packet in 25: ${loadMs.dropsOneIn25} ms`);console.log(`A 200 ms tunnel: ${loadMs.tunnel} ms`);- Lines 1 and 2 are comments. Each time is the middle of three visits.
- Line 3 holds the three recorded load times.
- Lines 5 to 7 print
Clean line: 661.8 ms,Drops 1 packet in 25: 1554.4 msandA 200 ms tunnel: 1010.2 ms.
Losing 1 packet in 25 is 4%. Yet the page took more than twice as long. Here is Asha's screen at the same three moments, on each line:
On the clean line, the photo is there at 300 ms. On the line that drops packets, it comes later. In the tunnel, it's still empty at 600 ms. Why it matters: a few lost packets can cost a page far more than their share of its bytes.
Leaf and Kettle ships a big order in numbered boxes. Every box gets a new number, even one packed a second time. Your delivery note lists the numbers you got: "boxes 1 to 26, 28 to 43". The shop sees that 27 is missing. It packs that tea again, in box 59. Meanwhile, you unpack the teapot from box 30. It doesn't wait for the tea.
- In real life: Each numbered box
- In QUIC: A QUIC packet, with its packet number
- In real life: The delivery note: "boxes 1 to 26, 28 to 43"
- In QUIC: An ACK frame, with its ranges
- In real life: Packing the tea again, in box 59
- In QUIC: Sending the lost bytes again, in a new packet
- In real life: The teapot that doesn't wait for the tea
- In QUIC: Other streams, which don't wait for a lost packet
QUIC numbers every packet once. The receiver lists what arrived. The sender declares a packet lost after 3 later ones arrive, or after a little more than a round trip, and sends its bytes again in a new packet. Only the stream that lost bytes waits for them. Every other stream carries on.
How the course recorded it:
- The server was the example HTTP/3 server of quiche 0.30.1, Cloudflare's QUIC library, as in the two lessons before. It sent with CUBIC, its default, and paced its packets.
- Every packet went through the course's relay, which imitates Asha's line. The relay dropped packets on purpose. On the patchy line, it dropped every 25th packet from the server that carried the page. In the tunnel, it dropped every packet both ways, from 250 to 450 ms.
- Asha's visits were in Chrome 157.0.8103.52, on a phone-sized screen, over HTTP/3. Chrome's network log, the NetLog, noted every packet and ACK frame Chrome got and sent.
- The server wrote a qlog, a log of every packet it sent and got. Its own log named each packet it declared lost.
- Each line got three visits, taking turns, after one that wasn't kept. The lesson shows the middle one.
Streams and the numbers that name them
One connection carries the whole page: the HTML and eight photos. Each response travels in its own stream, as HTTP prioritization internals showed. A stream is a lane of bytes inside the connection. Its number is its stream ID.
The ID is more than a counter. Its two lowest bits say what kind of stream it is. RFC 9000, the QUIC standard, sets them:
- The lowest bit says who opened it: 0 for the client, 1 for the server.
- The next bit says which way it goes: 0 for both ways, 1 for one way only.
In code, that's an even or odd check, twice:
// What a stream ID says about its stream (RFC 9000, section 2.1).function kindOf(id) { const opener = id % 2 === 0 ? "the client" : "the server"; const way = Math.floor(id / 2) % 2 === 0 ? "both ways" : "one way"; return `Stream ${id}: opened by ${opener}, ${way}`;} console.log(kindOf(0));console.log(kindOf(4));console.log(kindOf(2));console.log(kindOf(3));- Line 3 reads the lowest bit: an even ID was opened by the client.
- Line 4 halves the ID and checks again. That reads the second bit.
- Line 5 puts the answer into words.
- Lines 8 and 9 print
Stream 0: opened by the client, both waysandStream 4: opened by the client, both ways. - Lines 10 and 11 print
Stream 2: opened by the client, one wayandStream 3: opened by the server, one way.
So the client's two-way streams are 0, 4, 8, 12 and so on. That's where HTTP/3 puts requests: RFC 9114 sends the first request on stream 0, the next on 4, then 8. Here are the streams of Asha's clean visit, from Chrome's NetLog and the server's qlog. Read each row across:
| Stream | Who opened it? | Which way? | What did it carry? |
|---|---|---|---|
| 0 | Chrome | Both ways | The request and response for page.html. |
| 2 | Chrome | One way | Control messages, like its settings. |
| 3 | The server | One way | Control messages, like its settings. |
| 4 | Chrome | Both ways | The request and response for mint-tea.jpg. |
| 7 | The server | One way | QPACK, HTTP/3's header compression: the encoder's notes. |
| 8 | Chrome | Both ways | The request and response for black-tea.jpg. |
| 11 | The server | One way | QPACK: the decoder's notes. |
| 12 | Chrome | Both ways | The request and response for chamomile.jpg. |
| 15 | The server | One way | Nothing. Its type is a reserved one, sent to check that Chrome ignores types it doesn't know. |
| 16 | Chrome | Both ways | The request and response for green-tea.jpg. |
| 20 | Chrome | Both ways | The request and response for hibiscus.jpg. |
| 24 | Chrome | Both ways | The request and response for jasmine.jpg. |
| 28 | Chrome | Both ways | The request and response for oolong.jpg. |
| 32 | Chrome | Both ways | The request and response for rooibos.jpg. |
Three things stand out:
- Each request took the next free number: the page on 0, the Mint tea photo on 4, rooibos.jpg on 32. RFC 9000 says an ID is never used twice in a connection.
- Each side opened a one-way control stream for its settings: Chrome's 2, the server's 3. The server also opened two for QPACK, the way HTTP/3 compresses headers.
- Stream 15 carried nothing. RFC 9114 keeps some stream types with no meaning, to check that the other side ignores types it doesn't know. quiche opens one on purpose.
Why it matters: in a NetLog or a qlog, the stream ID tells you whose bytes a packet carried. Here, 4 is the photo.
Flow control: the receiver sets the limits
A server can send bytes faster than a phone uses them. The phone keeps them somewhere until the page reads them. If the server could send without limit, that store could overflow. Flow control stops that: the receiver says how much the sender may send.
Leaf and Kettle's stockroom has a shelf for each tea, and the room itself has a size. The warehouse may only send what fits. When the shop sells some mint tea, it phones: "you can send mint up to box 40 now". If the warehouse has more mint but no room for it, it says so, and waits for the next call.
- In real life: The shelf space for one tea
- In QUIC: One stream's limit
- In real life: The space in the whole stockroom
- In QUIC: The connection's limit
- In real life: "You can send mint up to box 40 now"
- In QUIC: A MAX_STREAM_DATA frame with a higher limit
- In real life: "I have more mint, but no shelf space"
- In QUIC: A STREAM_DATA_BLOCKED frame
QUIC has those two levels. RFC 9000 calls them:
- Stream flow control: a limit for each stream. The receiver raises it with a MAX_STREAM_DATA frame.
- Connection flow control: a limit for all streams together. The receiver raises it with MAX_DATA.
The first limits travel in the handshake, as transport parameters. A sender that reaches a limit, with more to send, should say so: STREAM_DATA_BLOCKED for a stream, DATA_BLOCKED for the connection.
Chrome is generous. Its NetLog shows the limits it sent the server: 15,728,640 bytes for the connection, and 6,291,456 for each stream. That's 15 MB and 6 MB, as Chromium's source sets them. quiche's server offered Chrome 10,000,000 and 1,000,000. A page of photos never comes close.
So when does a limit slow a page down? RFC 9000 answers in technical words. In plain ones: the limit has to stay above what the line carries in one round trip. Otherwise the sender stops and waits each round trip.
// Asha's line: how many bytes are on their way during one round trip.const bytesPerMs = 10_000_000 / 8 / 1000; // 10 Mbit/sconst roundTripMs = 40;const onTheWay = bytesPerMs * roundTripMs; for (const window of [16384, 131072]) { const verdict = window < onTheWay ? "holds the sender back" : "lets it run at full speed"; console.log(`A ${window}-byte window ${verdict}`);}- Line 2 turns 10 Mbit/s into bytes per ms: 1,250.
- Line 3 is the round trip on Asha's line: 40 ms.
- Line 4 multiplies them: 50,000 bytes are on their way during one round trip.
- Line 7 compares each window with that.
- It prints
A 16384-byte window holds the sender backandA 131072-byte window lets it run at full speed.
The course tested it. quiche's example client downloaded the Mint tea photo, 102,809 bytes, with five receive windows. Each window was fixed, so it couldn't grow. Pick a window:
- Bursts of packets
- 11
- STREAM_DATA_BLOCKED frames
- 9
- First time the window was full
- 88.1 ms
- First MAX_STREAM_DATA
- 25,527 bytes, at 137.7 ms
- Last packet of the photo sent
- 499.3 ms
The window filled 9 times. Each time, the server sent STREAM_DATA_BLOCKED and waited for the client's MAX_STREAM_DATA, about one round trip later.
With 16 KB, the photo took 547.3 ms. From 128 KB up, about 270 ms. With 16 KB, the window was full at 88.1 ms. The server sent STREAM_DATA_BLOCKED and stopped. At 137.7 ms, about one round trip later, MAX_STREAM_DATA raised the limit to 25,527 bytes.
Why not a whole window more? quiche's client asks for more once less than half of its window is left. The new limit is what it has read, plus one window. So even 64 KB filled up 2 times. Why it matters: a server or proxy with small windows slows every big download, on any line.
A limit on how many streams
The receiver also limits how many streams the other side may open (RFC 9000). It raises the limit with MAX_STREAMS. A client that wants more should send STREAMS_BLOCKED. RFC 9114 asks servers to allow at least 100 requests at a time. quiche and Chrome both allow 100.
The course gave Chrome a server that allowed 3. Chrome's NetLog shows 12 STREAMS_BLOCKED frames in the kept visit. As responses finished, the server sent MAX_STREAMS, and Chrome sent more requests. green-tea.jpg's request left at 359.7 ms instead of 93.6. The page loaded at 735.7 ms instead of 655.4. Why it matters: a stream limit lower than a page's requests makes them wait in the browser until earlier responses finish.
How QUIC finds a lost packet
Back to Asha's bus. In the visit the lesson keeps, the relay dropped 21 of the server's packets. The server found every one and sent its bytes again. How?
It starts with numbers. Every QUIC packet carries a packet number. Each new packet gets a higher one, and no number is used twice in a connection. RFC 9002, QUIC's loss recovery standard, puts it plainly: "Packet numbers never repeat".
The receiver answers with ACK frames. An ACK frame doesn't list every number. It lists ranges, like "9 to 26, 28 to 43". A number between two ranges didn't arrive. Here is one ACK frame Chrome sent, at 236.4 ms:
// One ACK frame Chrome sent: the packet numbers it got, as [first, last] ranges.const ranges = [[9, 26], [28, 43], [45, 52], [54, 54]]; const missing = [];for (let i = 1; i < ranges.length; i++) { const lastBefore = ranges[i - 1][1]; for (let n = lastBefore + 1; n < ranges[i][0]; n++) missing.push(n);}console.log(`Missing: ${missing.join(", ")}`);- Line 2 holds the ranges Chrome acknowledged, smallest first.
- Line 5 walks through the ranges, from the second one on.
- Line 6 takes the last number of the range before.
- Line 7 collects every number between that and the start of the next range.
- Line 9 prints
Missing: 27, 44, 53.
Three numbers are missing, for two different reasons:
- 27 and 53 were lost: the relay dropped them.
- 44 was never sent. quiche skips a number now and then, on purpose. RFC 9000 allows it. A receiver that acknowledges a number never sent is lying, perhaps to make the server send faster. quiche skipped 9 numbers in this visit.
So a gap means "not here yet". It doesn't mean lost. Packets can arrive out of order. How long should the sender wait before it decides?
A teacher calls the register in order. Ben doesn't answer. She doesn't mark him absent at once: he may just be slow. Once three more pupils after him have answered, she marks him absent. She does the same if a little longer than a usual pause has passed. And if nobody answers at all, she asks "Is anyone there?"
- In real life: Ben not answering
- In QUIC: A packet with no ACK
- In real life: Three pupils after him answering
- In QUIC: The packet threshold: 3 later packets acknowledged
- In real life: A little longer than a usual pause
- In QUIC: The time threshold: 9/8 of a round trip
- In real life: "Is anyone there?"
- In QUIC: A probe, when the probe timer fires
RFC 9002 gives two rules. Say a packet has no ACK, but a later one does. It's lost when:
- the packet threshold is met: a packet 3 or more numbers later was acknowledged;
- the time threshold is met: it was sent at least 9/8 of a round trip ago.
The round trip here is the larger of two: the server's smoothed average, and its latest measurement. quiche starts with exactly these: 3 packets and 9/8, and it checks both at once. If a packet it called lost turns up after all, it was only late. Then quiche waits longer from then on: more packets, and 5/4 of a round trip. Here are the rules as code, with two packets from Asha's visit:
// RFC 9002's two rules for declaring a packet lost, with its numbers.function isLost(packet, largestAcked, now, rttMs) { if (largestAcked >= packet.number + 3) return "lost: 3 later packets arrived"; if (now - packet.sentAt >= (9 / 8) * rttMs) return "lost: it waited 9/8 of a round trip"; return "not yet";} console.log(isLost({ number: 53, sentAt: 211.3 }, 56, 262.9, 50.2));console.log(isLost({ number: 154, sentAt: 400.2 }, 155, 450.4, 44.1));- Line 3 is the packet threshold. Line 4 is the time threshold.
- Line 8 is packet 53. At 262.9 ms, the server's largest acknowledged number was 56: three later. It prints
lost: 3 later packets arrived. - Line 9 is packet 154. Only 155 had been acknowledged, one later. But it had waited 50.2 ms, more than 9/8 of a 44.1 ms round trip. It prints
lost: it waited 9/8 of a round trip.
Packet 53 had waited only 51.6 ms, short of 56.5 ms. The packet threshold caught it first. Of the 21 losses, the packet threshold caught 12, the time threshold 8, and 1 met both at once. Why it matters: the time threshold catches losses near the end of a burst, where 3 later packets never come.
Packet 53's bytes went out again in packet 83. QUIC doesn't send a lost packet again. It sends its bytes again, in a new packet (RFC 9000). Each STREAM frame carries an offset, the position of its bytes in the stream, so Chrome knows where they go.
RFC 9002 explains why this matters. TCP reuses the same numbers for a resend. When an ACK comes back, TCP can't always tell which copy it's for. QUIC always can, so it measures round trips more accurately.
A resend can be lost too. Packet 103's bytes went out again in 128, and the relay dropped that one as well. They went out a third time, in 153. Pick any lost packet to see why it was declared lost, and when its bytes came again:
- Packet threshold
- largest acknowledged 30, needs 30: met
- Time threshold
- waited 51 ms, needs 52.1 ms: not met
- Its bytes went out again in
- packet 59, at 215.3 ms
- Chrome had them at
- 241.4 ms
It carried 2 bytes of mint-tea.jpg. The server sent it at 164.3 ms and declared it lost at 215.3 ms, when an ACK frame arrived.
What the losses cost
The page took 1554.4 ms instead of 661.8. Sending 21 packets again doesn't take 892.6 ms. The bigger cost is the sender slowing down. A loss tells CUBIC the line may be full, as Congestion control showed. At the first loss, its congestion window fell from 37,800 bytes to 26,460: 70% of it. On the clean line, the window grew to 246,330 bytes. On the patchy line, it never went past 37,800. Why it matters: on a lossy line, a page pays twice, once for the resends and again for a slower sender.
When nothing comes back: the probe timeout
Both rules need an ACK. What if no ACK comes at all? On the third line, Asha's bus goes through a tunnel. From 250 to 450 ms, the relay dropped every packet, both ways.
For that, the sender keeps a probe timer, the PTO (probe timeout). RFC 9002 says: if nothing is acknowledged in time, send one or two probe packets, to get an ACK back. Here is how long the timer runs:
// When the server's probe timer (PTO) fires (RFC 9002, section 6.2.1).const smoothedRtt = 49.5; // ms, the server's estimate before the tunnelconst rttVariation = 5;const maxAckDelay = 25; // Chrome sent none, so the default let pto = smoothedRtt + Math.max(4 * rttVariation, 1) + maxAckDelay;console.log(`First probe after ${pto} ms`);pto = pto * 2;console.log(`Second probe after another ${pto} ms`);- Line 2 is the server's smoothed round trip before the tunnel.
- Line 3 is how much its round trips varied.
- Line 4 is how long Chrome may hold an ACK back. Chrome didn't send a value, so it's the default, 25 ms.
- Line 6 adds them up, as RFC 9002 does, and quiche too. It allows four times the variation. Line 7 prints
First probe after 94.5 ms. - Line 8 doubles it, for the next time. Line 9 prints
Second probe after another 189 ms.
In the recording, the server sent packet 130 at 269.1 ms. No ACKs came back, so it stopped sending. The timer fired 95.6 ms later, a little late, as timers are. Step through what happened next:
- 250 msThe tunnel starts
- 269.1 msThe server stops sending
- 364.7 msThe timer fires
- 450 msThe tunnel ends
- 554.9 msThe timer fires again
- 598.8 msThe ACK comes back
- 618.9 msThe photo is complete
For the next 200 ms, nothing gets through, either way. 41 packets from the server and 13 of Chrome's are lost.
Three rules from RFC 9002 show here:
- A PTO declares nothing lost. quiche's probes don't mark packets lost. The 41 losses were declared only when an ACK showed the gap.
- The timer doubles each time it fires, so a dead line isn't flooded.
- The first PTO sent one probe, the second two. quiche sends at most two.
The tunnel ended at 450 ms. But the server sent no new bytes until its next probe, at 554.9 ms. Why it matters: after an outage, the page waits for the sender's next probe, not just for the line. The photo's Largest Contentful Paint came at 612 ms, against 288 ms on the clean line.
One lost packet holds up one stream
HTTP/3 and QUIC showed the idea with a model. Over TCP, a lost packet holds up every file. Over QUIC, it holds up only its own. Here it is in Asha's real visit.
Packet 205 carried 1,324 bytes of chamomile.jpg. The relay dropped it. From 567.4 ms, chamomile.jpg's later bytes reached Chrome, but had to wait. A stream is handed over in order, so the page can't use bytes after a gap. The missing bytes arrived at 770.4 ms: 203 ms later.
Meanwhile, 100,202 bytes of other photos arrived. Nothing before them was missing, so Chrome could use them at once. Most of them, 90,628 bytes, were oolong.jpg's. RFC 9000 says why: "QUIC does not provide any means of ensuring ordering between bytes on different streams." Each stream waits only for its own gaps.
One TCP connection keeps one order for every byte. If it had carried the same bytes at the same moments, all 100,202 would have waited until 770.4 ms too. RFC 9114 says a lost packet under HTTP/2 "causes all active transactions to experience a stall". Here are the two side by side. Read each row across:
| Question | TCP | QUIC |
|---|---|---|
| What gets a number? | Each byte. A resend carries the same numbers again. | Each packet. A resend goes in a new packet, with a new number. |
| Can the sender tell a resend's ACK from the first one's? | Not always. RFC 9002 calls this the retransmission ambiguity problem. | Yes. Every packet number is used once. |
| How many gaps can one ACK report? | Three SACK ranges. | Many ACK ranges. |
| What waits for a lost packet? | Every byte after it, from every file. | Only the bytes after it in the same stream. |
| What happens when no ACK comes at all? | A retransmission timeout, and the window collapses. | A probe timeout. The probes are new packets, and the window stays. |
Why did chamomile.jpg wait so long? The server declared packet 205 lost at 589.5 ms. But quiche puts lost bytes back into their stream, and the stream waits its turn in the scheduler from HTTP prioritization internals. The server spent that time on oolong.jpg. Why it matters: a resend is scheduled like any other bytes, so priorities decide which photo recovers first.
0-RTT: an early request, and a copy of it
TLS without the wait met 0-RTT: a browser that has connected before sends its request in its very first flight, before the handshake ends. Bytes sent this way are called early data. RFC 9001 brings it to QUIC, using what the browser kept from its last connection. Here is what it does in Chrome, and what can go wrong.
The course started quiche's server with early data on. It's off unless you ask, with --early-data. Chrome loaded the page, waited for the server to close the idle connection, then loaded it again on a new one:
// The page's request and its first byte, in ms, on a new connection.// The middle of three: a first visit, then a return visit in 0-RTT.const firstVisit = { requestStart: 44.8, responseStart: 89.4 };const returnVisit = { requestStart: 1.6, responseStart: 44.6 }; const asked = firstVisit.requestStart - returnVisit.requestStart;const answered = firstVisit.responseStart - returnVisit.responseStart;console.log(`The request left ${asked.toFixed(1)} ms sooner`);console.log(`The first byte came ${answered.toFixed(1)} ms sooner`);- Lines 3 and 4 hold when Chrome sent the page's request, and when its first byte came back, on each visit.
- Lines 6 and 7 subtract the return visit from the first one.
- Lines 8 and 9 print
The request left 43.2 ms soonerandThe first byte came 44.8 ms sooner.
That's one round trip. Chrome's NetLog says why: its 0-RTT state went from NotAttempted on the first visit to AttemptedAndSucceeded on the second. The server's qlog shows 2 packets of the type 0-RTT, carrying streams 2 and 0: the control stream and the page's GET request.
What Chrome sends early
Chrome puts a request in 0-RTT only if it's safe to repeat: a GET, HEAD, OPTIONS or TRACE, or one marked as idempotent. Every other request waits for the handshake. If a server answers 425 Too Early, Chrome sends the request again, without early data. That's what RFC 8470 asks of browsers: they "MUST NOT send unsafe methods" in early data.
The copy
At a busy counter, you hand in your order slip before you show your loyalty card. Someone behind you photocopies the slip and hands in the copy. The kitchen makes a second tea. It never sees the copy's card: it starts on a slip as soon as one arrives.
- In real life: Your order slip
- In QUIC: The request in 0-RTT packets
- In real life: Showing your loyalty card
- In QUIC: Finishing the handshake
- In real life: The photocopy of your slip
- In QUIC: A copy of the 0-RTT packets, sent again
- In real life: The second tea
- In QUIC: The server running the request again
The course did exactly that. After the server had closed the second connection, the relay sent its first 4 packets from Chrome again, from a new address. The server's log shows GET /page.html a third time. The copy's handshake never finished: the server gave up with "Failed to establish connection". But the request had already run.
That's the risk RFC 9001 describes: 0-RTT in QUIC "is similarly vulnerable to replay attack". It adds: "Disabling 0-RTT entirely is the most effective defense against replay attack." RFC 9114 makes HTTP/3 servers follow RFC 8470, which gives a server three choices:
- refuse early data altogether;
- wait for the handshake to finish before acting on a request;
- answer 425 Too Early to the risky ones.
A CDN that passes early data on adds an Early-Data: 1 header, so your origin server can decide. Why it matters: for a GET of a page, a copy costs only work. For anything that changes data, it could happen twice. So keep GET requests free of side effects.
Reading it yourself: qlog and qvis
QUIC is encrypted, so a packet capture shows very little. Instead, each side writes its own log. Every number in this lesson came from two of them.
An airliner carries a flight recorder. It notes every moment of the flight, each with its time. After a bumpy landing, investigators read it, and draw charts to see the whole flight at a glance.
- In real life: The flight recorder
- In QUIC: The server's qlog, or Chrome's NetLog
- In real life: One entry, with its time
- In QUIC: One event, like quic:packet_sent
- In real life: The investigators' charts
- In QUIC: qvis's views
qlog is the standard format for QUIC's logs. The IETF's qlog drafts define the file and the QUIC events in it. quiche's server writes one per connection when the QLOGDIR setting names a folder. Each event is one JSON record on its own line. Here are four of the server's records around packet 53, shortened. Their times count from when the server's connection began:
{"time":188.3,"name":"quic:packet_sent","data":{"header":{"packet_type":"1RTT","packet_number":53},"frames":[{"frame_type":"stream","stream_id":4,"offset":52928,"raw":{"payload_length":1324}}]}}{"time":239.9,"name":"quic:packet_received","data":{"header":{"packet_type":"1RTT"},"frames":[{"frame_type":"ack","acked_ranges":[[9,26],[28,43],[45,52],[54,56]]}]}}{"time":239.9,"name":"quic:recovery_metrics_updated","data":{"cf_lost_packets":{"total":2,"delta":1}}}{"time":239.9,"name":"quic:packet_sent","data":{"header":{"packet_type":"1RTT","packet_number":83},"frames":[{"frame_type":"stream","stream_id":4,"offset":52928,"raw":{"payload_length":1324}}]}}- Line 1: the server sends packet 53. Its STREAM frame holds 1,324 bytes of stream 4, from offset 52,928.
- Line 2: an ACK frame arrives. Its last two ranges skip 53.
- Line 3: quiche counts a lost packet.
cf_lost_packetsis quiche's own field. The drafts also define apacket_lostevent that names the rule. quiche 0.30.1 counts losses in this field instead. - Line 4: packet 83 carries the same stream, offset and length.
The whole file from the patchy visit is published with the lesson. The exercises use it.
Charts with qvis
qvis draws charts from these logs. It runs in your browser: its upload page says "No data is transfered to the server." It reads Chrome's NetLog too, if the file's name ends in .netlog. To try it:
- Record a NetLog with
chrome://net-export, as How Chromium loads resources showed. - Rename the file so it ends in .netlog.
- In qvis, open Manage files, upload it, and press Import.
- Open Sequence, and pick the client's trace.
Here is the NetLog of Asha's patchy visit in qvis, around the first packet the relay dropped:
Each arrow is a packet, with its number and its frames. Chrome's ACKs read "ack 9-26,28": 27 is missing. qvis's Multiplexing view draws every stream's bytes in the order they came. It marks a frame that arrived after a gap in purple, and the one that filled a gap in black. So the lost packets and their resends stand out at a glance.
One snag: qvis's code reads the 0.3 layout of qlog. quiche 0.30.1 writes the layout of the current drafts, and qvis stops with "File did not start with the proper qlog header". For quiche's side, the text is the tool: search it for a packet number. Why it matters: the qlog shows what the server decided, and the NetLog what Chrome saw. Together they say whether a slow page was the line, the server or the browser.
What a frontend team can do
Loss recovery happens in the server and the browser. But your pages and your questions to the server team still count:
- Keep GET requests free of side effects. They can travel in 0-RTT, and a copy can run twice. Put actions behind POST.
- Ask about 0-RTT. Does your server or CDN accept early data? For which requests? Does it answer 425 Too Early, or pass
Early-Data: 1on? - Check the limits. At least 100 streams, as RFC 9114 asks, and stream windows well above one round trip of data.
- Mark what matters. On quiche, resent bytes wait their turn by priority. So
fetchpriority="high"on the hero image helps it recover first. - Send fewer bytes. On a lossy line, every packet is a chance to lose one, and each loss slows the sender.
- Record both sides. A NetLog from Chrome and a qlog from the server. Open the NetLog in qvis.
Each QUIC message comes from one side of the transfer. Sort them:
- A MAX_STREAM_DATA frame
- A MAX_DATA frame
- A STREAM_DATA_BLOCKED frame
- An ACK frame with ranges
- Declaring a packet lost
- Sending a probe when the PTO fires
- Skipping a packet number on purpose
Sort each frame or action by the side that sends or does it.
Common mix-ups
- "A lost packet is sent again as it was." No. Its bytes go out again in a new packet, with a new number. Packet 53's bytes went out in packet 83.
- "A gap in Chrome's ACK always means a lost packet." No. quiche skipped 9 numbers on purpose in one visit. They were never sent.
- "QUIC declares a packet lost as soon as a later one arrives." No. It waits for 3 later packets, or 9/8 of a round trip, because packets can arrive out of order.
- "When the probe timer fires, the packets are lost." No. A PTO only sends probes. The losses are declared when an ACK shows the gap.
- "QUIC has no head-of-line blocking." No. Inside one stream, bytes after a gap still wait. chamomile.jpg waited 203 ms.
- "0-RTT is safe because it's encrypted." No. A copy of the encrypted packets made the server run the request again.
- "Flow control and congestion control are the same." No. Flow control protects the receiver's memory. Congestion control protects the line.
Practice exercises
Answer first, then use the hints.
What does the reader print for stream 9?
// What a stream ID says about its stream (RFC 9000, section 2.1).
function kindOf(id) {
const opener = id % 2 === 0 ? "the client" : "the server";
const way = Math.floor(id / 2) % 2 === 0 ? "both ways" : "one way";
return `Stream ${id}: opened by ${opener}, ${way}`;
}
console.log(kindOf(9));Stream 9: opened by the server, both ways. 9 is odd, so the server opened it. Half of 9, rounded down, is 4, which is even, so it goes both ways. HTTP/3 never uses this kind: RFC 9114 says a client must treat one as an error.
Chrome sent this ACK frame later in the same visit. What does the code print?
// One ACK frame Chrome sent: the packet numbers it got, as [first, last] ranges.
const ranges = [[131, 153], [155, 178], [181, 187]];
const missing = [];
for (let i = 1; i < ranges.length; i++) {
const lastBefore = ranges[i - 1][1];
for (let n = lastBefore + 1; n < ranges[i][0]; n++) missing.push(n);
}
console.log(`Missing: ${missing.join(", ")}`);Missing: 154, 179, 180. In the recording, 154 and 179 were lost. 180 was never sent: quiche skipped it. The ACK can't tell them apart, only the sender can.
Packet 60 was sent at 100 ms. At 146 ms, the largest acknowledged packet is 62. The round trip is 40 ms. What does it print?
// RFC 9002's two rules for declaring a packet lost, with its numbers.
function isLost(packet, largestAcked, now, rttMs) {
if (largestAcked >= packet.number + 3) return "lost: 3 later packets arrived";
if (now - packet.sentAt >= (9 / 8) * rttMs) return "lost: it waited 9/8 of a round trip";
return "not yet";
}
console.log(isLost({ number: 60, sentAt: 100 }, 62, 146, 40));lost: it waited 9/8 of a round trip. 62 is only two later, so the packet threshold isn't met. But 46 ms have passed, and 9/8 of 40 ms is 45 ms.
In the lost-packet playground, pick packet 205. In which packet did its bytes go out again?
Packet 302. Packet 205 was declared lost by the packet threshold, but its bytes waited their turn in chamomile.jpg's stream, so they went out much later.
Open the server's qlog from the patchy visit. How many bytes of the Mint tea photo did packet 27 carry?
2. Packet 27 was tiny: 28 bytes in all, with a STREAM frame of just 2 bytes of the photo. The relay dropped every 25th packet whatever its size, so this small one was the first loss.
This detector calls packet 11 lost, though it arrived. Fix line 4 the RFC 9002 way. What does it print then?
// A loss detector. Packets 10 to 13 left in order,
// but 11 took a slower path and arrived after 12.
function isLost(packet, largestAcked) {
return largestAcked > packet.number;
}
console.log(isLost({ number: 11 }, 12));
console.log(isLost({ number: 11 }, 13));// A loss detector. Packets 10 to 13 left in order,
// but 11 took a slower path and arrived after 12.
function isLost(packet, largestAcked) {
return largestAcked >= packet.number + 3;
}
console.log(isLost({ number: 11 }, 12));
console.log(isLost({ number: 11 }, 13));false twice. With >, a single later packet marks 11 lost, though it was only late. With >= packet.number + 3, 12 and 13 aren't enough. RFC 9002 says not to use a packet threshold below 3.
Check your understanding
For each question, ask who knows what: the side that sent the bytes, or the side that got them.
Question 1 of 8What does the stream ID reader print for stream 6?
Read the code, then predict// What a stream ID says about its stream (RFC 9000, section 2.1). function kindOf(id) { const opener = id % 2 === 0 ? "the client" : "the server"; const way = Math.floor(id / 2) % 2 === 0 ? "both ways" : "one way"; return `Stream ${id}: opened by ${opener}, ${way}`; } console.log(kindOf(6));Choose an answer to see the explanation.
Question 2 of 8Packet 53 is missing. Chrome has acknowledged 54 and 55, and not much time has passed. Is 53 lost yet?
Choose an answer to see the explanation.
Question 3 of 8In the tunnel, the server's probe timer fired. What did it do with the packets still unacknowledged?
Choose an answer to see the explanation.
Question 4 of 8The server sends a lost packet's bytes again. What number does the new packet get?
Choose an answer to see the explanation.
Question 5 of 8While chamomile.jpg waited for its lost bytes, oolong.jpg's bytes kept arriving. What did Chrome do with them?
Choose an answer to see the explanation.
Question 6 of 8Why did a 16 KB receive window slow the photo down on Asha's line?
Choose an answer to see the explanation.
Question 7 of 8Which requests does Chrome 157 send in 0-RTT, before the handshake is confirmed?
Choose an answer to see the explanation.
Question 8 of 8The relay copied Chrome's first 0-RTT packets and sent them again from a new address. What did quiche's example server do?
Choose an answer to see the explanation.
Key takeaways
- A stream ID's two lowest bits say who opened the stream and which way it goes. Requests use 0, 4, 8 and so on.
- The receiver sets limits per stream and per connection. A window smaller than a round trip of data slows a download.
- Packet numbers never repeat. ACK frames list ranges. Lost bytes go out again in a new packet.
- A packet is lost after 3 later ones are acknowledged, or after 9/8 of a round trip. With no ACKs at all, the PTO probes, and doubles.
- A loss holds up only its own stream. But each loss also makes CUBIC slow the whole connection.
- 0-RTT saves a round trip, and a copy of it can run twice. Keep GET requests free of side effects.
- The server's qlog and Chrome's NetLog show both sides. qvis draws them.
Remember the one-liner.
QUIC numbers every packet once, lists what arrived, and resends only the missing bytes, so each stream waits only for its own.
Sources
The facts on this page come from these standards, docs and source files:
- RFC 9000: QUIC, sections 2 and 2.1, Streams and Stream Types, IETF.
- RFC 9000, sections 4.1 to 4.6, Flow Control, IETF.
- RFC 9000, section 12.3, Packet Numbers, IETF.
- RFC 9000, sections 13.3 and 19.3, Retransmission and ACK Frames, IETF.
- RFC 9000, section 21.4, Optimistic ACK Attack, IETF.
- RFC 9002: QUIC Loss Detection and Congestion Control, sections 3, 4 and 6, IETF.
- RFC 9001: Using TLS to Secure QUIC, sections 4.6 and 9.2, IETF.
- RFC 9114: HTTP/3, sections 6.1, 6.2.3 and 10.9, IETF.
- RFC 8470: Using Early Data in HTTP, IETF.
- qlog: Structured Logging for Network Protocols (main schema draft), IETF QUIC working group.
- QUIC event definitions for qlog (draft), IETF QUIC working group.
- qvis, the hosted QUIC and HTTP/3 visualizer, Robin Marx and contributors.
- qvis's source at 3db6acf239, GitHub, quiclog/qvis.
- Providing network details for bug reports (net-export), The Chromium Projects.
- Chromium 157.0.8103.52: net/quic and net/http, Chromium source.
- quiche 0.30.1, Cloudflare.
Coming next: How performance standards are made. You'll see how the APIs this course measures with are designed and shipped, and how to take part.