TCP connections and slow start

A connection before the first byte

DNS lookups ended with the browser holding a number: 199.36.158.100, the address behind completefrontend.com. So now it can ask for the page, right?

Not yet. Here are two times the course recorded on its first visit to completefrontend.com:

1// The course's first visit to completefrontend.com, in ms from the start.2const visit = { domainLookupEnd: 26, requestStart: 83.9 };3 4console.log(`The browser knew the address at ${visit.domainLookupEnd} ms.`);5console.log(`It asked for the page at ${visit.requestStart} ms.`);

• Line 1 is a comment. Each time is in milliseconds (ms), counted from the moment the visit started. A millisecond is a thousandth of a second.
• Line 2 holds two recorded times. The names come from the browser's own stopwatch, which The navigation timeline reads in full.
• Line 4 prints The browser knew the address at 26 ms.
• Line 5 prints It asked for the page at 83.9 ms.

For almost 58 ms, the browser had the address and still didn't ask. What was it waiting for? It was opening a connection to the server. On this site, that takes two steps. The first is TCP, and it's what this lesson is about. The second is TLS, the step that makes the connection private. That's the next lesson.

Here is that visit's connection, drawn to scale. Time goes down:

A trip across and back is a round trip. The hello took one. The page took about three more, because it came back in batches.

TCP, the Transmission Control Protocol, is the set of rules that carries the page from the server to the browser. Why does a page need rules to travel? Because the network only moves small chunks of data, called packets. And it moves them carelessly. It can lose a packet, deliver one twice, or deliver them out of order.

TCP fixes all of that. It cuts the page into numbered pieces and checks that each one arrives. It sends again whatever goes missing. And it hands the pieces to the browser in the right order. A long letter sent by post needs the same care:

TCP has names for these. A segment is one numbered piece of the page. It holds about 1,460 bytes, and it travels inside a packet. An acknowledgment, or ACK, is the short note the browser sends back to say which segments arrived.

All this care keeps the page whole. But it costs time, in three ways:

• The handshake. Before any data moves, the browser and the server say hello. That takes one round trip.
• Slow start. A new connection sends only a little at first, then more each round trip.
• Loss. A lost packet slows the connection down, and holds up everything behind it.

A round trip is the time it takes a message to reach the server and an answer to come back. Latency vs bandwidth showed that round trips, not megabits, decide how fast most pages load. That's why the picture at the top counts them. The next two sections take it apart: first the hello, then the batches.

The three-way handshake

Before a single byte of the page can travel, the browser and the server greet each other. It works like a phone call:

TCP opens a connection with three messages, so it's called the three-way handshake:

• SYN, short for synchronize. The browser asks the server to open a connection.
• SYN-ACK. The server agrees, and acknowledges the browser's SYN.
• ACK. The browser acknowledges the server's answer.

Here's the part that saves time. The browser doesn't wait after its ACK. It sends the request right behind it. So three messages cost only one round trip of waiting: the SYN going out, and the SYN-ACK coming back. In the picture at the top, that's the first round trip: hello? is the SYN, and hello! is the SYN-ACK.

On an HTTPS site like this one, TLS's first message takes the request's place behind the ACK. The request follows once TLS is done.

How long is that round trip? Back to the course's first visit:

1// The same visit: when TCP started, and when TLS took over.2const visit = { connectStart: 26, secureConnectionStart: 45.6 };3 4const handshakeMs = visit.secureConnectionStart - visit.connectStart;5console.log(`The TCP handshake took ${handshakeMs.toFixed(1)} ms.`);

• Line 2 holds two recorded times. connectStart is when the browser started connecting. secureConnectionStart is when TCP was done and TLS began.
• Line 4 subtracts the first from the second.
• Line 5 prints The TCP handshake took 19.6 ms.

That's one round trip to completefrontend.com's server. The rest of the connection, 38.2 ms, went on TLS. So that answers the question from the start: of the almost 58 ms, TCP took 19.6 and TLS took 38.2.

A round trip gets longer with distance. Ilya Grigorik works out an example in his book, High Performance Browser Networking. Between New York and London, light in an optical fibre needs at least 56 ms to go there and back. So a handshake between those two cities takes at least 56 ms, however fast the line is.

Why it matters: no amount of bandwidth makes the handshake shorter. The only way to skip it is to not need a new connection. The section on reuse comes back to this.

One TCP feature trims it. TCP Fast Open lets a repeat connection carry data in its very first message. It has limits, though: on how much data, and on which requests.

Congestion window and slow start

The hello is done, and the request has left. Now the server sends the page. Say the page is 50 KB, and your line is fast. Does the whole page arrive in the next round trip?

No. The server doesn't know how fast your line is. It doesn't know how busy the network in between is, either. If it sends too much at once, the routers on the way, the machines that pass packets along, drop what they can't hold. So it starts small, and learns as it goes, like a new waiter:

In TCP, the plates on one trip are the congestion window. It's how much the server may send before it stops and waits for ACKs. Congestion means a crowded network, like a crowded road.

The window grows as ACKs come back. For each segment acknowledged, it grows by one segment. In one round trip, every segment sent gets its ACK. So each round trip, the window doubles. This first phase is called slow start. The name is odd, because the window grows fast. It's the start that's small.

Where does it start? Since 2013, a new connection may start with 10 segments, at most 14,600 bytes. That's about 14 KB. The number comes from RFC 6928. RFCs are the numbered documents the internet's rules are written in. The rule before it allowed 2 to 4 segments. This first size is called the initial window. Here's the doubling, one round trip at a time:

1// A new connection may send 10 segments before it waits.2let windowSize = 10;3 4for (let trip = 1; trip <= 4; trip++) {5  console.log(`Round trip ${trip}: ${windowSize} segments, ${windowSize * 1460} bytes`);6  windowSize = windowSize * 2;7}

• Line 1 is a comment. Line 2 starts the window at 10 segments.
• Line 4 repeats for four round trips.
• Line 5 prints the window, in segments and in bytes. Each segment holds 1,460 bytes.
• Line 6 doubles the window for the next round trip.
• It prints Round trip 1: 10 segments, 14600 bytes and Round trip 2: 20 segments, 29200 bytes. Then Round trip 3: 40 segments, 58400 bytes and Round trip 4: 80 segments, 116800 bytes.

So how many round trips does the 50 KB page take? Keep sending windows until the whole file is out:

1const fileBytes = 50000;2let windowSize = 10;3let sent = 0;4let trips = 0;5while (sent < fileBytes) {6  sent += windowSize * 1460;7  windowSize *= 2;8  trips++;9}10console.log(`A ${fileBytes / 1000} KB file takes ${trips} round trips.`);

• Line 1 is the file: 50,000 bytes, or 50 KB.
• Line 2 starts the window at 10 segments. Lines 3 and 4 count the bytes sent and the round trips.
• Line 5 repeats until the whole file is sent.
• Line 6 sends one window. Line 7 doubles it. Line 8 counts one more round trip.
• Line 10 prints A 50 KB file takes 3 round trips.

Two round trips carry 14.6 + 29.2 = 43.8 KB. That's not quite 50, so a third is needed. Add the round trip for the hello, and the last byte arrives four round trips after the browser started connecting. That's before TLS adds its own.

Why it matters: a faster line doesn't change that count. On a new connection, the window decides how much may go in each round trip, not the line.

The window doesn't double forever. It stops doubling when one of these happens:

• It reaches the browser's limit. The browser tells the server how much it can take at once. That's the receive window. The server never sends more than it, however big the congestion window gets.
• It reaches a threshold. TCP can set a size where slow start ends. After it, the window grows by only one segment per round trip.
• A packet is lost. The section on loss shows what happens then.

Does a bigger start really help? Google tested it on its search pages, and RFC 6928 reports the results. On average, search results arrived 11.7% sooner in one data center, and 8.7% sooner in another. On paths with long round trips, the gain was over 20%. Saving a round trip matters most where round trips are long.

Try it yourself. Pick a file size and a starting window. The third button loses a packet in round trip 3. The section on loss explains what happens then.

1let window = initialSegments;2let slowStart = true;3while (delivered < needed) {4  const lost = round === lossAt;5  send(window);6  delivered += lost ? window - 1 : window;7  if (lost) slowStart = false;8  window = lost ? window / 2 : slowStart ? window * 2 : window + 1;9}

In the panel's code:

• Line 6 counts what got through. A lost segment doesn't count, because it has to be sent again.
• Line 8 sets the next window. It doubles in slow start. After a loss, it halves, then grows by one segment per round trip.

The 14 KB rule and its limits

The first round trip of data carries about 14 KB. That gives web developers a rough guide, often called the 14 KB rule. Put what the browser needs to start showing the page into the first 14 KB. Then it can start building the page after one round trip of data, instead of waiting for more.

It's the new waiter's first trip again. Bring the drinks and the bread first, so the table can start while the main course is on its way.

What does the page need first? The top of the HTML, and the CSS for the first screen, often written inside the HTML. Here's a quick check for one page:

1const firstRoundTrip = 10 * 1460;2const headers = 600;3const html = 9000;4const inlineCss = 4000;5const start = headers + html + inlineCss;6 7console.log(`The first round trip carries ${firstRoundTrip} bytes.`);8console.log(`This page's start is ${start} bytes, so ${start <= firstRoundTrip ? "it fits" : "it needs another round trip"}.`);

• Line 1 is the first round trip: 10 segments of 1,460 bytes.
• Lines 2 to 4 are the parts of this page's start: the response's headers, the HTML, and the CSS written inside it.
• Line 5 adds them up.
• Line 7 prints The first round trip carries 14600 bytes.
• Line 8 prints This page's start is 13600 bytes, so it fits.

Why it matters: when the start fits, the browser has everything the first screen needs after one round trip of data. When it doesn't, part of it comes a round trip later.

Don't count to the byte, though. It's a rough guide, not a law. Read this table across: each row is something that changes the sum, and how it changes it.

So keep the start of the HTML small and compressed. Leave out anything the first screen doesn't need. Later stages of the course show how.

Packet loss and retransmission

So far, every packet arrived. On a real network, some don't. A router with a full queue drops what it can't hold. A crowded café's Wi-Fi loses some too.

How does TCP notice? The ACKs tell it. Say the browser's ACKs keep saying "I have everything up to segment 2", while later segments arrive. Then segment 3 is missing. And if no ACK comes back for too long, something was lost too.

Then TCP does two things. It sends the missing segment again. That's a retransmission. And it slows down, because it reads the loss as a sign that the network is too full. Think of a garden hose:

That's how classic TCP recovers. Grigorik's book calls it AIMD, for multiplicative decrease and additive increase. In plain words: cut the window in half after a loss, then add one segment at a time. Here it is, with a loss in round trip 3:

1const lostIn = 3;2let windowSize = 10;3const sizes = [];4for (let trip = 1; trip <= 5; trip++) {5  sizes.push(windowSize);6  if (trip < lostIn) windowSize *= 2;7  else if (trip === lostIn) windowSize /= 2;8  else windowSize += 1;9}10console.log(`A loss in round trip ${lostIn}: ${sizes.join(", ")} segments.`);

• Line 1 says when the loss happens: in round trip 3.
• Line 2 starts the window at 10 segments. Line 3 is a list for the window of each round trip.
• Line 4 repeats for five round trips. Line 5 writes down this round trip's window.
• Line 6 doubles the window before the loss.
• Line 7 halves it after the round trip that had the loss.
• Line 8 adds one segment in every round trip after it.
• Line 10 prints A loss in round trip 3: 10, 20, 40, 20, 21 segments.

Without the loss, round trip 5 would carry 160 segments. With it, only 21. So one lost packet early in a page load costs many round trips later.

Newer methods get back up to speed faster. One of them, CUBIC, is the default in Linux, Windows and Apple's systems, RFC 9438 says. But a loss still costs time.

One loss holds up everything behind it

A loss has a second cost. Remember Ravi reading the letter in order. TCP hands data to the browser in order too. So when segment 3 is lost, segments 4, 5 and 6 can arrive, but the browser doesn't get them yet. They wait in TCP's buffer, a waiting area in memory, until a new copy of segment 3 arrives.

This is called head-of-line blocking. It's like the queue at the tea shop's counter. When the person at the front can't find their purse, nobody behind them gets served.

Why it matters: HTTP/2 sends many files over one TCP connection. HTTP/2 is a version of HTTP, the language browsers and servers speak, and this site's first visit used it. But TCP doesn't know about files. It only sees bytes in order. So one lost packet stalls every file on the connection, even the ones it has nothing to do with. HTTP/3 solves this, and HTTP/3 and QUIC shows how.

Keeping connections alive and reused

Every new connection pays twice. It spends a round trip saying hello, and it starts with a small window that has to grow. A connection that stays open pays neither again.

HTTP/1.1 keeps a connection open after a response, unless told otherwise. This is called keep-alive, or a persistent connection. The next request skips the handshake, and finds a window that has already grown. Here's what the hello alone costs six requests, with each round trip at 100 ms:

1const roundTripMs = 100;2const requests = 6;3 4const newEachTime = requests * 2 * roundTripMs;5const keptAlive = 2 * roundTripMs + (requests - 1) * roundTripMs;6 7console.log(`A new connection for each request: ${newEachTime} ms.`);8console.log(`One connection, kept alive: ${keptAlive} ms.`);9console.log(`Keep-alive saved ${newEachTime - keptAlive} ms.`);

• Line 1 sets one round trip to 100 ms. Line 2 is the number of requests.
• Line 4 gives each request its own connection. Each one takes two round trips: one for the hello, one for the request and its answer.
• Line 5 keeps one connection open. The first request takes two round trips, and each of the other five takes one.
• Lines 7 and 8 print A new connection for each request: 1200 ms. and One connection, kept alive: 700 ms.
• Line 9 prints Keep-alive saved 500 ms.

Grigorik puts the saving at (N − 1) × RTT for N requests. RTT is short for round trip time. In words: one round trip for every request after the first. And this sum leaves out slow start, which a kept connection also skips.

How many connections does a browser open? On HTTP/1.1, a connection carries one request at a time. So browsers open up to six connections to each host. Each of them pays its own handshake and its own slow start. HTTP/2 and HTTP/3 send all of a host's files over one connection. HTTP/1.1 and HTTP/2 compares them.

An open connection can also cool down. If it sits idle for a while, TCP may shrink its window back to the start. It's like a waiter back from a long break, who starts again with a few plates. This is called slow-start restart. Grigorik recommends turning it off on servers, so that connections kept open stay fast.

Which of this page's own files reused a connection? Press Look:

1const [page] = performance.getEntriesByType("navigation");2console.log(page.nextHopProtocol, page.transferSize, "bytes");3 4for (const file of performance.getEntriesByType("resource")) {5  const connect = file.connectEnd - file.connectStart;6  const kind = file.connectStart === 0 ? "hidden" : connect > 0 ? "new connection" : "reused";7  count(kind, file.nextHopProtocol);8}

In the panel's code:

• Line 2 reads the page's protocol, and how many bytes its HTML took on the network.
• Line 5 works out how long each file spent connecting.
• Line 6 sorts the files. No time connecting means the file reused a connection. A zero connectStart means another site hid its timings. A site shares them only if it sends a Timing-Allow-Origin header.

What is it costing?

Each delay below comes mostly from one of TCP's three costs. For each one, ask: is the page waiting to say hello, waiting for the window to grow, or waiting for a lost packet?

Common mix-ups

• "Bandwidth decides how fast a big file starts." No. A new connection sends about 14 KB in its first round trip, however fast the line is. The window grows from there.
• "The 14 KB rule means pages must be under 14 KB." No. It's about what the first round trip carries. The rest of the page just takes more round trips.
• "The handshake takes three round trips." No. It's three messages, but one round trip. The request leaves right behind the last one.
• "Packet loss only delays the lost packet." No. It halves the window. And everything behind the lost packet waits for it.
• "More connections are always faster." No. Each new one pays its own handshake and slow start, and they all share the same line.
• "A connection's speed is set when it opens." No. It grows in slow start, shrinks after a loss, and can start over after sitting idle.

Practice exercises

Answer first, then use the hints.

const fileBytes = 50000;
let windowSize = 4;
let sent = 0;
let trips = 0;
while (sent < fileBytes) {
  sent += windowSize * 1460;
  windowSize *= 2;
  trips++;
}
console.log(`A ${fileBytes / 1000} KB file takes ${trips} round trips.`);

const roundTripMs = 100;
const requests = 10;

const newEachTime = requests * 2 * roundTripMs;
const keptAlive = 2 * roundTripMs + (requests - 1) * roundTripMs;

console.log(`A new connection for each request: ${newEachTime} ms.`);
console.log(`One connection, kept alive: ${keptAlive} ms.`);
console.log(`Keep-alive saved ${newEachTime - keptAlive} ms.`);

const firstRoundTrip = 10 * 1460;
const headers = 600;
const html = 12000;
const inlineCss = 4000;
const start = headers + html + inlineCss;

console.log(`The first round trip carries ${firstRoundTrip} bytes.`);
console.log(`This page's start is ${start} bytes, so ${start <= firstRoundTrip ? "it fits" : "it needs another round trip"}.`);

const lostIn = 2;
let windowSize = 10;
const sizes = [];
for (let trip = 1; trip <= 5; trip++) {
  sizes.push(windowSize);
  if (trip < lostIn) windowSize *= 2;
  else if (trip === lostIn) windowSize /= 2;
  else windowSize += 1;
}
console.log(`A loss in round trip ${lostIn}: ${sizes.join(", ")} segments.`);

Check your understanding

For each question, ask which cost it's about: the hello, the growing window, or a lost packet.

Key takeaways

• A new TCP connection spends one round trip saying hello before the request can leave.
• Slow start begins at 10 segments, about 14 KB, and doubles every round trip.
• A faster line doesn't remove those round trips. The window decides how much goes each time.
• Put what the first screen needs into the first 14 KB, counted compressed and with headers.
• A lost packet halves the window, and holds up everything behind it.
• Keep-alive saves a round trip for every request after the first. Idle connections can cool down.

Sources

The facts on this page come from these standards, books and docs:

• Navigation Timing Level 2, W3C.
• Building Blocks of TCP, High Performance Browser Networking, Ilya Grigorik.
• RFC 9293: Transmission Control Protocol (TCP), IETF.
• RFC 5681: TCP Congestion Control, IETF.
• RFC 6928: Increasing TCP's Initial Window, IETF.
• RFC 9438: CUBIC for Fast and Long-Distance Networks, IETF.
• HTTP/1.X, High Performance Browser Networking, Ilya Grigorik.
• PerformanceResourceTiming: nextHopProtocol, MDN.
• RequestTimingView.ts: the Network panel's Timing tab, Chrome DevTools source.

Coming next: TLS without the wait. Remember the other 38.2 ms of the first visit's connection? That was TLS, making the line private. TLS 1.3, resumption and good certificates make most of its round trips disappear.