Streams & progress

Streams: read a body piece by piece

When you call fetch(), the server sends back a response. The response has a body: the data itself, such as JSON, text, or a file.

await response.json() waits until the whole body has arrived. Then it turns the body into a JavaScript value. That works well for a small JSON object.

A stream is another way to read the body. Your code gets the body in small pieces, called chunks, as they arrive. With a stream, you can:

• show a progress bar while a big file downloads,
• show text on the page as soon as it arrives,
• work through a large file without keeping all of it in memory at once.

In this lesson, a chunk is usually a Uint8Array. A Uint8Array is a list of bytes: whole numbers from 0 to 255. You can count the bytes to show progress, or turn them into text.

response.body is the body as a stream. Its type is ReadableStream: a stream that your code reads from. Each chunk in it is a Uint8Array of bytes.

Some responses have no body. Then response.body is null. Two examples:

• a 204 No Content response, which means "done, nothing to send back",
• a response to a HEAD request, which asks for the headers only.

The table compares the two ways to read a body. Each row asks one question about both ways.

getReader() gives you a reader: the object that reads the chunks. It also locks the stream. Locked means only this one reader can read it. When you call reader.releaseLock(), the lock goes away, and another reader can read what is left.

Do you need two readers from the start? You have two choices:

• call response.clone() before you read, to get a second copy of the response,
• call stream.tee() to split the stream into two. You'll see it near the end of the lesson.

Each reader.read() returns a promise. The promise gives an object with two fields:

• done is false while there are chunks, and true at the end.
• value is the next chunk of bytes, or undefined at the end.

1const response = new Response(new Uint8Array([65]));2const reader = response.body.getReader();3const { done, value } = await reader.read();4console.log(done);5console.log(value.byteLength);

Line 1 makes a response whose body is one byte, the number 65. Line 3 reads the first chunk. Line 4 prints false, because a chunk came back. Line 5 prints 1, because the chunk holds one byte.

response.bodyUsed tells you whether the body has been read. It starts as false.

1const response = new Response("tea");2console.log(response.bodyUsed);3const reader = response.body.getReader();4await reader.read();5console.log(response.bodyUsed);

Line 2 prints false: nothing has been read yet. Line 3 takes a reader. That locks the stream, but it reads nothing. Line 4 reads the first chunk. Now line 5 prints true. A later response.json() would fail, because the body was already used.

To read the whole stream, call read() in a loop until done is true.

1const response = new Response(stream);2const reader = response.body.getReader();3const decoder = new TextDecoder();4let text = "";5while (true) {6  const { done, value } = await reader.read();7  if (done) break;8  text += decoder.decode(value, { stream: true });9}10text += decoder.decode();11console.log(text);

This loop reads a whole stream and turns its bytes into text. In plain words, line by line:

• Line 1 puts a stream into a response. The stream comes from setup.js and holds three chunks of bytes.
• Line 2 takes a reader.
• Line 3 makes a TextDecoder: a tool that turns bytes into text.
• Line 6 waits for the next chunk.
• Line 7 stops the loop when done is true.
• Line 8 turns the chunk into text and adds it to text.
• Line 10 finishes the decoding, and line 11 prints Hi 🌊!.

A progress bar shows how much of the body has arrived. To make one, count the bytes as each chunk comes in.

To show a percentage, you also need the total size. The server may send it in the Content-Length header. A header is a line of extra information that comes with the response. Content-Length says how many bytes the body has. Then the percentage is received / total * 100.

If the header is missing, you don't know the total. Then show the bytes received, but no percentage. A made-up percentage would be wrong.

1const chunks = [3, 2];2let received = 0;3for (const bytes of chunks) {4  received += bytes;5  console.log(received + " bytes");6}

Line 1 lists two chunk sizes: 3 bytes and 2 bytes. Line 4 adds each size to received. Line 5 prints the total so far, so you see 3 bytes and then 5 bytes. A percentage would also need the total size from Content-Length.

1function makeSlowResponse({ totalBytes, chunkBytes, delayMs, contentLength }) {2  let sent = 0;3  const stream = new ReadableStream({4    async pull(controller) {5      await wait(delayMs);6      const size = Math.min(chunkBytes, totalBytes - sent);7      controller.enqueue(new Uint8Array(size).fill(65));8      sent += size;9      if (sent >= totalBytes) controller.close();10    },11    cancel() {12      console.log("reader cancelled the slow server");13    },14  });15  return new Response(stream, {16    headers: contentLength ? { "Content-Length": String(totalBytes) } : {},17  });18}

Press Hide Content-Length and start again. The same bytes arrive, but the bar can't know the total, so it shows bytes only.

There is one more catch. Servers often compress a response: they squeeze it smaller so it is faster to send. Then Content-Length may count the squeezed bytes, while your reader gets the larger unsqueezed bytes. So progress can look wrong, or even go past 100%.

Sometimes you want to stop early. The user may leave the page, pick a smaller file, or press Stop. Call reader.cancel() to say you want no more chunks. To end the fetch request itself, pass an AbortController's signal to fetch instead.

1const stream = new ReadableStream({2  start(controller) {3    controller.enqueue("first chunk");4  },5  cancel(reason) {6    console.log(reason);7  },8});9const reader = stream.getReader();10console.log((await reader.read()).value);11await reader.cancel("user stopped");

A stream you make yourself can hold any kind of value, so this chunk is a string. Line 3 puts the chunk "first chunk" into the stream. Line 10 reads it and prints it. Line 11 calls reader.cancel("user stopped"). That runs the stream's cancel function on line 5 with the reason, so line 6 prints user stopped.

Bytes are not letters. A TextDecoder turns bytes into text. It uses UTF-8, the usual way to store text as bytes.

In UTF-8, some characters take more than one byte. The emoji 🌊 takes 4 bytes. A chunk can end in the middle of a character. Then the rest of its bytes come in the next chunk.

There are two ways to handle this:

• decoder.decode(chunk, { stream: true }). The stream: true option tells the decoder that more chunks are coming. It keeps a half-finished character until the rest arrives.
• TextDecoderStream. It does the same job inside a pipeline: you connect it to the stream, and text comes out.

1const bytes = new ReadableStream({2  start(controller) {3    controller.enqueue(new TextEncoder().encode("A "));4    controller.enqueue(new TextEncoder().encode("🌊 B"));5    controller.close();6  },7});8const text = bytes.pipeThrough(new TextDecoderStream());9for await (const piece of text) console.log(piece);

Lines 3 and 4 put two byte chunks into the stream. Line 8 connects the stream to a TextDecoderStream with pipeThrough. Line 9 prints each text chunk: "A " and then "🌊 B".

TextDecoderStream handles split characters for you. A TextDecoder in your own loop gives you more control over each read.

1const response = new Response(stream);Run2const reader = response.body.getReader();3const decoder = new TextDecoder();4let text = "";5while (true) {6  const { done, value } = await reader.read();7  if (done) break;8  text += decoder.decode(value, { stream: true });9}10text += decoder.decode();11console.log(text);

Watch the emoji in the step-through. Its 4 bytes are split over three chunks. With stream: true, the decoder waits until it has all 4 bytes. Without it, the decoder must finish each chunk at once. So the broken bytes turn into �, the replacement character.

The slow server above is a fake one, made inside this page. Your browser can also read a real response as a stream. This experiment fetches location.href, the address of this lesson page. Then it shows:

• the Content-Length and Content-Encoding headers, if the browser lets your code see them (Content-Encoding says whether the body was compressed),
• how many chunks arrived,
• how many bytes arrived.

1const response = await fetch(location.href);2const total = Number(response.headers.get("Content-Length"));3const reader = response.body?.getReader();4let received = 0;5let chunks = 0;6while (reader) {7  const { done, value } = await reader.read();8  if (done) break;9  received += value.byteLength;10  chunks += 1;11}12console.log({ received, chunks, total: Number.isFinite(total) ? total : null });

You can also make your own ReadableStream. You give it an object with up to three functions. This object is called the underlying source:

• start(controller) runs once, when the stream is made.
• pull(controller) runs when the reader wants more chunks.
• cancel(reason) runs when the reader gives up.

The controller is how the source adds chunks. controller.enqueue(chunk) puts a chunk into the stream's queue, its waiting line of chunks. controller.close() ends the stream.

pull runs only when the reader wants more. So a slow reader gets chunks slowly. This is called backpressure: a slow reader holds back the source, so chunks don't pile up in memory.

A TransformStream changes each chunk as it passes through. pipeThrough() connects a stream to it and gives back a new stream with the changed chunks. In the lab below, line 18 sends "ada" and "lovelace" through a TransformStream that makes them uppercase.

1const source = new ReadableStream({2  start(controller) {3    controller.enqueue("ada");4  },5  pull(controller) {6    controller.enqueue("lovelace");7    controller.close();8  },9  cancel(reason) {10    console.log("cancelled", reason);11  },12});13const uppercase = new TransformStream({14  transform(chunk, controller) {15    controller.enqueue(chunk.toUpperCase());16  },17});18const transformed = source.pipeThrough(uppercase);19for await (const chunk of transformed) {20  console.log(chunk);21}

1const stream = new ReadableStream({2  start(controller) {3    controller.enqueue(new TextEncoder().encode("one"));4    controller.enqueue(new TextEncoder().encode("two"));5    controller.close();6  },7});8const [left, right] = stream.tee();9console.log(await new Response(left).text());10console.log(await new Response(right).text());

Line 8 calls tee(). It splits the stream into two branches, left and right. Both branches get the same chunks. Lines 9 and 10 read each branch, so both print onetwo. One thing to know: if one branch is read slowly, the stream keeps its chunks in memory until that branch catches up.

You can also read a stream with for await (const chunk of stream). It gives you one chunk each time round the loop. Node 22 and newer browsers support it, but some browsers don't yet. So the lab checks for it first. The getReader() loop works everywhere.

Download progress uses response.body. Upload progress goes the other way: it shows how much of your data has left the browser.

fetch has no upload progress events. An event is a signal that something happened, which your code can listen for. The older XMLHttpRequest, often called XHR, does have them, on xhr.upload.onprogress. That is why many upload widgets still use XHR.

1// fetch has download streams, but no upload progress events.2const xhr = new XMLHttpRequest();3xhr.upload.onprogress = (event) => {4  if (event.lengthComputable) {5    console.log(Math.round((event.loaded / event.total) * 100) + "%");6  } else {7    console.log(event.loaded + " bytes uploaded");8  }9};10xhr.open("POST", "/same-origin-upload");11xhr.send(file);

fetch can send a stream as the request body, with the option duplex: "half". But only some browsers support it, mainly Chromium-based ones such as Chrome and Edge. And it still gives no progress events. The Cancellation lesson shows how to stop a request. The XMLHttpRequest lesson is coming soon.

Streams are not a better fetch. They are a lower-level tool, and they take more code. Use them when early chunks help. Wait for the whole body when it is small and you need all of it.

Each row of the table is a common task. Read across to see what to use and why.

• “Every response has a body stream.” Not always. response.body is null when there is no body, for example for a 204 response or a HEAD request.
• “Content-Length always gives perfect progress.” The header can be missing. CORS can hide it from your code. And if the server compressed the body, the header may count the compressed bytes, while your reader gets more bytes.
• “I can read the stream and then call json().” No. Reading the stream uses up the body. Pick one way to read it, or call response.clone() first.
• “Chunks are strings.” Fetch body chunks are Uint8Array bytes. Use a TextDecoder to turn them into text.
• “fetch has upload progress because it has download streams.” No. fetch has no upload progress events. Use XHR when you need them.

Key takeaways

• response.body is a ReadableStream of Uint8Array chunks, when the response has a body.
• getReader() locks the stream, and the first read uses up the body.
• A download percentage needs a total you can trust, usually from Content-Length. Without one, show bytes only.
• Text split across chunks needs TextDecoder with { stream: true }, or TextDecoderStream.
• Use XHR for real upload progress events. fetch streams only help with downloads.

Final definition: streaming means reading a response body chunk by chunk, so your code can use the bytes before the whole body has arrived.

Up next: WebSocket & Server-Sent Events.