cf.completefrontend
My progress
FUNDAMENTALS · MODULE 3

Uploading files

How a file gets from the user's device to your storage: through your server or straight to object storage with a pre-signed URL, in resumable chunks with honest progress, checked on both sides, and served back without opening a security hole.

Why uploads are harder than they look

Picture a video app for teachers. A teacher records a 40 minute lesson on her phone, a 1.2 GB file, and taps Upload on a train. At 70 percent the train enters a tunnel, the connection drops for eight seconds, and the bar jumps back to zero with "Upload failed".

At home she tries again. The app's API server reads the whole body into memory, runs out at about 900 MB, and crashes, taking every other user on that server with it. Nothing here is exotic: it is what happens when a team treats a file upload like any other JSON request.

Most API calls in this module send a few kilobytes and finish in under a second. A file upload breaks every one of those assumptions. It can be gigabytes, it can take minutes, it travels over the slowest link the user has (on home and mobile connections, upload bandwidth is often a fifth of download bandwidth or less), and its content is chosen by a stranger.

The five problems every upload design must answer

  1. Where do the bytes go? Through your API servers, or straight from the browser to object storage such as Amazon S3 or Google Cloud Storage?
  2. What happens when the network drops? Start again from zero, or continue from the last byte the server confirmed?
  3. What does the user see while waiting? A real progress bar needs real numbers, and not every browser API gives them.
  4. Is the file what it claims to be? A file named cat.jpg can hold an HTML page, a script, or malware. The browser can check a little; the server must check everything again.
  5. How do you serve it back safely? A user upload served from your main domain can run script in your users' sessions.
Before and after. Left, the naive way: the browser sends one request through the API server, which buffers 1.2 gigabytes before writing to storage; a drop at 70 percent starts again from zero, and the path hits 413 errors, timeouts, and out of memory crashes. Right, the real way: the API only signs a URL, the browser sends 8 MiB chunks straight into a quarantine bucket, a scanner checks the file, and clean files move to the clean bucket; a drop costs one chunk.naive: one requestBrowserAPIbuffers 1.2 GBstoragedrop at 70%:start from 0413, timeouts,out of memoryreal: direct and chunkedBrowserAPIsigns a URL8 MiB chunksquarantinescannercleana drop costsone chunk
Same file, same user. On the left your API carries every byte and a drop starts over; on the right your API only signs a URL, the bytes go straight to storage in chunks, and nothing is shown to others until it is checked.

A few words we will use

  • Object storage: a service that stores files (objects) under keys, such as S3, Google Cloud Storage (GCS), Azure Blob Storage, or Cloudflare R2. It is cheap per gigabyte and built to take huge numbers of large writes.
  • Pre-signed URL: a URL that carries a time-limited signature, letting whoever holds it do one specific thing (for example, PUT one object) without having storage credentials.
  • Chunk or part: a slice of the file, sent as its own request, so a failure only costs that slice.
  • Quarantine: a storage area for files that are uploaded but not yet checked. Nothing in it is shown to other users.

Check your understanding · Exercise 1

Which of these make a 1 GB upload different from a normal JSON API call? Select all that apply.

The simple way: multipart through your server

The oldest upload on the web is an HTML form with an input of type file and enctype="multipart/form-data". In JavaScript you do the same with FormData: append the file and any other fields, and hand the object to fetch as the body. The browser builds the request for you.

ts
async function uploadAvatar(file: File) {
  const form = new FormData();
  form.append("avatar", file); // name, type, and bytes go together
  form.append("crop", JSON.stringify({ x: 0, y: 0, size: 512 }));

  // No Content-Type header: the browser sets it, with the boundary.
  const res = await fetch("/api/me/avatar", { method: "POST", body: form });
  if (!res.ok) throw await toApiError(res);
  return (await res.json()) as { avatarUrl: string };
}
toApiError is the error helper from the API contracts lesson: uploads fail in the same shapes as other calls.

What a multipart body looks like

multipart/form-data packs several fields into one body. The browser picks a random boundary, a string that does not appear in the data, and puts it in the Content-Type header. Each part starts with two dashes and the boundary, then its own small headers, a blank line, and its value; the body ends with the boundary and two more dashes.

An annotated multipart form-data request. The Content-Type header says multipart/form-data with boundary XyZ42, which the browser writes. The first part, after dash dash XyZ42, is a form-data field named crop holding a small JSON value. The second part is a field named avatar with filename me.jpg and Content-Type image/jpeg, followed by the raw JPEG bytes. Notes point out that the file name and type are client hints, not facts. The body ends with dash dash XyZ42 dash dash.POST /api/me/avatarContent-Type:multipart/form-data;boundary=XyZ42the browser writesthe boundary here--XyZ42Content-Disposition: form-data;name="crop"{"x":0,"y":0,"size":512}part 1: a text field--XyZ42Content-Disposition: form-data;name="avatar"; filename="me.jpg"Content-Type: image/jpeg(the raw JPEG bytes)name and type areclient hints,never facts--XyZ42--two dashes: the end
Each part starts with -- and the boundary, has its own headers, then its value. The filename and the part's Content-Type come from the client, so the server treats them as hints.

Two details catch people. First, never set Content-Type: multipart/form-data yourself in fetch: your header has no boundary, so the server cannot find where the parts start and fails with a parse error. Second, the filename and the part's Content-Type are written by the client. They are hints, never facts: the browser guesses the type from the file extension, and an attacker can write anything there.

On the server: stream, don't buffer

A server can buffer a multipart upload: read the whole body into memory (or a temp file), then parse it. Or it can stream it: parse the parts as bytes arrive and pipe the file part on to disk or object storage, holding only a small window in memory.

Buffering is what many frameworks do by default, and it is how the video app in chapter 1 crashed: ten users each uploading 500 MB need 5 GB of memory at once. A streaming parser such as busboy in Node.js holds only a small buffer per upload, whatever the file size. If you send files through your servers at all, stream them.

Limits pile up in the path

Even a streaming server sits behind other machines, and each has its own limits. A large upload has to pass all of them:

  • Reverse proxy body size. nginx rejects bodies over 1 MB by default with status 413 (the client_max_body_size setting). Teams often find out when someone first uploads a 2 MB photo.
  • Timeouts. Load balancers, proxies, and serverless platforms close requests that run too long, often 30 to 60 seconds for idle connections, and serverless functions also cap the request body at a few megabytes. A 500 MB file on a 10 Mbps uplink needs about 7 minutes.
  • Worker capacity. Each slow upload holds a connection and often a worker for minutes. Fifty slow uploads can tie up a server that normally handles thousands of quick requests per second.

So multipart through your server is the right answer for small files you handle right away: an avatar under 5 MB, a small CSV import, a receipt PDF. Past a few megabytes, move the bytes out of your server's path.

Check your understanding · Exercise 2

A developer writes fetch("/api/upload", { method: "POST", headers: { "Content-Type": "multipart/form-data" }, body: form }) and the server answers 400. What is true? Select all that apply.

Direct to storage with pre-signed URLs

Object storage is built for the traffic your API servers hate: many large, slow writes. So for anything bigger than a few megabytes, the standard design lets the browser send the bytes straight to storage, while your API stays in charge of who may upload what.

The browser cannot hold your storage credentials: anyone could read them from the page. Instead your API signs a pre-signed URL: an ordinary HTTPS URL to the bucket whose query parameters name the action, an expiry time, and a signature made with your secret key. Storage lets the request through only if the signature matches exactly.

The three-step flow

A sequence diagram between the browser, your API, and storage. The browser sends POST /uploads with the file's size and type. The API answers with a PUT URL that expires in 5 minutes. The browser sends PUT with the 40 megabyte file straight to storage, and storage answers 200 with an ETag. The browser sends POST /uploads/u1/complete to the API. The API sends HEAD for the object to storage, which answers that it exists with 40 megabytes. The API answers the browser with status processing.BrowserAPIStoragePOST /uploads (size, type)PUT URL, expires 5 minPUT file, 40 MB200, ETagPOST /uploads/u1/completeHEAD objectexists, 40 MBstatus: processing
Your API handles three small requests; the 40 MB goes straight to storage. The API checks the object really exists before it believes the client's complete call.
  1. Ask. The browser sends POST /uploads with the file's name, size, and type. The API checks the user is logged in, has quota left, and may upload this kind of file, then creates an upload record with status pending and picks the storage key itself, such as quarantine/u_8f3k2/original.
  2. Send. The API returns the signed URL, valid for a few minutes. The browser sends the file to that URL with PUT. Your servers never see the bytes.
  3. Confirm. When storage answers 200, the browser calls POST /uploads/u_8f3k2/complete. The API checks the object exists (a HEAD request to storage) and moves the record to processing. Many teams also subscribe to storage events (S3 Event Notifications, GCS Pub/Sub notifications), so the backend learns about the object even if the tab closes first.

S3: pre-signed PUT or POST policy

S3 offers two kinds of signed browser upload, and the difference matters for limits.

Both are checked by storage itself, so the limits hold even if someone skips your UI.
QuestionPre-signed PUTPre-signed POST policy
Request the browser sendsPUT with the raw file as the bodyPOST with a multipart/form-data body: policy fields first, file last
What the signature can pinKey, expiry, and any headers you choose to sign, such as Content-TypeA policy document of conditions: key or key prefix, expiry, Content-Type prefix, and a size range
Size limitNo range. You can sign an exact Content-Length if you know itcontent-length-range gives a min and max in bytes, enforced by S3
Largest single upload5 GB5 GB
Progress and resumingEasy progress with XHR; resume means multipart uploadEasy progress with XHR; no resume

Use the POST policy when storage must enforce a size range; use PUT for the simplest client code and for multipart uploads. Either way, keep the expiry to minutes, not the 7 day maximum S3 allows.

Google Cloud Storage, Azure, and R2

Other clouds use the same idea under different names. Google Cloud Storage has V4 signed URLs (also up to 7 days) and POST policy documents. For large files it has resumable uploads: a first signed request starts a session and returns a session URI, and the client sends the file there in pieces, asking where to continue after a failure.

Azure Blob Storage calls its signed URLs shared access signatures (SAS). Cloudflare R2 speaks the S3 API, so S3 pre-signing code works with a different endpoint.

The bucket needs CORS

The upload is now a cross-origin request from app.example.com to my-bucket.s3.amazonaws.com, so for a PUT the browser first sends a preflight OPTIONS request. Add a CORS rule to the bucket that allows your origin, the PUT or POST method, and the headers you send, such as Content-Type. For multipart uploads, also list ETag in ExposeHeaders: the client needs each part's ETag to finish, and JavaScript cannot read a response header that is not exposed.

Decision Where should the bytes of a user upload go?

Multipart through your API

FormData POST to your server

Server streams the file part to storage

Pros

  • One request, one place for auth and checks
  • No CORS or signing setup
  • Easy to process the file right away

Cons

  • Every byte costs your bandwidth and workers
  • Hits proxy body limits and timeouts
  • No resume: a drop restarts from zero

Pre-signed single PUT or POST

API signs a URL for one key

Browser sends the whole file to storage

Browser or storage event confirms

Pros

  • Your servers handle two tiny JSON calls
  • Storage scales the bandwidth
  • POST policy enforces a size range

Cons

  • Up to 5 GB per request
  • Needs bucket CORS and a completion step
  • A drop at 90 percent costs everything

Resumable chunks (S3 multipart, GCS resumable, tus)

Start an upload, send parts, finish

Ask the server for the offset after a failure

Pros

  • A drop costs one chunk at most
  • Works for files of many gigabytes
  • Parallel parts can fill a fast link

Cons

  • More requests and more client code
  • Abandoned uploads need cleanup
  • Small files pay extra round trips for nothing

What we pick: Multipart through your API for small files you process at once (under about 5 MB). A pre-signed single upload for medium files on decent connections. Resumable chunks for anything over about 100 MB, or any upload from a phone that might take more than a minute. Large products often run two of these side by side, picked by file size.

Check your understanding · Exercise 3

Your API returns pre-signed S3 PUT URLs for profile photos. Which statements are true? Select all that apply.

Chunked and resumable uploads

A single request is all or nothing: if a 2 GB upload fails at 1.8 GB, the user starts again from zero. On a phone, a long upload almost always meets a drop: a tunnel, a lift, a switch from Wi-Fi to cellular, or the OS pausing a background tab.

The fix has two parts. Chunking splits the file into slices and sends each as its own request, so a failure costs one slice. Resuming lets the client ask the server "how much do you already have?" after any interruption, even a page reload, and continue from exactly there.

Slicing a file in the browser

A File is a Blob, and blob.slice(start, end) returns a new Blob for that byte range without reading anything into memory. The browser reads the bytes from disk as it sends each slice, so a 4 GB file does not need 4 GB of RAM.

ts
const PART = 8 * 1024 * 1024; // 8 MiB

async function uploadParts(file: File, urls: string[], done: Set<number>) {
  const etags: { PartNumber: number; ETag: string }[] = [];
  for (let i = 0; i < urls.length; i++) {
    const partNumber = i + 1;
    if (done.has(partNumber)) continue; // already on the server
    const body = file.slice(i * PART, (i + 1) * PART);
    const res = await putWithRetry(urls[i], body); // backoff + jitter
    const etag = res.headers.get("ETag"); // needs ExposeHeaders in CORS
    if (!etag) throw new Error("Missing ETag: check bucket CORS");
    etags.push({ PartNumber: partNumber, ETag: etag });
    done.add(partNumber);
  }
  return etags;
}
One part at a time for clarity. Real clients run 3 or 4 parts at once and merge the ETags with those listed by the server.

S3 multipart upload

S3's version has three calls, all of which your API can make or pre-sign. CreateMultipartUpload returns an UploadId. The browser then sends each part with UploadPart (a PUT to a URL pre-signed for that part number) and keeps the ETag that S3 returns for it. Finally CompleteMultipartUpload sends the list of part numbers and ETags, and S3 joins them into one object.

A 45 MiB file in an S3 multipart upload with UploadId 2f9a, split into six parts: five of 8 MiB and a last one of 5 MiB. Parts 1 to 3 are stored, part 4 failed and will be retried, part 5 is sending, and part 6 is waiting. To finish, the client sends Complete with each part number and its ETag. Every part except the last must be at least 5 MiB, and there can be at most 10,000 parts.45 MiB file, UploadId 2f9a, 8 MiB partspart 18 MiBstoredpart 28 MiBstoredpart 38 MiBstoredpart 48 MiBretrypart 58 MiBsendingpart 65 MiBwaitingComplete([1,"a1f"], [2,"9c0"], ... [6,"e42"])last part may be under 5 MiB; at most 10,000 parts
Each part is its own request, retried on its own. S3 only joins them when the client sends CompleteMultipartUpload with every part number and ETag.
  • Part size. Every part except the last must be at least 5 MiB. A part can be up to 5 GiB.
  • Part count. Part numbers run from 1 to 10,000, so the part size caps the file size: 10,000 parts of 8 MiB is about 78 GiB. A 200 GiB file needs parts of at least 21 MiB.
  • Resuming. ListParts with the UploadId returns the parts S3 already has. After a reload, the client asks your API for that list and only sends the rest.
  • Cleanup. Parts of an unfinished upload are stored, and billed, until you complete or abort it. Add a lifecycle rule with AbortIncompleteMultipartUpload after a few days, or abandoned uploads quietly cost money forever.

tus: an open protocol for resuming

tus is an open, HTTP-based protocol for resumable uploads. Vimeo, Cloudflare Stream, and Supabase Storage accept it, and tus-js-client and Uppy implement the browser side. Instead of numbered parts it uses one growing upload and a byte offset: how many bytes the server has stored so far.

A sequence diagram between the browser and a tus server. POST with Upload-Length 100 megabytes creates an upload, and the server answers 201 with Location /files/u7. A PATCH starting at offset 0 drops after 40 megabytes. HEAD on /files/u7 returns Upload-Offset 40 megabytes. A new PATCH with Upload-Offset 40 megabytes sends the rest, and the server answers 204 with Upload-Offset 100 megabytes.Browsertus serverPOST, Upload-Length: 100 MB201, Location: /files/u7PATCH, offset 0drops at 40 MBHEAD /files/u7Upload-Offset: 40 MBPATCH, Upload-Offset: 40 MB204, Upload-Offset: 100 MB
After the drop the client does not guess. It asks the server how many bytes arrived and continues from exactly there, so the first 40 MB is never sent twice.
  1. POST with Upload-Length: 104857600 creates the upload. The server answers 201 Created with a Location URL for it.
  2. PATCH to that URL with Upload-Offset: 0, Content-Type: application/offset+octet-stream, and the bytes. The server answers 204 with the new Upload-Offset.
  3. After a failure, HEAD on the URL returns the server's Upload-Offset. The client slices the file from there and sends the next PATCH. If the offsets disagree, the server answers 409 Conflict instead of writing bytes in the wrong place.

Every tus request except OPTIONS carries Tus-Resumable: 1.0.0, and optional extensions add checksums, expiry, and parallel uploads. The IETF HTTP working group is also drafting a resumable upload standard that grew out of tus.

Resuming after a reload

Chunks handle a drop while the page is open. Surviving a closed tab needs two more things. First, save the upload's identity (the UploadId or tus URL) in localStorage or IndexedDB, keyed by a fingerprint of the file such as name, size, and lastModified; tus-js-client does this for you.

Second, get the bytes back. After a reload the File object is gone, and a page cannot reopen a file the user picked earlier. Either ask the user to pick it again (then match the fingerprint and continue), or copy the file into the origin private file system (OPFS) or IndexedDB before uploading, which costs disk space but lets the upload continue on its own.

Choosing a chunk size

Smaller chunks lose less on each drop, but every chunk is another request with its own round trip, headers, and signature. Larger chunks are efficient on a stable link but lose more when it breaks. The lab below lets you try both: pick a chunk size, drop the network, and resume.

Chunk size
After a drop
A 120 megabyte file in 15 chunks of 8 megabytes. The server has stored 0 megabytes, 0 megabytes are in flight, and the network is up. So far 0 megabytes were sent in 0 requests, and 0 megabytes were wasted.server has 0 MBonline0120 MB0 MB sent, 0 MB wasted, 0 requests
Green is stored on the server, the active bar is the chunk in flight. A drop throws away only the chunk in flight, unless the client cannot resume, in which case everything goes.

Ready to upload a 120 MB file.

In practice, 8 to 16 MiB parts work well for S3, with 3 or 4 in flight. More parallel parts rarely help: they compete for the same uplink and slow the rest of the page.

Check your understanding · Exercise 4

You upload a 120 GiB video with S3 multipart upload using 8 MiB parts. What happens? Select all that apply.

Progress the user can trust

A user who sees nothing move for ten seconds assumes the upload is stuck and retries, or leaves. A good progress display answers three questions: is it moving, how far along is it, and how long is left? Honest numbers depend on which browser API sends the bytes.

XMLHttpRequest has upload events

The old XMLHttpRequest is still the standard way to get upload progress. Its upload property is an event target that fires loadstart, then progress repeatedly (the spec says about every 50 ms), then load, error, abort, or timeout, then loadend. Each progress event carries loaded and total in bytes, plus lengthComputable, which is true when the total is known.

ts
function putWithProgress(url: string, body: Blob, onBytes: (sent: number) => void, signal: AbortSignal) {
  return new Promise<string | null>((resolve, reject) => {
    const xhr = new XMLHttpRequest();
    xhr.open("PUT", url);
    xhr.upload.onprogress = (e) => onBytes(e.loaded);
    xhr.onload = () =>
      xhr.status < 300 ? resolve(xhr.getResponseHeader("ETag")) : reject(new Error(`HTTP ${xhr.status}`));
    xhr.onerror = () => reject(new Error("network"));
    xhr.onabort = () => reject(new DOMException("Aborted", "AbortError"));
    signal.addEventListener("abort", () => xhr.abort());
    xhr.send(body);
  });
}
Wrapping XHR in a promise with an AbortSignal lets the rest of the app treat it like fetch.
A timeline of an upload with XMLHttpRequest, over about 9 seconds. Upload events: loadstart at 0 seconds, repeated progress events until 8 seconds (the spec fires them about every 50 milliseconds; the drawing shows fewer), then the upload's load event at 8 seconds. The UI shows uploading from 0 to 100 percent between 0 and 8 seconds, then finishing until 9.2 seconds. The response, status 200, arrives at 9.2 seconds, and only then is the upload done.upload eventsloadstartprogress, progress...loadUI showsuploading 0 to 100%finishingresponse2000s2s4s6s8s10s
upload.onprogress reaches 100 percent when the last byte leaves the browser. Storage still has to answer, so show "finishing" until the response arrives.

Notice what loaded measures: bytes handed to the network, not bytes safely stored. The bar can reach 100 percent before storage has answered. Show 100 percent only after the load event with a success status, and show "finishing" in between.

One more quirk: adding a listener to xhr.upload makes a cross-origin request always send a CORS preflight, even when it would otherwise count as simple. For a pre-signed PUT that is already the case, but it explains a surprise OPTIONS request on a form-style POST.

fetch has no upload progress

fetch gives download progress through response.body, but no upload progress. The only partial workaround is a streaming body: pass a ReadableStream with duplex: "half" and count bytes as the browser pulls them. Today only Chromium-based browsers support this, only over HTTP/2 or HTTP/3, and the count runs ahead of what actually went over the wire.

So for a progress bar on a single large request, use XHR. With chunked uploads, fetch works too if you count the bytes of finished chunks, but the bar then moves one chunk at a time: with 8 MiB chunks on a 2 Mbps uplink it sits still for about 30 seconds. XHR per chunk adds smooth movement inside each chunk.

Way to sendProgress inside a requestCancelNotes
XMLHttpRequestYes, upload.onprogressxhr.abort()All browsers; the safe default for big single requests
fetch with a Blob bodyNoAbortControllerUse with chunks and count finished chunks
fetch with a stream bodyApproximateAbortControllerChromium only, HTTP/2 or newer, duplex: "half" required
Upload libraries (Uppy, tus-js-client)YesYesUse XHR inside and handle chunks and retries

Time left without the jitter

Upload speed jumps around, so a raw estimate flips between "2 minutes" and "40 minutes". Smooth it with an exponential moving average, for example speed = 0.2 * latest + 0.8 * previous, wait about 3 seconds before showing any estimate, and round to friendly units: "about 3 minutes left". For several files, base both the bar and the estimate on total bytes, not on files finished.

Accessible progress

  • Use a progress element with max and value, or role="progressbar" with aria-valuenow, aria-valuemin, and aria-valuemax, and give it a name such as "Uploading lesson.mp4".
  • Announce milestones in a polite live region (25, 50, 75 percent, done, failed), not every percent: a screen reader reading every change is unusable.
  • Always offer Cancel, and Pause and Resume for big files. Make failures say what to do: "Connection lost. We'll keep trying" beats "Error".
  • Warn with a beforeunload handler while an upload is in progress and cannot resume, so a stray tab close does not lose ten minutes of work.

Check your understanding · Exercise 5

Which statements about upload progress are true? Select all that apply.

Validation on both sides

Validation happens twice, for two reasons. In the browser it is kindness: tell the user in 50 ms that their 3 GB file is too big, not after a 10 minute upload. On the server it is security: the only check an attacker cannot skip. Browser checks copy the server's rules; they never replace them.

What the browser can check

  • The picker filter. accept="image/png,image/jpeg,.pdf" on the file input shows only matching files in the picker. It is a hint: the user can switch the picker to "All files", and drag and drop ignores it entirely.
  • Size. file.size is exact and free. Check it against the same limit the server enforces, and say the limit in the message: "This video is 2.4 GB. The limit is 2 GB."
  • Declared type. file.type is a guess the browser makes from the file extension. Rename setup.exe to cat.jpg and file.type says image/jpeg. For unknown extensions it is an empty string.
  • Real type, by sniffing. Read the first bytes with file.slice(0, 16).arrayBuffer() and compare them with known magic numbers: the fixed bytes most formats start with. This catches honest mistakes, like a PDF renamed to .jpg, before any upload.
  • Image dimensions. createImageBitmap(file) gives width and height, so you can reject an image that is too small or too large before sending it.
ts
const SIGNATURES: { type: string; offset: number; bytes: number[] }[] = [
  { type: "image/jpeg", offset: 0, bytes: [0xff, 0xd8, 0xff] },
  { type: "image/png", offset: 0, bytes: [0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a] },
  { type: "image/gif", offset: 0, bytes: [0x47, 0x49, 0x46, 0x38] }, // GIF8
  { type: "image/webp", offset: 8, bytes: [0x57, 0x45, 0x42, 0x50] }, // WEBP after RIFF....
  { type: "application/pdf", offset: 0, bytes: [0x25, 0x50, 0x44, 0x46, 0x2d] }, // %PDF-
];

async function sniff(file: File): Promise<string | null> {
  const head = new Uint8Array(await file.slice(0, 16).arrayBuffer());
  const hit = SIGNATURES.find((s) => s.bytes.every((b, i) => head[s.offset + i] === b));
  return hit?.type ?? null;
}
A small allow list beats a big deny list. Anything that matches no signature is rejected.

Try it below. Each sample has a name, the type the browser would guess from it, and its real first bytes. Notice which checks catch which tricks, and which trick gets past the browser entirely.

Pick a file

file.type says image/jpeg. Really: a real JPEG photo.

The first 8 bytes of cat.jpg in hex: FF D8 FF E0 00 10 4A 46. browser: extension allowed: pass. browser: size under 10 MB: pass. browser: magic bytes allowed: pass. server: decode and re-encode: accepted.first 8 bytes of cat.jpgFF.D8.FF.E0.00.10.4AJ46Fbrowser: extension allowedpassbrowser: size under 10 MBpassbrowser: magic bytes allowedpassserver: decode and re-encodeacceptedthe browser lets it upload
The browser checks are quick and kind, but someone using curl skips all three. Only the server row counts for security, and only decoding and re-encoding removes what hides inside a valid image.

What the server must check again

The server repeats every rule, using facts it controls rather than anything the client said. With direct uploads, the "server" check splits in two: limits the storage signature enforces while the bytes land, and checks a worker runs on the stored object before it is accepted.

  • Size where the bytes land: a POST policy's content-length-range, a signed exact Content-Length, or a byte counter in your upload server that aborts at the limit. A check after the upload is too late to stop the cost.
  • Real type from content: sniff the stored bytes (a library such as libmagic, or the image decoder itself) and compare with your allow list. Ignore the client's Content-Type and file name.
  • Decode and re-encode images. Decoding proves the file is an image. Re-encoding to a fresh JPEG, WebP, or AVIF drops anything hidden in it: metadata, trailing data, and polyglot payloads (one file that is valid as both an image and, say, HTML).
  • Bombs. Read image dimensions from the header before decoding, and reject anything over a pixel budget (a tiny PNG can claim 50,000 by 50,000 pixels and need 10 GB to decode). For archives, cap the number of entries and the total expanded size: a 42 KB zip can expand to petabytes.
  • File names. Never use the user's name as a path or key. Keep it as display data, cut to a sane length, and strip control characters, path separators, and the right-to-left override character, which can make a name ending in gpj.exe display as exe.jpg.
  • Active formats. SVG, HTML, and XML can carry scripts. Either refuse them, convert SVG to a raster image, or serve them only in the safe way chapter 8 describes.
CheckBrowserServer or worker
Sizefile.size, before uploadSigned policy or byte counter, while bytes land
Typeaccept filter, file.type, sniff first bytesSniff stored bytes; decode with a real parser
Image pixelscreateImageBitmap width and heightRead header first, then decode within a budget
Hidden contentCannot checkRe-encode; virus scan
NameShow it, trimmedSanitize; never a path; random key
Who may uploadHide the buttonAuth and quota check before signing

Return rejections in your API's usual error format (see the API contracts lesson): 413 for too large, 415 for a refused type, 422 for anything else, so the UI can show the reason next to the file.

Check your understanding · Exercise 6

A user picks a file named photo.jpg. The browser reports file.type === "image/jpeg". What do you actually know? Select all that apply.

Shrinking images before they leave the device

A phone photo is usually 12 megapixels (4032 by 3024), up to 48 in high resolution modes, saved as a 3 to 8 MB JPEG or HEIC file. If your app never shows it wider than 2048 pixels, most of those bytes are wasted, and on a 2 Mbps mobile uplink each one costs time and data.

Resizing in the browser first is often the biggest single speed-up an upload feature can get. A 4.5 MB photo scaled to 2048 pixels and saved as JPEG at quality 0.85 typically lands between 300 and 600 KB: about ten times fewer bytes, and ten times less waiting.

A bar chart of one phone photo. The original 12 megapixel JPEG is 4.5 megabytes and takes about 18 seconds to upload at 2 megabits per second. Resized to 2048 pixels and saved as JPEG at quality 0.85 it is about 0.45 megabytes and takes about 1.8 seconds. A note says decoding the 12 megapixel photo needs about 48 megabytes of memory.original 12 MP4.5 MB: 18 s at 2 Mbps2048 px JPEG0.45 MB: 1.8 sdecoding 12 MP needs about 48 MB of RAM
A tenth of the bytes is a tenth of the wait on a slow uplink. The cost moves to the device: decoding needs memory, so resize one or two photos at a time, in a worker.

The steps

  1. Decode the file with createImageBitmap(file), which also works inside a Web Worker.
  2. Scale by drawing the bitmap onto a smaller canvas. Keep the aspect ratio, and never scale up.
  3. Encode with canvas.toBlob(callback, "image/jpeg", 0.85), or in a worker with OffscreenCanvas and convertToBlob({ type, quality }).
  4. Upload the new Blob, giving it a file name with the right extension.
ts
// Runs in a Web Worker so 40 photos don't freeze the page.
async function shrink(file: File, maxSide = 2048): Promise<Blob> {
  const bitmap = await createImageBitmap(file); // applies EXIF orientation
  const scale = Math.min(1, maxSide / Math.max(bitmap.width, bitmap.height));
  const w = Math.round(bitmap.width * scale);
  const h = Math.round(bitmap.height * scale);
  const canvas = new OffscreenCanvas(w, h);
  const ctx = canvas.getContext("2d");
  if (!ctx) throw new Error("No 2D context");
  ctx.imageSmoothingQuality = "high";
  ctx.drawImage(bitmap, 0, 0, w, h);
  bitmap.close(); // free the decoded pixels now, not at GC time
  return canvas.convertToBlob({ type: "image/jpeg", quality: 0.85 });
}
If the browser cannot encode the type you ask for, it returns a PNG instead. Check blob.type before naming the file.

EXIF orientation

A phone camera stores pixels the way the sensor sees them and writes a separate EXIF Orientation tag (values 1 to 8) saying how to rotate them for display. A portrait photo is often stored as a landscape image with a "rotate 90 degrees" tag.

Since about 2020, all major browsers apply that tag by default, both for an img (CSS image-orientation: from-image) and when drawing to a canvas, so the pixels you draw are already upright. The classic bug is old code, written when browsers ignored the tag, that reads EXIF and rotates by hand: in a modern browser it rotates twice. Test with real portrait photos from an iPhone and an Android phone.

Traps

  • Memory. Decoded pixels take 4 bytes each: a 12 megapixel photo is 48 MB in memory, a 48 megapixel one about 192 MB. Process photos one or two at a time, call bitmap.close(), and expect low-end phones to fail on huge images. iOS Safari has also limited canvas area in the past (about 16.7 million pixels on many devices), so test the largest images you accept.
  • HEIC. iPhones shoot HEIC by default. iOS usually hands web pages a JPEG copy when the user picks a photo, but desktop Chrome and Firefox cannot decode HEIC at all. If createImageBitmap fails, upload the original and let the server convert it.
  • Formats with transparency. JPEG has no alpha channel, so a PNG logo with a transparent background turns black or white. Keep PNG or use WebP for those.
  • Originals that matter. For a print shop, a photo archive, or medical images, the user wants the original. Resize only when the product shows a smaller version anyway.

Re-encoding also drops EXIF metadata, including GPS location, a real privacy win for photos posted publicly. The server should still strip metadata itself, because not every client resizes and an attacker never will. The server, or an image CDN as in the image optimization lesson, still makes the exact sizes and formats the app displays.

Check your understanding · Exercise 7

An app resizes photos in the browser before upload, with resize code written in 2018. Users report that some portrait photos arrive sideways. What is the likely cause and fix? Select all that apply.

After the upload: scanning, processing, and serving safely

When storage answers 200, the file is uploaded but not yet trusted. Before anyone else sees it, it must be checked and usually processed: scanned for malware, re-encoded, cut into thumbnails, or transcoded. This work is slow and can fail, so it runs in the background, never inside the upload request.

Quarantine first, then promote

New files land in a quarantine bucket (or prefix) that no user can download from. Each new object triggers a storage event (for example an S3 Event Notification into a queue), and a worker runs the checks from chapter 6 and a virus scan. Clean files are copied to the real bucket and the database record moves to ready. Bad files are deleted, and the record moves to rejected with a reason.

An architecture in four layers. In the browser, the uploader and the status view. In your backend, the upload API, a queue, and a scan and process worker. In storage, a quarantine bucket and a clean bucket. On a separate user-content domain, a CDN that serves files with signed URLs. The uploader asks the upload API for a slot, then sends bytes straight to the quarantine bucket. A storage event goes into the queue, the worker scans and processes the file, writes it to the clean bucket, and marks it ready, which the status view shows. The CDN serves files from the clean bucket.browseruploaderstatus viewyour backendupload APIqueuescan, processstoragequarantinecleanuser-content domainCDNsigned GET URLs1. slot2. bytes3. eventready
The bytes go from the browser to quarantine without touching your API. Everything after that is background work, and users only ever download from the clean bucket, through a different domain.
  • The scanner. ClamAV is the common open-source choice, run as a worker that pulls from the queue. Clouds also sell managed scanning, such as Amazon GuardDuty Malware Protection for S3 and Microsoft Defender for Storage. Keep the signature database updated: a scanner with old signatures misses new malware.
  • Hashes. Compute a SHA-256 of each file. It lets you skip scanning a file you have already cleared, block a file that was reported once, and store one copy of identical uploads.
  • Scanner failures. A scan can time out or crash on a strange file. Retry a few times, then keep the file in quarantine and alert a human; never treat "scan failed" as "clean".

The file is a small state machine

The database record for each upload moves through a few states, and the UI shows each one. This is what makes the system honest: "uploaded" and "ready" are different words because they are different facts.

A state machine for an upload record. pending goes to uploading when the client starts sending. uploading goes to processing on complete, and processing goes to ready when the file is clean. uploading goes to interrupted on a drop, and interrupted goes back to uploading on resume. processing goes to rejected for a bad file. pending goes to expired after 24 hours without an upload, and interrupted goes to expired when abandoned.pendingstartuploadingcompleteprocessingcleanreadydropresumeinterruptedbad filerejected24 hexpiredabandoned
Uploaded and ready are different states. Interrupted can come back; expired and rejected cannot, and a cleanup job deletes what they leave behind.

The client learns that a file moved from processing to ready by polling GET /uploads/:id every few seconds, or through a push channel the app already has (server-sent events or a WebSocket). A chat attachment can show a local preview at once, as an optimistic update, and swap in the real URL when the file is ready.

Serving uploads without opening an XSS hole

Here is the attack. A user uploads invoice.html, or an SVG with a script element inside, and shares the link. If you serve it from app.example.com with a type the browser renders, the script runs on your origin: it can read the victim's page data and call your API with their session. This is stored XSS, which the XSS lesson covers.

  • A separate site. Serve user files from a different registrable domain, such as example-usercontent.com, not a subdomain of your app. Google uses googleusercontent.com and GitHub uses githubusercontent.com for exactly this reason. A script that does run there has no cookies and no access to your app's origin.
  • Your own Content-Type. Set it from the type the server detected, never from what the client claimed, and send X-Content-Type-Options: nosniff so the browser does not guess a more dangerous type.
  • Content-Disposition: attachment for anything that is not an allow-listed inline type (images, video, PDF if you trust the viewer). The browser then downloads it instead of rendering it. Put the cleaned file name in it, with filename*=UTF-8''... for non-ASCII names.
  • A strict CSP on user content: Content-Security-Policy: sandbox; default-src 'none' stops scripts even if a file is rendered.
  • Private files stay private. Keep the bucket private, put a CDN in front, and give allowed users short-lived signed GET URLs, as on the upload side.
http
HTTP/2 200
content-type: application/pdf
content-disposition: attachment; filename="invoice.pdf"; filename*=UTF-8''invoice.pdf
x-content-type-options: nosniff
content-security-policy: sandbox; default-src 'none'
cache-control: private, max-age=300
Served from the user-content domain. Every header here is chosen by the server, not by the uploader.

Check your understanding · Exercise 8

Which measures stop an uploaded HTML or SVG file from running script against your app's users? Select all that apply.

Limits, quotas, and what breaks in production

Uploads are the easiest way to cost a company money by accident, or on purpose: one script uploading 1 GB every second fills 86 TB in a day. Decide limits with the product team, and enforce each one at the earliest point that cannot be skipped.

The limits worth having

LimitExampleWhere it is enforced
Size per fileAvatar 5 MB, video 4 GBSigned policy or upload server; repeated in the UI
Types per featureAvatars: JPEG, PNG, WebPWorker sniffs and decodes; UI accept as a hint
Storage quota per user or team15 GB on the free planAPI checks before signing; settles on complete
Rate100 upload starts per user per hourAPI rate limiter on POST /uploads
Pending uploadsAt most 20 unfinished per userAPI refuses new slots; cleanup job expires old ones
Signature lifetime5 to 15 minutesStorage rejects expired URLs

Quota has a timing problem: at signing time you only know the size the client claims. A good pattern is to reserve the claimed size when you sign, so ten parallel uploads cannot each squeeze under the quota, then settle using the real object size when the upload completes, and release the reservation if it expires.

Failure modes you will meet

What you seeUsual causeFix
403 SignatureDoesNotMatch on the PUTThe browser sent a Content-Type different from the one that was signedSend exactly the signed headers; sign the type the client will send
403 saying the request has expiredSlow start, a retry with an old URL, or a device clock far offShort-lived URLs, fetched again on each retry
CORS error in the console, nothing at the APIThe bucket has no CORS rule for this origin or methodAdd a rule for the origin, method, and request headers
Multipart complete fails: missing ETagsETag is not in the bucket's exposed headersAdd ETag to ExposeHeaders
413 before any app logProxy body limit, such as nginx's 1 MB defaultRaise it for the upload route, or go direct to storage
Two records for one fileThe user double-clicked, or a retry created a new slotIdempotency key on POST /uploads
Storage bill grows, nothing visibleAbandoned multipart parts and orphaned quarantine objectsLifecycle rules plus a job that expires pending records
Mobile uploads stall when the app is in the backgroundThe OS suspends the tabResumable protocol; resume on visibilitychange; tell the user to keep the tab open

The double-record bug ties back to the retries lesson. POST /uploads creates something, so a retry after a timeout can create a second slot and a second file. Send an Idempotency-Key generated once per picked file, and have the API return the existing slot when it sees the key again. Chunk retries are already safe; only the creation step needs this care.

In real products

  • Google Drive and YouTube APIs offer a resumable upload type: one request starts a session and returns a session URL, the client sends the file in chunks, and after a drop it asks how many bytes the server has.
  • Dropbox has upload sessions: start, append chunks with an offset, then finish with the path. Same idea, different names.
  • Uppy, from the team behind tus, is a widely used browser upload UI that can send to tus servers, to S3 with pre-signed multipart URLs, or as plain XHR multipart, with progress, retries, and pause built in.

Check your understanding · Exercise 9

A user on a free plan with a 15 GB quota and 1 GB used starts ten 2 GB uploads at once. What should the API do? Select all that apply.

Uploads in a system design interview

Uploads come up in many frontend design questions: a photo feed, chat attachments, a Drive or Dropbox clone, a video course platform, an avatar editor. Raise the topic as soon as the requirements mention files, and ask two questions first: how big can files get, and who will see them?

A decision tree. First question: is the file small, under about 5 megabytes, and handled right away? If yes, use multipart through your API. If no, second question: is it over about 100 megabytes, or sent from a phone? If yes, use resumable chunks. If no, use a pre-signed single upload. All three paths end in the same box: quarantine, scan, and serve from a separate domain.small, handled now?yesmultipart via APInobig or on mobile?yeschunks + resumenopre-signed uploadquarantine, scan, serve safely
Size and network decide how the bytes travel. What happens after they land is the same for every path: nothing is trusted until it is checked.

Walk the interviewer through this tree out loud. Most products need two branches: small files through a simple path, large ones through a resumable one. Saying where the line sits, and why, shows you thought about the users' networks and not only the happy path.

What to draw

  1. The client pipeline: pick, validate, (resize), request a slot, upload chunks, confirm. One box per step.
  2. The three-party sequence: browser, your API, storage. Make it clear the bytes skip your API.
  3. The background side: quarantine bucket, event queue, scanner and processing worker, clean bucket, CDN on a separate domain.
  4. The file record's states: pending, uploading, processing, ready, rejected, and how the UI learns about each.

How deep to go by level

LevelWhat a strong answer covers
JuniorFormData upload with a progress bar from XHR, size and type checks in the UI, and the knowledge that the server must check again
Mid-levelPre-signed direct uploads with a completion step, chunked uploads with retries, image resizing on the client, and accessible progress with cancel
SeniorResumable protocol choice (S3 multipart, tus, GCS resumable) with part sizing, quarantine and scanning, serving from a separate domain with safe headers, quotas with reservations, idempotent slot creation, and cleanup of abandoned uploads
StaffCost and abuse model (egress, storage growth, rate limits), regional storage and data residency, deduplication by hash, and how the upload state machine fits the app's offline and sync story

Sentences that score

  • "Small avatars go multipart through our API; anything over 5 MB goes straight to object storage with a pre-signed URL, so our servers never carry the bytes."
  • "For videos I'd use resumable chunks, 8 to 16 MiB, three in flight, with backoff per chunk, and store the upload ID so a reload can resume."
  • "Uploaded isn't ready: files sit in quarantine until a worker scans and re-encodes them, and the UI shows Processing until then."
  • "User files are served from a separate domain with Content-Disposition and nosniff, because an uploaded SVG is an XSS."

Think first

Design the attachment upload for a chat app where users send photos and files up to 100 MB from phones. What do you say in two minutes?

Summary

  • Treat an upload as a job with states (pending, uploading, processing, ready, rejected), not as one request. Uploaded is not ready.
  • Small files can go multipart through your API; anything bigger goes straight to object storage with a short-lived pre-signed URL for a key the server picked.
  • Large or mobile uploads must be resumable: chunks with per-chunk retries, and a way to ask the server how much it has (S3 multipart, GCS resumable, tus). Size parts for S3's 5 MiB minimum and 10,000 part limit.
  • Use XHR's upload.onprogress for progress; fetch cannot report upload progress in most browsers. Confirm completion only on the server's answer.
  • Validate in the browser for speed and kindness, and again on the server for security: real type from the bytes, size enforced where bytes land, images decoded and re-encoded.
  • Resize photos in a worker before upload for a roughly tenfold cut in bytes; modern browsers already apply EXIF orientation, so don't rotate twice.
  • Quarantine, scan, and process in the background; serve user files from a separate domain with server-chosen headers, nosniff, and attachment.
  • Set limits per file, per user, and per hour, reserve quota at signing time, make slot creation idempotent, and clean up abandoned uploads.

In the interview

Questions interviewers ask about this. Answer each out loud first, then compare.

Question 1

Why not just POST every file to your API server?

Question 2

How does a pre-signed upload stay secure if the browser talks to storage directly?

Question 3

A 2 GB upload on mobile keeps failing. How do you fix it?

Question 4

How do you show accurate upload progress?

Question 5

A user uploads an SVG profile picture. What can go wrong and how do you prevent it?

Practice it

These design problems lean on uploading files.

References

Read next