Design an AI image generation studio
The web app of an image generator like Midjourney or Adobe Firefly: write a prompt, wait while a GPU job runs, get a grid of results, then vary, upscale, and edit them. The hard parts are modelling slow, costly server jobs on the client, showing honest progress, charging credits fairly, and keeping a growing tree of generations and a big gallery fast.
The question
Design the web app of an AI image generator, like the Midjourney web editor or Adobe Firefly. The user writes a prompt, picks settings such as aspect ratio and style, and presses Generate. A GPU job runs for 10 to 60 seconds and returns a grid of four images.
From any result the user can make variations, upscale it, or repaint a region. Every generation costs credits, and all of them land in a gallery the user can search later. Walk me through the client: how jobs are submitted and tracked, how progress reaches the screen, how credits work, and how the results and the gallery stay fast.
Real-life examples: Midjourney web app, Adobe Firefly, Leonardo.ai, Ideogram, ChatGPT image generation, Canva Magic Media
Requirements and what makes this hard
An image studio looks like a form with a big button. You type a sentence, press Generate, and pictures appear. The twist is the wait in between: a model runs on a rented GPU for roughly 10 to 60 seconds, and OpenAI's image docs warn that complex prompts can take up to 2 minutes.
Many image models, Stable Diffusion among them, are diffusion models: they start from random noise and remove a little of it in each of many steps (often 20 to 50) until an image appears. Because the picture forms step by step, the server can report progress and even send blurry previews along the way. That GPU time is also the most expensive thing in the product, and many users wait for the same pool of GPUs at once.
Before drawing anything, ask questions. Each one below changes the design, and the answer in brackets is what this guide assumes.
- Is generation synchronous or a job? (A job. The server returns a job id at once and the work happens later, because many load balancers and proxies close a request that sends nothing for 60 seconds, the default in AWS load balancers and nginx.)
- Does the user pay per image? (Yes, in credits: a prepaid balance. A standard grid of 4 costs a few credits, an upscale costs more.)
- Are there speed tiers? (Yes. A fast mode that costs credits and a slower, cheaper or free mode that waits for idle GPUs, like Midjourney's Fast and Relax modes.)
- Can the user leave while a job runs? (Yes. The job keeps running on the server and the result shows up in the gallery later.)
- What can the user do with a result? (Vary it, upscale it, remix it, or repaint a region. Each is a new job linked to its parent.)
- Is there moderation? (Yes, on the prompt and on the output images, all on the server.)
- How many images does a heavy user have? (Tens of thousands: on a busy day a heavy user runs 200 jobs, which is 800 images. The gallery must stay fast.)
Functional requirements
- Compose a prompt: text, a negative prompt (what to leave out), reference images, a style preset, and parameters such as aspect ratio, number of images, model version, and seed (the number that picks the starting noise, so the same seed and prompt give similar images).
- See the credit cost of the current settings before submitting, and the remaining balance.
- Submit a generation and follow it: queued with a position, running with a percentage and, where the model allows it, blurry previews that sharpen.
- Cancel a queued or running job, with the credits returned as a cancel policy shown up front says.
- Act on a result: variations (subtle or strong), upscale (a bigger version), remix (rerun with an edited prompt), and inpaint (repaint a masked area) or outpaint (extend the canvas). See the family tree of a result.
- Browse, filter, download, share, and delete past generations in a gallery.
Non-functional requirements
| Requirement | Target | Why it matters here |
|---|---|---|
| Submit feedback | Under 100 ms to show the job card | The press must feel instant even though the work takes a minute. |
| Progress freshness | A new state or preview within about 1 to 2 s of the server knowing it | Waiting is the main experience. Stale progress feels broken. |
| Correct billing | One charge per intent, refunds on failure | A double click must not cost twice. Trust is the product. |
| Gallery scroll | 60 fps with 20,000 items | Heavy users have huge histories. |
| Memory | Under about 300 MB on a laptop after an hour | Each full-size result decodes to 4 to 16 MB of pixels. |
| Resilience | No lost jobs if the tab closes or the network drops | The job lives on the server, the client is a viewer. |
| Accessibility | Every action works by keyboard, state changes are announced | Hover-only toolbars on images are the classic failure. |
Some rough numbers to design with (estimates, typical of 2025 and 2026 products): a standard job runs for 10 to 60 seconds; a heavy user runs up to 200 jobs on a busy day; each job produces 4 images; a standard result is about 1 megapixel (1024 by 1024) and an upscale about 4 (2048 by 2048), roughly 1 to 8 MB as PNG; a 256 px gallery thumbnail is about 15 to 30 KB as AVIF or WebP.
Out of scope
- The model, the GPU scheduler, and the moderation classifiers. We treat them as a server API.
- Canvas mechanics of the mask brush, layers, and undo. The photo-editor guide covers those; we only cover how a mask becomes a job.
- Uploading reference images. The file-uploader guide covers chunked uploads; we only send the resulting asset id.
- Payments and plan checkout. We read the plan and the balance, we don't sell them.
Check your understanding · Exercise 1
Which answers to scope questions would change the client design the most? Select all that apply.
What makes this hard
Plenty of apps call a slow API. Four things together make this one different, and each drives a later chapter.
- Slow and variable. A job runs for 10 to 60 seconds, and it may first wait in a queue for anywhere from zero seconds to many minutes, depending on load and the user's plan. The UI must stay useful for that whole time and be honest about how long is left.
- Expensive and metered. Each job burns GPU time that someone pays for. The client must show the price up front, never charge twice for one intent, and make refunds visible when the server fails.
- Generative and branching. Users rarely stop at the first grid. Every vary, upscale, or inpaint is a new job that points at its parent, so the history is a tree, not a list.
- Heavy media. Each job returns 4 images of about 1 megapixel. A decoded 1024 by 1024 image takes 4 MB of memory (4 bytes per pixel), and a 2048 by 2048 upscale takes 16 MB. A session with 100 grids would hold about 1.6 GB of pixels if the app kept everything decoded.
Compare this with the llm-chat-ui guide. A chat answer is also slow and costly, but it is text that streams over an open request in one tab, and the user reads it as it arrives. Here the useful output arrives at the end, the job outlives the request, and the result is a set of big files. So the core of this design is a job model, not a stream reader.
It also differs from the progress-bar-queue guide, where the client itself runs a queue of uploads with a concurrency limit. In the studio the queue lives on the server, shared by thousands of users. The client cannot speed it up; it can only report it honestly and let the user choose a faster, more expensive lane.
The wireframe above names the parts of the main screen we will design: the composer at the bottom (prompt, chips for parameters, a cost label on the Generate button), the job feed on the left (one card per job, newest first, each card a grid of 4 when done), the detail panel on the right for the selected image and its actions, and the gallery behind a tab, for everything older.
Think first
Why can't the Generate button just await fetch('/generate') and show the images when the promise resolves?
A minute-long request is fragile: idle timeouts at load balancers and proxies (often 60 to 100 seconds by default) or a mobile network switch can kill it, losing a job the user paid for. It also lives in one tab, and it leaves no room for queue position, progress, or cancel.
A job id returned at once fixes all three, because any tab can ask about that id at any time. The next chapter weighs the options.
The generation job: submit, states, and idempotency
Start with the naive version: Generate sends POST /generate, the server holds the request open until the GPU is done, and returns 4 image URLs. On a quiet afternoon this works. Then a launch day comes, the queue grows to 3 minutes, and requests start dying at the load balancer's 60-second idle timeout. The user sees an error, presses Generate again, and is charged twice: the first job was still running.
Decision How does the client start a generation and learn its result?
What we pick: The job resource. Replicate predictions work this way, and so do Stability's slower endpoints, which return an id at once and answer 202 on a results URL until the image is ready. OpenAI's Images API instead holds one request open, optionally streaming partial images; that suits a server calling a provider, not a browser tab that may close. The extra parts of a job are exactly the features users want: position, progress, cancel, and a result that waits for them.
The job's states
A job is a small state machine (a fixed set of states and the only moves allowed between them; the state-machines lesson covers the idea). Writing it down first stops the classic bugs: a cancelled job that later shows images, or a failed job that keeps a spinner forever.
progress event that arrives after succeeded has a lower version and is ignored.| State | Meaning | What the card shows | Credits |
|---|---|---|---|
pending | Created on the client, not yet confirmed by the server | A card with the prompt and a faint spinner | None yet |
queued | Accepted, waiting for a GPU | Position in line and an ETA range | Reserved |
running | On a GPU | Percentage and, if available, a blurry preview | Reserved |
succeeded | Images stored | The grid of 4 | Charged |
failed | Server or model error | A short reason and Retry | Refunded |
blocked | The output check removed every image (a blocked prompt is rejected at submit and never becomes a job) | A neutral notice, no images | Refunded (policy) |
cancelled | The user stopped it | A muted card with Run again | Refunded, or partly charged if it ran |
The last four are end states: nothing leaves them, so the client ignores any event that tries to move a job out of one. Updates can also arrive out of order (a slow poll response landing after a newer pushed event), so each carries a version that only goes up, and the store drops anything older. A pending card that the server rejects (402, 409, or 422, covered later) is removed, and the composer shows why: that request never became a job.
One intent, one charge
The double-charge story above has a standard fix: an idempotency key. The client makes a random id (crypto.randomUUID()) when the user presses Generate and sends it in an Idempotency-Key header. The server stores the key with the job it created. If the same key arrives again, from a retry after a timeout, a double click, or a second tab replaying a saved draft, the server returns the existing job instead of creating a new one.
async function submit(draft: Draft): Promise<Job> {
// One key per press of Generate; retries reuse it.
const key = draft.intentKey ?? (draft.intentKey = crypto.randomUUID());
const tempId = `tmp-${key}`;
jobs.add({ id: tempId, state: "pending", prompt: draft.prompt, version: 0 });
const res = await fetchWithRetry("/api/jobs", {
method: "POST",
headers: { "Content-Type": "application/json", "Idempotency-Key": key },
body: JSON.stringify(draft.params),
});
if (!res.ok) {
// 402, 422, or 409 price_changed: no job exists, and the composer
// shows why. Pressing again after a fix is a new intent.
jobs.remove(tempId);
draft.intentKey = undefined;
throw await res.json();
}
const job: Job = await res.json(); // 202 with { id, state: "queued", ... }
jobs.replace(tempId, job); // swap the temp card for the real id
draft.intentKey = undefined; // the next press is a new intent
return job;
}
Three details matter. The key is made per intent, so fetchWithRetry (backoff as in the retries-and-backoff lesson) can safely resend the same body after a network error. The button is also disabled between the press and the 202, but that is a nicety: the key is what protects the wallet, because a disabled button does nothing for a retry or a second tab. And the temporary card is an optimistic update (see the optimistic-updates lesson): it appears in under 100 ms, then swaps to the real id.
One edge case: a retry can reach the server while the first attempt is still being processed. The IETF draft that defines the header says the server should then answer 409 Conflict. Our API marks that answer request_in_progress and retryable, so fetchWithRetry waits a moment and resends with the same key; by then the job exists, and the server returns it.
Idempotency-Key, so the server returns the job it already created instead of starting and charging another one.Check your understanding · Exercise 2
The submit request times out on a train, and the client retries 3 seconds later. Which statements are true with the design above? Select all that apply.
Getting progress to the browser
The job exists on the server. Now the card on screen has to follow it: queued at position 14, then running at 35%, then a blurry preview, then 4 sharp images. Waiting is most of the experience, so this channel deserves real thought.
Decision How do job updates reach the page?
What we pick: SSE for the live channel, with polling as the fallback and as the way to catch up after any gap. The traffic is one-way, small, and bursty, which is what SSE is for (the choosing-realtime lesson walks through the general rule). Writes stay on HTTP. If the product already runs a WebSocket for chat or collaboration, ride on that instead; the event shapes stay the same.
The lab below runs one job with example timings (made up to show the shape, not measured from any product). On a busy day a fast job waits 8 s in the queue and runs for 32 s. Compare the requests made and how late the first preview shows up with polling at 2 s, polling at 5 s, and SSE. Then cancel a job, force a failure to watch the refund, and close the tab mid-job to see the result waiting after a reload.
Progressive previews
Because the picture forms over many steps, the server can send a few low-resolution snapshots along the way. Midjourney, for example, shows a blurry grid that sharpens as the job runs, and OpenAI's image API can stream 0 to 3 partial images before the final one (fewer if the image finishes quickly). Previews make a 40-second wait feel shorter because the user can already judge the composition and cancel a bad one early.
- Send previews as URLs to small files (for example 256 px WebP, about 10 to 20 KB), not as base64 inside the event. Events stay tiny, and the images go through the image CDN (content delivery network).
- Keep only the latest preview per job. Replace the
src; don't stack them. Revoke any object URLs you created. - Cap the rate: a preview every 3 to 5 seconds is plenty. More costs bandwidth and decode time for no extra meaning.
- Show previews blurred with a clear running label, so nobody mistakes a draft for the result.
An honest wait
A job has two waits: the queue wait (until a GPU picks it up) and the run time (on the GPU). They behave very differently. Run time is fairly predictable from the model and settings; the lab above assumes 32 s for a fast grid. Queue wait depends on everybody else: Midjourney's docs say Relax mode jobs can wait from 0 to 30 minutes, less for people who use Relax mode less.
- While queued, show the position ("12th in line") and a range from the server ("about 1 to 3 min"), never a fake progress bar that crawls.
- While running, show a real percentage from the step count (step 14 of 28 is 50%).
- Label every estimate as an estimate, and when it is exceeded, say so ("taking longer than usual") instead of freezing at 99%.
- Offer the way out next to the wait: cancel, or switch this job to fast mode if the plan allows it.
The tab closes, the job goes on
Because the job lives on the server, closing the tab loses nothing. On every page load the client asks GET /jobs?state=active and rebuilds the cards, then opens the event stream. When the stream reconnects after a drop, it sends Last-Event-ID; if the server can no longer replay that far back, it says so and the client refetches active jobs. Either way, the store ends up equal to the server.
Last-Event-ID replays short ones, and a refetch of active jobs covers long ones. Either way, versioned updates go through one applyJob, so nothing is applied twice.When a job finishes and the user is elsewhere, tell them in proportion to the wait. Another tab of the app: a toast and a count in the page title. The browser in the background: a system notification through the Notification API, asked for only after the user opts in, for example by ticking "notify me when done" on a long Relax job. The app fully closed: a web push or an email, if the user asked for it (the push-notifications lesson and the notifications-center guide cover delivery).
Check your understanding · Exercise 3
A user starts 3 jobs, then their laptop sleeps for 10 minutes. What should happen when it wakes? Select all that apply.
Credits, quotas, and speed modes
Every product in this space meters GPU time. Midjourney sells monthly Fast GPU hours (its docs say an image prompt usually takes about one minute of GPU time, and a Creative or Subtle upscale about 2 minutes). Adobe Firefly spends generative credits. The unit differs, but the client problem is the one the first chapter named: price up front, one charge per intent, visible refunds.
Show the price before the press
The price depends on the settings: model version, number of images, size or quality, speed mode, and the action (a fresh grid, a variation, an upscale). A user who toggles HD and sees nothing change on the button will be surprised by their balance later.
Decision Where is the cost of the current settings computed?
What we pick: The versioned price table. The button label updates in the same frame as the toggle, and the expectedCost check means the user is never charged a number they did not see. If the server's price differs, it refuses with 409 and the client shows the new price before the user confirms again.
| Action (example table, not a real price list) | Base cost | Modifiers |
|---|---|---|
| New grid of 4, standard | 4 credits | x2 for HD, x0.5 for draft quality |
| Variation (subtle or strong) | 3 credits | same multipliers |
| Upscale one image | 8 credits | only one image, so no count multiplier |
| Inpaint a region | 4 credits | price does not depend on mask size |
| Any job in relaxed mode | 0 credits | plan must include relaxed mode |
| Turbo speed | x2 | Midjourney's Turbo mode uses double the Fast time |
Reserve, settle, refund
When does the money move? Charging at the end lets a user with 4 credits start 10 jobs at once and overdraw. Charging at submit makes failures feel like theft unless refunds are visible. The usual answer, borrowed from card payments, is a hold: reserve at submit, settle when the job succeeds, release on failure, block, or cancel.
- The balance the UI shows is
available = total - held. Show the held part too ("212 credits, 4 on hold") so the number never jumps without a reason. - Submit with too few available credits gets
402 Payment Requiredwith the shortfall; the composer offers relaxed mode or a top-up, and keeps the draft. - A refund arrives as an event on the same stream as the job update, and the job card says it: "Failed: server error. 4 credits returned."
- Cancel policy must be visible before the click. A queued job costs nothing to cancel. A running one may be charged for GPU time used, which is how Replicate bills a cancelled prediction that had started. Whatever the rule, the cancel button's label or tooltip states it.
Fast and relaxed lanes
Speed tiers turn the shared queue into a product choice. Midjourney's docs describe Fast mode (uses the monthly GPU hours), Relax mode (unlimited images on the Standard plan and up, but jobs wait for free GPUs), and an experimental Turbo mode (up to 4 times faster, at double the Fast time). The plan also limits concurrency: the docs list 3 concurrent fast image jobs on the Basic plan and 12 on Pro, with up to 10 more jobs waiting in the queue.
- Make the mode a visible chip next to the Generate button with its cost, not a setting buried three menus deep.
- When an option is not in the plan, show it disabled with the reason in text ("Relaxed mode needs the Standard plan"), not just greyed out. A disabled control needs a visible explanation for keyboard and screen reader users too, so use
aria-disabledwith a description, not thedisabledattribute that removes it from the tab order. - When the concurrency limit is reached, accept the job into a client-visible "waiting for a slot" state instead of an error, if the server queues it; otherwise show how many are running and when the next slot frees up. The rate-limited-queue guide covers 429 handling and retry timing.
Check your understanding · Exercise 4
A user has 10 credits available and quickly submits three grids at 4 credits each. What should the client and server do? Select all that apply.
The prompt composer
The composer is where a job is born. It looks like a text box, but it holds a structured request: prompt text, a negative prompt, reference images, a style, and five or six parameters.
Power users type everything as text: Midjourney users write --ar 16:9 --seed 42 --no text at the end of a prompt for the aspect ratio, the seed, and what to leave out. New users want buttons. The composer must serve both without two sources of truth.
Decision How does the composer hold prompt text and parameters?
What we pick: A structured draft with two views. The parser runs on paste and on space after a flag, turning --ar 16:9 into an aspect ratio chip. The draft object is the only truth: the price label, validation, and the submit body all read from it.
--zoom stays in the text with an error, and nothing the user typed disappears silently.type Draft = {
prompt: string; // free text, flags removed
negative?: string; // "what to leave out", like Midjourney's --no
references: { assetId: string; weight: number }[]; // uploaded image prompts
stylePreset?: string; // "watercolor", "product-shot"...
params: {
model: "v7" | "v6"; // example names, like Midjourney's --v
aspectRatio: `${number}:${number}`;
count: 1 | 2 | 4;
seed?: number; // 0 to 4294967295, like Midjourney's --seed
speed: "fast" | "relaxed" | "turbo";
};
intentKey?: string; // idempotency key, set on Generate
updatedAt: number;
};
Weights and syntax
Some models accept weights on parts of the prompt. Midjourney uses a double colon (cat::2 dog::1), and popular Stable Diffusion web UIs use brackets like (cat:1.3). The composer does not need the model's maths, only enough tokenizing to highlight the syntax, catch unbalanced brackets, and keep the cursor stable while it decorates. A textarea with a highlight layer behind it, or a small contenteditable editor, does this; the mention-input and rich-text-editor guides cover decorating text as you type.
Drafts and history
- Save the draft to
localStorageon every change (debounced 300 ms), keyed by workspace. A reload or crash never loses a long prompt. Reference images are stored as asset ids, not blobs. - Prompt history is just past jobs: every job stores its full draft, so "reuse prompt" copies
job.draftinto the composer, chips included. No separate history store to keep in sync. - Arrow Up in an empty prompt box recalls the last prompt, like a terminal. It is a small feature that heavy users love.
Validation and blocked prompts
Validate what the client can know: prompt length (count characters with Intl.Segmenter or the server's token rule, and show a live counter only near the limit), parameter ranges, how many reference images, and whether the plan allows the chosen speed. Show the problem next to the field and keep Generate enabled with the reason in its description, or disabled with a visible reason; never fail silently.
Moderation is different: the rules live on the server and must stay there. When a prompt is blocked, the API returns a stable code and at most a coarse category. OpenAI's image API, for example, returns the code moderation_blocked, optionally with the stage (input or output) and broad labels such as violence, and its docs advise a generic message for users. The composer shows a neutral message, keeps the prompt, links the content policy, and highlights no words, which would teach people to evade the filter.
Check your understanding · Exercise 5
A user pastes a red fox in snow --ar 3:2 --seed 7 --zoom 2 where --zoom is not a supported flag. What should the composer do? Select all that apply.
Results and iteration: a tree of generations
A finished job shows a grid of 4 images. The user rarely stops there. They pick image 3, ask for strong variations, pick one of those, upscale it, and then repaint a hand that came out wrong: four jobs, each after the first started from one image of an earlier job. Midjourney's actions are a good model: Vary (Subtle) for small changes, Vary (Strong) for bigger ones, Upscale (Subtle or Creative) to double the size, and Vary Region in its editor to repaint a selected area.
The naive store is a flat list of jobs in time order. It answers "what did I make today" but not "where did this image come from" or "show me every version of this fox". Once a user branches twice, the flat list scatters one idea across 40 cards.
Decision How are follow-up actions recorded?
What we pick: Parent pointers. Each child job records which image it came from and which action made it. A job is immutable once it ends, so the tree never needs merging or conflict handling: new branches only add leaves.
The lab below starts from one prompt. Make variations, upscale, and inpaint, and watch the tree grow, the credits add up, and the data of each node. Notice that an upscale has a single output while a variation has four, and that every node points at exactly one image of its parent.
Pick a job and one of its images, then make a variation, an upscale, or an inpaint.
Inpainting and outpainting as jobs
Inpainting repaints a masked region of an image; outpainting extends the canvas beyond its edges. Both start in a canvas editor where the user paints a mask; the photo-editor guide covers the brush, zoom, and undo. What leaves the editor is a mask PNG the size of the source image, a few KB because it is mostly flat colour. Models disagree on the format (many Stable Diffusion tools read white as repaint, OpenAI's edit API wants an alpha channel), so the editor exports one format and the server converts.
- Export the mask with
canvas.toBlob(cb, "image/png")at the source image's real resolution, not the on-screen size. - Upload it as an asset (
POST /assets, see the file-uploader guide) and getmaskAssetId. - Submit a job with
action: "inpaint",parent: { jobId, imageIndex },maskAssetId, and the new prompt for the region. - The result is a child node. The source image is untouched, so going back is just selecting the parent: no undo stack needed across jobs.
parent pointer. The source image is untouched, so going back is just selecting the parent.Comparing results
Choosing between 4 or 12 similar images is the real work. Offer a compare view: pick 2 to 4 images and show them side by side at the same size, or one over the other with a split slider for subtle variations. Keep it a URL state (?compare=img_1,img_7) so it survives reload and can be shared with a teammate. For full-screen viewing and swipe between images, reuse the image-lightbox component rather than building another viewer.
Check your understanding · Exercise 6
A user makes strong variations of image 2 of job A, then upscales image 4 of that variation job. Which statements about the data are true? Select all that apply.
The gallery: thousands of generations
On a busy day a heavy user makes 200 jobs, which is 800 images, and over a few months the history reaches tens of thousands. The gallery is where they go back to find "that blue fox from last week". Large personal libraries are covered in depth in the google-photos guide (justified grid, scroll scrubber) and masonry layouts in the pinterest guide. Here we focus on what is different for generated images.
- Search by prompt, not by content. Every image comes with its own description: the prompt. Server-side full-text search over prompts, plus filters for model, action (fresh, variation, upscale, inpaint), aspect ratio, favourites, and date, covers most needs.
- Filters live in the URL (
?q=fox&model=v7&action=upscale), so back and forward work and a filtered view can be bookmarked. - Cursor pagination ordered by
createdAtdesc, 60 items a page. New jobs that finish while browsing are prepended from the job stream, not refetched, so the scroll position never jumps. - Group by job. A grid of 4 belongs together. Showing one cover image with a "4" badge per job cuts the item count by 4 and keeps the gallery calm; expanding shows all four.
- Virtualize. 20,000 cells cannot all be in the DOM. A windowed grid keeps about 3 screens of rows mounted (see the virtualization lesson). Generated images have known aspect ratios from their parameters, so every cell's size is known before the image loads, and nothing shifts.
One image, many sizes
A 2048 by 2048 PNG upscale can be 6 MB or more. Loading that into a 200 px gallery cell wastes 99% of the bytes and 16 MB of decoded memory. The storage keeps the original; an image CDN (or a resize step at job completion) serves sized variants.
srcset and sizes. The original never loads unless someone downloads it.| Variant | Size | Format | Approx. bytes | Used for |
|---|---|---|---|---|
| placeholder | a dominant colour, or a 16 px blur | inline in JSON | under 100 B for a colour, a few hundred for a blur | Shown instantly while the cell loads |
| thumb | 256 px long edge | AVIF, WebP fallback | 15 to 30 KB | Gallery grid, tree nodes |
| preview | 1024 px | AVIF or WebP | 80 to 200 KB | Detail panel, compare view, lightbox |
| original | full, up to 2048 px or more | PNG as generated | 2 to 8 MB | Download only |
Use srcset with width descriptors and a sizes attribute so the browser picks the variant for the cell's real width and device pixel ratio. The image-optimization and cdn-caching lessons cover formats and cache headers. Variant URLs are immutable (they include the image id and size), so they get Cache-Control: public, max-age=31536000, immutable unless the image is private, in which case they are signed URLs with a short expiry and private caching.
Download, share, delete
- Download offers PNG (original), JPEG, and WebP, converted on the server so the provenance metadata (next chapter) is written into each file. Set the file name from the prompt (first 40 characters, slugified) with the
Content-Dispositionheader, so downloads are not all calledimage.png. - Share creates a public link with an unguessable id, revocable at any time. Let the owner choose whether the prompt is shown on the shared page; for many people the prompt is their craft.
- Delete is optimistic with an undo toast for 5 to 10 seconds, then a soft delete on the server. Deleting a parent keeps its children; their "source" link shows a placeholder.
Check your understanding · Exercise 7
The gallery feels slow on a laptop after scrolling through 3,000 images. Which changes help? Select all that apply.
Safety and provenance
Image generators check content twice: the prompt before the job runs (cheap, fast, catches most problems) and the output images after (catches what an innocent-looking prompt produced). Both checks run on the server. The client's job is to present the outcome well and to never be the thing that enforces the rule.
state: blocked for that tile and never a URL to blur.Decision One of the 4 output images fails moderation. What does the client receive and show?
What we pick: Withhold the image on the server and show a neutral tile. Blurring on the server is also safe, because the original pixels never leave it; Stability's API, for example, returns a blurred image with the finish reason CONTENT_FILTERED. A client-side blur is fine only for content that is allowed but sensitive and that the user opted into (a mature-content setting with a reveal button). It is never a substitute for not sending forbidden content.
- Keep the message neutral and short. Do not name the classifier or show scores; a coarse category is the most any API exposes, and even that is better kept in logs and support tools than shown to the user.
- Keep the prompt in the composer so the user can rephrase. Offer "Think this is a mistake? Send for review", which files an appeal with the job id.
- Repeated blocks may lead to account limits. If so, warn before that happens, not after.
Content credentials and watermarks
Generated images should say that they are generated. Two techniques, used together, do that.
- Content Credentials follow the C2PA standard (Coalition for Content Provenance and Authenticity). The file carries a signed manifest: which tool made it, when, and which edits followed. Adobe Firefly applies Content Credentials automatically to assets whose pixels are fully generated, and calls them tamper-evident: changing the file breaks the signature. Their weakness is that embedded metadata is easy to strip, by a screenshot or by many upload pipelines, which is why C2PA also describes durable credentials that pair the manifest with a watermark or fingerprint pointing to a cloud copy.
- Invisible watermarks live in the pixels. Google DeepMind's SynthID, launched in beta in August 2023 for Google's Imagen model, embeds a mark the eye cannot see that is designed to stay detectable after filters, colour changes, and lossy compression such as JPEG. It survives where metadata does not, though Google says it is not foolproof against extreme edits, and reading it needs Google's detector.
In the UI, show a small "Content Credentials" entry in the detail panel listing the model, the date, and the edit chain, which you already have from the generation tree. On shared public pages, add a Report button that sends the image id and a reason to a moderation queue; reporting works without an account, with rate limits.
Check your understanding · Exercise 8
Which statements about safety and provenance in the studio are true? Select all that apply.
Architecture, data model, and API
With the hard decisions made, the parts fall out of them. The rule behind every boundary: the server owns jobs, credits, and images; the client owns the draft and the view. The client keeps a cache of server truth that it updates from events, and nothing on the client is the only copy of anything the user paid for.
applyJob, so views never talk to the network.| Part | Responsibility | Why it is separate |
|---|---|---|
| Composer | Edits the draft, parses flags, shows price and validation | Pure UI on a local object; must stay instant |
| Draft store | The draft object, persisted to localStorage | Survives reloads; nothing else writes it |
| Price table | Versioned costs fetched once, cost(draft) | Pricing rules change without a deploy |
| Job service | Submit with idempotency key, cancel, retry | One place for keys, backoff, and 402/409 handling |
| Job store | Normalized jobs and images by id, with versions | Every view reads the same job; events update one record |
| Event channel | One EventSource per tab, reconnect, catch-up refetch | Transport can change (SSE, socket, polling) without touching views |
| Wallet store | Available and held credits from events | The balance moves on submit, settle, and refund, not on page loads |
| Job feed and detail panel | Cards, grid, actions, compare | Read-only views of the stores |
| Gallery | Paged, filtered, virtualized history | Different data shape (pages) and its own URL state |
| Canvas editor | Mask painting, exports a mask PNG | Heavy code, loaded only when the user opens it |
The job store is a server-state cache, the kind TanStack Query or a small normalized store handles well. Two paths write to it: query results (active jobs, gallery pages) and stream events (job.updated). Both go through one applyJob(job) function that keeps the higher version, and wallet.updated events go through a matching applyWallet. That single entry point is what makes reconnects, duplicates, and out-of-order events harmless.
Multiple tabs each open their own stream, which is fine under HTTP/2, where streams share one connection. If many tabs over HTTP/1.1 is a real case, one tab can hold the stream and share events with the others over BroadcastChannel, a browser API that passes messages between tabs of the same origin (the multi-tab-sync guide covers choosing that leader tab).
Idempotency-Key; the read path carries a version on every job.Think first
Why does the wallet store update from server events instead of subtracting the price on the client when the user presses Generate?
Because the client only knows its own tab. Another tab, a job finishing, a refund, a monthly reset, or a top-up all change the balance. Events carry the server's number, so every tab converges on the truth.
The client can still show an optimistic hold right after the press, as long as the next wallet.updated event replaces it. The server number wins.
The data model
Four entities cover the studio: the Draft from the composer chapter, plus Job, Image, and Wallet. Where each one lives matters as much as its fields.
type JobState = "pending" | "queued" | "running"
| "succeeded" | "failed" | "blocked" | "cancelled";
type Job = {
id: string; version: number; state: JobState;
action: "generate" | "vary-subtle" | "vary-strong" | "upscale" | "remix" | "inpaint";
parent?: { jobId: string; imageIndex: number };
draft: Draft; // what was asked, for reuse and lineage
queue?: { position: number; etaSec: [number, number] };
progress?: { step: number; steps: number; previewUrl?: string };
imageIds: string[]; // filled when succeeded
cost: { held: number; charged: number; refunded: number };
error?: { code: string; message: string; retryable: boolean };
createdAt: string;
};
type Image = {
id: string; jobId: string; index: number;
width: number; height: number; // known before load: no layout shift
state: "ok" | "blocked";
urls?: { thumb: string; preview: string; original: string };
placeholder?: string; // dominant colour or tiny data URL
alt?: string; // user-edited alt text
};
type Wallet = { available: number; held: number; priceVersion: number };
version. The tree is not stored anywhere: it is the set of parent pointers.- Server cache (job store):
Job,Image,Wallet. Never edited directly; onlyapplyJobandapplyWalletwrite. localStorage: theDraftand UI preferences (grid size, default speed).- URL: the selected job and image, compare set, gallery filters. Shareable and back-button friendly.
- Component state: hover, open menus, the mask being painted.
The API
POST /api/jobs Idempotency-Key: 6f1c...
{ "action": "vary-strong", "parent": { "jobId": "j_81", "imageIndex": 2 },
"draft": { ... }, "priceVersion": 42, "expectedCost": 3 }
-> 202 { "id": "j_95", "state": "queued", "version": 1,
"queue": { "position": 12, "etaSec": [40, 120] } }
-> 402 { "code": "insufficient_credits", "needed": 3, "available": 1 }
-> 409 { "code": "price_changed", "priceTable": { ... } }
-> 409 { "code": "request_in_progress", "retryable": true }
-> 422 { "code": "prompt_blocked", "message": "..." }
POST /api/jobs/j_95/cancel -> 200 { job } (refund per policy)
GET /api/jobs?state=active -> 200 { jobs: [...] }
GET /api/images?cursor=c_9&q=fox&model=v7&limit=60
-> 200 { items: [...], nextCursor: "c_10" }
GET /api/events (text/event-stream)
id: 5121
event: job.updated
data: {"id":"j_95","version":4,"state":"running","progress":{"step":14,"steps":28}}
Errors share one shape: a stable code the client switches on, a human message, and retryable. The client retries with backoff only when retryable is true (network errors, 503, and the in-progress 409); it never retries 402, 422, or price_changed, which need the user to change something. Event ids are increasing integers per user, so Last-Event-ID resumes exactly.
Check your understanding · Exercise 9
Which responses should the job service retry automatically with the same idempotency key? Select all that apply.
Performance and accessibility
The studio's performance problem is not JavaScript; it is pixels. A long session creates hundreds of large images, and every decoded one costs width times height times 4 bytes. The plan is to keep bytes small on the wire, few images decoded at once, and the main thread free while images arrive.
Loading big images gently
- Placeholders first. Each
Imagecarries a dominant colour or a tiny blurred data URL, painted as the cell background. The grid has shape and colour before any byte of the image arrives. - Right size, modern format. Thumbnails in AVIF or WebP through
srcset, as in the gallery chapter. The 4 tiles of a job card use the thumb variant, about 4 times 25 KB instead of 4 times 2 MB for the originals. - Decode before you swap. Await
img.decode()on the final image before replacing the preview, so the swap happens in one frame instead of flashing an empty box. Useloading="lazy"for gallery cells below the fold andfetchpriority="high"only for the image the user just opened. - Preload the neighbour. In the detail viewer, when image 2 of 4 is open, start loading the preview of image 3 with
new Image().src. Arrow Right then feels instant. Only one ahead, not the whole set. - Let go of bitmaps. Virtualized cells unmount and their images can be freed. Never keep originals in a JavaScript
Mapas a cache; let the HTTP cache hold bytes and the browser decide what stays decoded. Revoke everyURL.createObjectURLyou make for masks or downloads. - Respect the network. In browsers that expose it (Chromium-based ones),
navigator.connection.saveDatais true when the user asked to save data; then skip previews during the run and load only thumbnails.
Core Web Vitals targets still apply, measured at the 75th percentile of real users: LCP (Largest Contentful Paint) of at most 2.5 s on the studio's first screen (the composer and the latest job card, not the gallery), INP (Interaction to Next Paint) of at most 200 ms for chip toggles and Generate, and CLS (Cumulative Layout Shift) near 0 thanks to known aspect ratios. Add two product metrics: time from Generate to the job card appearing, and time from the server's succeeded to the images painted.
Accessibility
- Alt text. A generated image has no author-written description, but it has a prompt. Offer the prompt as a draft alt text in an editable field in the detail panel ("Alt text: a red fox in snow, watercolor"), saved to
Image.altand used on share pages and downloads. Strip parameters and weights from it. - Announce state changes politely. One visually hidden
aria-live="polite"region for the job feed announces only transitions: "Job queued, position 12", "Job finished, 4 images", "Job failed, credits returned". Never announce every percentage; 28 announcements per job would drown the user. - Keyboard grid. The 4 images of a job and the gallery cells are a grid: one tab stop with arrow keys moving focus inside it (roving
tabindex, see the keyboard-navigation lesson and the grid pattern in aria-patterns). Enter opens the image, and the actions menu is reachable with Shift+F10 or a visible button. - No hover-only actions. Vary, upscale, and download buttons that appear only on hover are invisible to keyboard, touch, and screen-magnifier users. Show them on focus too, and always in the detail panel.
- Reduced motion. With
prefers-reduced-motion: reduce, replace the sharpening animation of previews with a plain swap and stop shimmer effects.
Check your understanding · Exercise 10
Which choices make the job feed usable with a screen reader? Select all that apply.
Running it in the interview
In 45 minutes you cannot cover all of this. Spend the time where the problem is special: the job model and the progress channel. Everything else can be a sentence plus a pointer to the trade-off.
| Minutes | What to do | What to draw |
|---|---|---|
| 0 to 5 | Scope: async jobs, credits, speed modes, actions, gallery size | Nothing yet; write the numbers down |
| 5 to 15 | Job model: 202 plus id, states, idempotency key | The state machine |
| 15 to 23 | Progress: SSE vs polling, previews, ETA, tab closes | The reconnect sequence |
| 23 to 30 | Architecture and data model: stores, applyJob, parent pointers | The component diagram |
| 30 to 38 | One deep dive the interviewer picks: credits, tree, gallery, or safety | Whatever that topic needs |
| 38 to 45 | Performance, accessibility, follow-ups | Image variants table |
Say the driving facts out loud early: jobs are slow, they cost money, they branch, and their results are heavy images. Then tie each decision back to one of them. Interviewers look for the moment you say "a double click must not charge twice" before they ask.
Think first
Follow-up: "The GPU cluster is overloaded and queue times hit 10 minutes. What changes in the client?"
Nothing breaks, because jobs already outlive requests. The UI shifts its message: show position and a wide ETA range, offer "notify me when done" through the Notification API, and suggest relaxed mode for jobs that are not urgent.
Polling fallbacks slow down to every 10 to 15 s while queued, and the server can raise the SSE reconnect delay with the retry: field so dropped streams come back more slowly. Never show a progress bar that moves when nothing is happening.
Think first
Follow-up: "How would you add a team workspace where several people see each other's generations live?"
The model already fits: jobs are immutable server records and the store applies versioned events. Scope the event stream by workspace instead of by user, add createdBy to jobs, and filter the feed.
Credits move to a workspace wallet with per-member limits, and the 402 message names whose budget ran out. Live presence and comments are separate features that sit on top.
Think first
Follow-up: "Why not let the browser call the model provider's API directly?"
The provider key would ship to every browser and could be stolen and used to run up the bill. Credits, moderation, idempotency, and storage of results all need a trusted server in the middle.
The client talks only to your API. Your server talks to the provider and receives its webhooks (Replicate, for example, can call a URL when a prediction starts, produces output, and completes), then turns them into your own job events.
Summary
- A generation is a server job, not a request:
POSTreturns 202 with an id in milliseconds, and any tab can follow that id until an end state. - Write the job's state machine down first: queued, running, succeeded, failed, blocked, cancelled. End states are final, and versioned updates make late or duplicate events harmless.
- An idempotency key made per press of Generate, reused on every retry, is what stops double charges; a disabled button is only a nicety.
- Use SSE for one-way progress with
Last-Event-IDresume, polling as the fallback, and a refetch of active jobs on every load and long gap. Show queue position and ETA ranges honestly. - Price the draft on the client from a versioned price table, hold credits at submit, settle on success, refund visibly on failure, and let the server reject a stale price with 409.
- Record every follow-up action as a child job with
parent: { jobId, imageIndex }. Immutable jobs plus parent pointers give a generation tree with no merging. - Blocked outputs are withheld on the server, never blurred on the client. Keep provenance by doing format conversions on the server, where C2PA manifests can be written.
- Images dominate cost: sized variants through
srcset, placeholders, virtualization, one-ahead preloading, and no in-memory cache of full-size bitmaps. - Accessibility: prompt-based draft alt text, polite announcements of state changes only, a keyboard grid, and no hover-only actions.
Check your answer
Tick what your own design covered. Pick the level you're interviewing for: each level includes the ones before it.
References
- Midjourney docs: GPU Speed (Fast, Relax, Turbo) GPU time per job, relax queue waits, turbo cost.
- Midjourney docs: Comparing Midjourney Plans Concurrency and queue limits per plan.
- Midjourney docs: Variations Subtle and strong variations as follow-up jobs.
- Midjourney docs: Upscalers Subtle and Creative upscales double the size and cost extra GPU time.
- OpenAI API: Image generation guide Partial image streaming, latency, and moderation errors by stage.
- Replicate: Prediction lifecycle A real job state machine, deadlines, and billing for cancelled jobs.
- Replicate: Set up webhooks Start, output, and completed events sent to your server.
- Stability AI: API reference Async generations polled until 202 becomes 200, and filtered outputs.
- Adobe Firefly: Content Credentials overview Provenance metadata applied to generated images.
- C2PA: Content Credentials specifications The standard behind signed provenance manifests.
- C2PA: Explainer Manifests, signing, and durable credentials when metadata is stripped.
- Google DeepMind: Identifying AI-generated images with SynthID Invisible watermarks that survive common edits.
- MDN: Using server-sent events EventSource, event ids, and reconnection.
- IETF draft: The Idempotency-Key HTTP Header Field The header used to make retried POSTs safe.
Concepts behind it
- State machines for UIModel a flow as a fixed set of states and the events that move between them, so impossible combinations like "loading and failed at once" can't happen and every button does the right thing in every state.
- Short pollingThe browser asks the server "anything new?" on a timer: the simplest way to keep a page fresh, and the right answer more often than people think.
- Server-sent eventsOne HTTP response that never finishes: the server keeps writing small text events to it, and the browser's
EventSourceparses them, reconnects on its own, and resumes where it left off. The simplest push channel when data flows from server to client. - Choosing a real-time techniqueHow to pick between short polling, long polling, server-sent events, WebSockets, and WebRTC for each feature, starting from who talks, how often, and how fresh the data must be, then checking scale, proxies, battery, and the fallback you will ship.
- Retries, timeouts, and backoffHow a client survives a flaky network and a struggling server: give every request a deadline, retry only the failures that can heal, space the retries out with exponential backoff and jitter, and stop retrying when the server or the network is clearly down.
- Optimistic updatesShowing the result of a user's action at once, before the server confirms it, and quietly undoing it if the server says no: the trick that makes likes, checkboxes, and chat messages feel instant on any network.
- WebhooksAn HTTP request that another company's server sends to yours the moment something happens, signed so you can trust it and retried until you answer: the first hop of many updates that end on a user's screen.
- Images and mediaSending each user an image of the right size and format, reserving its space, and loading it at the right moment: images are usually the largest share of a page's bytes and its LCP element.
- CDN cachingKeeping shared copies of your responses on servers near every user, so most requests never travel to your origin: the biggest lever for global speed and for surviving traffic spikes.
- List virtualizationRendering only the rows the user can see in a long list, so a list of 100,000 items costs about as much as a list of 30.
- Keyboard navigationMaking every action work without a mouse: a Tab order that follows the page, a focus ring you can always see, arrow keys inside composite widgets, skip links past repeated content, and no place where focus gets stuck.
- ARIA widget patternsThe tested recipes for custom widgets from the ARIA Authoring Practices Guide: combobox, listbox, menu, tabs, dialog, tree, grid, disclosure, and accordion, with the roles, states, and keys each one promises, and how to pick the right one.
- Web push notificationsMessages your server sends through the browser's push service, which wake your service worker to show a notification even when your site isn't open: powerful for re-engagement, and easy to ruin with a bad permission prompt.
Try next
- Design an LLM chat interface with streaming answersMedium
- Design a throttled progress bar queueMedium
- Design a client-side rate-limited action queueMedium
- Design a web photo editorHard
- Design an image lightbox with zoomMedium
- Design PinterestMedium
- Design Google PhotosHard
- Design a file uploaderMedium
- Design a notification centerMedium
- Design state sync across browser tabsMedium
- Design an @-mention inputMedium
- Design a rich text editorHard