Design a browser-based music studio

Requirements: what kind of studio are we building?

A DAW (digital audio workstation) is the program musicians use to record, arrange and mix songs. Desktop DAWs like Ableton Live, Logic Pro and FL Studio are native apps with direct access to the sound card. Our job is the same experience inside a browser tab, where we share one thread with the UI, cannot touch the sound card directly, and must ask permission for every microphone and MIDI device.

Several neighbouring guides already cover parts of this, and we link them instead of repeating them. The audio player guide covers the media element, Media Session and createMediaElementSource, and the video player guide covers turning media events into one player state and accessible transport controls. The photo editor guide covers non-destructive edit stacks and heavy work in workers. The undo and redo guide covers command history.

The Figma and Excalidraw guides cover canvas rendering, and Google Docs covers merging concurrent edits. The file uploader guide covers resumable chunked uploads, and the virtualized list guide covers drawing only the rows on screen. Here we focus on what is special about making music in a browser.

Start with questions. Each answer below changes a part of the design; the middle column is what this guide assumes.

Functional requirements

• Create a project with a tempo (for example 120 BPM, beats per minute) and a time signature such as 4/4; play, stop, loop a region and jump with a playhead.
• Add audio tracks and instrument tracks; each has volume, pan, mute, solo and a chain of effects such as EQ (an equalizer that boosts or cuts frequency bands), compressor, delay and reverb.
• Import audio files by drag and drop; move, trim, split and fade clips on the timeline without changing the original file.
• Record from a microphone with a count-in and a metronome, and hear yourself while recording if you want to.
• Play instruments from a MIDI keyboard or the computer keyboard; edit notes in a piano roll.
• Draw automation: a fader or effect setting that changes over time, such as a fade out over the last 8 bars.
• Undo and redo every edit; save automatically; reopen the project on another computer.
• Export the mix to WAV or MP3, and export single tracks (stems).

Non-functional requirements

• No audible glitches. A click or dropout in the music is the worst bug. The audio thread has about 2.7 ms to produce each block of 128 samples at 48 kHz, and it must never miss that deadline because the UI is busy.
• Exact timing. Notes must land within about 1 ms of their place on the grid. Listeners hear rhythm errors of a few milliseconds on drums.
• Low latency while playing and recording. Pressing a key on a MIDI keyboard should sound within about 20 ms (an estimate of where players start to notice). Recorded takes must line up with the music to within a millisecond or two after correction.
• Never lose a take. A 4 minute vocal recording cannot be redone if the tab crashes; it must be on disk within seconds of being captured.
• Responsive UI. Dragging clips and zooming a 5 minute timeline stay at 60 frames per second, with interactions answering within 100 ms.
• Accessible: transport, faders and the track list work by keyboard and screen reader.

Out of scope

• Live jamming with remote musicians in real time: the network is too slow, and we explain why with numbers in the architecture chapter.
• Video scoring, notation (sheet music) and third-party desktop plugins such as VST: browsers cannot load native plugins.
• Media playback basics, Media Session and lock screen controls: see the audio player guide.
• The social feed, comments and publishing songs: separate product surfaces.

What makes a music studio hard

Most web apps are allowed to be a little late. A feed that renders 50 ms late is fine. A drum hit that plays 50 ms late ruins the groove. Five traits of the problem drive every later decision.

• Two clocks. The UI runs on the main thread, where timers fire late whenever a long task blocks it. Sound runs on a separate audio clock that never stops. Anything musical must be planned on the audio clock, not triggered by a main thread timer.
• A hard real-time deadline. The browser asks the audio engine for 128 samples at a time. At 48,000 samples per second that is a new block every 2.67 ms. If any code on that path is late once, the speaker plays silence for that block and the user hears a click.
• Latency everywhere. Sound takes time to leave the speakers (output latency) and to arrive from the microphone (input latency). Bluetooth headphones add 100 ms or more (an estimate; it varies by device and codec). Recording on top of a song means correcting for both.
• Heavy data. One minute of stereo audio, decoded to 32-bit floats at 48 kHz, is 48,000 x 60 x 2 x 4 bytes = 23 MB. A 20 track, 4 minute project can hold well over a gigabyte of decoded audio if we are careless.
• Long-lived, precious work. People spend hours in one session. A crash that loses a take or an hour of edits is worse than a slow page. Saving must be continuous and local first.

Here is the screen we are building. The transport bar sits at the top, track headers with name, mute, solo and arm-to-record buttons run down the left, and the timeline fills the middle with clips drawn as waveforms or note blocks. The mixer and the piano roll open in a bottom panel, and the browser panel on the right lists loops and imported files.

Notice what the screen hides. The timeline is a view of a project document, like a document editor shows text. The sound you hear is a second view of the same document, produced by an audio engine that reads the project and plans the notes on the audio clock. Keeping those two views in step, while the user edits during playback, is the core of the design.

Producing sound: the Web Audio graph

The naive approach is one audio element per track: load each file, call play() on all of them, and set volume for the faders. It works for a demo with two loops. Then it breaks: each element starts when its own decoder is ready, so tracks drift apart by tens of milliseconds.

There is also no way to start a clip at an exact future time, no effects, no way to sum tracks into a master, and no way to render the mix to a file. The audio player guide uses one media element because a podcast is one stream. A studio mixes dozens of streams sample by sample.

The Web Audio API is built for this. You create an AudioContext, which owns a clock (currentTime, in seconds) and a rendering thread. You create nodes (sources, gains, filters, analysers) and connect them into a graph.

The browser pulls audio through the graph on its own thread, 128 samples at a time, and sends the result to the speakers. Your JavaScript never touches the samples unless you ask it to.

Channel strips, buses and the master

A studio graph mirrors a hardware mixing desk. Every track gets a channel strip: an input gain, an insert chain of effects, a pan node, a fader gain, a mute gain and an analyser for its meter. Strips feed buses (a reverb bus that several tracks send a little signal to, a drums bus that groups the kit) and every bus feeds the master strip, which has a limiter so the mix never clips, and then audioContext.destination, the speakers.

function createStrip(ctx: AudioContext, out: AudioNode) {
  const input = ctx.createGain();      // clips connect here
  const pan = new StereoPannerNode(ctx);
  const fader = ctx.createGain();      // volume
  const mute = ctx.createGain();       // 0 or 1, kept separate from the fader
  const meter = new AnalyserNode(ctx, { fftSize: 1024 });
  input.connect(pan).connect(fader).connect(mute);
  mute.connect(out);
  mute.connect(meter);                 // meter is a dead end, not on the path
  return { input, pan, fader, mute, meter, inserts: [] as AudioNode[] };
}

function setFaderDb(strip: Strip, db: number, ctx: AudioContext) {
  const gain = db <= -60 ? 0 : 10 ** (db / 20);
  // a short ramp avoids a click when the value jumps
  strip.fader.gain.setTargetAtTime(gain, ctx.currentTime, 0.01);
}

Two details in that code matter. Faders are shown in decibels but a GainNode wants a linear factor, so -6 dB becomes about 0.5. And we never set gain.value directly during playback: a jump from 1 to 0 in one sample makes an audible click, so we ramp over about 10 ms with setTargetAtTime or linearRampToValueAtTime. Every node property that can change over time is an AudioParam, and its scheduled ramps are also how automation lanes play back.

Sources are cheap and one-shot

The surprise for most people is that an AudioBufferSourceNode can be started only once: calling start() a second time throws an InvalidStateError. That is by design, because source nodes are meant to be thrown away. The decoded AudioBuffer is the expensive, shared part; the source node that plays it is a few bytes.

So each time a clip plays, or a drum sample hits, the engine creates a new source, connects it to the track strip and starts it at an exact time. After it ends, it is disconnected and garbage collected.

function playClip(ctx: AudioContext, buffer: AudioBuffer, strip: Strip,
                  when: number, offset: number, duration: number) {
  const src = new AudioBufferSourceNode(ctx, { buffer });
  src.connect(strip.input);
  src.start(when, offset, duration);   // seconds on ctx.currentTime's clock
  src.onended = () => src.disconnect();
  return src;                          // kept so stop() can cancel it on edits
}

The context starts suspended

Browsers block sound until the user interacts with the page (the autoplay policy). An AudioContext created on page load usually starts in the suspended state, and its clock does not move. Call ctx.resume() inside the handler of the first click or key press, for example on the Play button, and check ctx.state before playing. Create one context for the whole app and keep it: each context holds a real audio device stream, and creating many wastes resources.

Keeping time: the look-ahead scheduler

The naive sequencer uses a timer: at 120 BPM a sixteenth note lasts 125 ms, so setInterval(playNextNote, 125) should do. Try it while the user drags a clip and React re-renders the timeline. A 40 ms long task delays the timer by 40 ms, so that hi-hat lands 40 ms late, then the next one is on time again. The beat stumbles.

Timers in JavaScript promise a minimum delay, never an exact one, and hidden tabs throttle them further. The audio clock does not have this problem. audioContext.currentTime advances in steady 128-sample steps on the audio thread, whatever the main thread is doing, and start(when) takes a time on that clock.

So the fix is to split the job between two clocks, as Chris Wilson described in his article 'A Tale of Two Clocks'. A sloppy main thread timer wakes up often, and on each wake-up it schedules every note that falls in the next short window, using exact times on the audio clock. The timer can be late; the notes cannot.

const LOOKAHEAD_S = 0.1;   // schedule this far ahead of the audio clock
const INTERVAL_MS = 25;    // how often the main thread wakes up

function tick() {
  const horizon = ctx.currentTime + LOOKAHEAD_S;
  while (true) {
    const ev = song.nextEventAtOrAfter(nextTick);
    if (!ev) break;
    const when = anchor.ctxTime + tempoMap.secondsBetween(anchor.tick, ev.tick);
    if (when >= horizon) break;
    voices.add(engine.play(ev, Math.max(when, ctx.currentTime)));
    nextTick = ev.tick + 1;
  }
}
const timer = setInterval(tick, INTERVAL_MS); // or post ticks from a Worker

From bars and beats to seconds

Musicians think in bars and beats, not seconds, and the project stores positions the same way, as integer ticks. A common resolution is 960 ticks per quarter note (PPQ). Storing ticks means a tempo change moves every note with the music, which is what a musician expects. The engine converts ticks to seconds only at the moment it schedules.

Worked example at 120 BPM in 4/4: one beat is 60 / 120 = 0.5 s, so one tick is 0.5 / 960 = about 0.52 ms. Bar 3, beat 2 is 2 full bars (8 beats) plus 1 beat = 9 beats = 8,640 ticks = 4.5 s from the start. A sixteenth note is 240 ticks, 125 ms.

Tempo can change during a song, so the project has a tempo map: a sorted list of points, each with a tick and a BPM. Converting tick to seconds walks the segments and adds up each segment's length at its own tempo. Precompute the seconds at each tempo point once per edit, then a lookup is a binary search plus one multiply. Gradual tempo ramps need an integral over the ramp; keep them out of the first version unless the interviewer asks.

type TempoPoint = { tick: number; bpm: number; startSec: number };
const PPQ = 960;

function tickToSeconds(map: TempoPoint[], tick: number): number {
  let i = map.length - 1;
  while (i > 0 && map[i].tick > tick) i--; // binary search in real code
  const p = map[i];
  return p.startSec + ((tick - p.tick) / PPQ) * (60 / p.bpm);
}

Loops, stops and edits during playback

The scheduler keeps an anchor: one pair that says song tick T plays at context time C. When the scheduling window crosses the loop end, the scheduler schedules the remaining events up to the loop end, then moves the anchor so the loop start tick maps to the context time where the loop end would have played, and continues. No gap, no double hit. A tempo change during playback also just moves the anchor to the current position.

Every source the scheduler starts goes into a voices set with its start time. On Stop, the engine calls stop() on every voice and ramps the master down over 10 ms. On an edit, it stops voices that belong to the changed clip and have not started yet, rewinds nextTick to the current position, and lets the next tick re-plan. That is the payoff of a short look-ahead: at most 100 ms of audio is ever committed.

Drawing the playhead from the audio clock

The playhead is drawn in a requestAnimationFrame loop that reads the audio clock, never counted with its own timer. ctx.currentTime is not what the user hears: samples processed now reach the speaker only after the output latency. With Bluetooth headphones that difference can be 100 ms or more, and a playhead that ignores it runs visibly ahead of the sound.

ctx.getOutputTimestamp() returns contextTime, the audio clock position of the sample the speaker is playing now, with the matching performance.now() time, so draw from that pair. A rough fallback is ctx.currentTime - ctx.baseLatency - ctx.outputLatency. Feature detect outputLatency: Chrome and Firefox have had it for years, but Safari added it only in version 18.4.

Custom processing on the audio thread: AudioWorklet

Native nodes cover mixing, filters, delay, compression and convolution reverb. Sooner or later a studio needs something they do not have: an accurate peak and loudness meter, a tuner, a distortion with a custom curve, a synth voice, or a way to get raw microphone samples for recording. That code must run on the audio rendering thread, inside the 128-sample deadline.

The old answer was ScriptProcessorNode, which ran your callback on the main thread. Any long task made it miss the deadline and the audio crackled, and it is now deprecated. The current answer is AudioWorklet: you load a module with ctx.audioWorklet.addModule('meter.js'), and the browser runs your AudioWorkletProcessor class on the audio thread itself.

Its process(inputs, outputs, parameters) method is called once per render quantum, the block of frames the engine renders at a time. The quantum is 128 frames by default, about 2.67 ms at 48 kHz and 2.9 ms at 44.1 kHz. Chrome 153 added a renderSizeHint option that asks for a different size (read the result from ctx.renderQuantumSize), and other browsers may follow, so read the block length from the arrays instead of hard-coding 128.

class PeakMeter extends AudioWorkletProcessor {
  constructor() {
    super();
    this.peak = 0;
    this.frames = 0;
  }
  process(inputs, outputs) {
    const input = inputs[0];
    for (let ch = 0; ch < input.length; ch++) {
      const data = input[ch];
      outputs[0][ch]?.set(data);           // pass audio through
      for (let i = 0; i < data.length; i++) {
        const v = Math.abs(data[i]);
        if (v > this.peak) this.peak = v;
      }
    }
    this.frames += outputs[0][0].length;   // 128 unless the context chose another size
    if (this.frames >= 2048) {             // about 23 times a second at 48 kHz
      this.port.postMessage(this.peak);
      this.peak = 0;
      this.frames = 0;
    }
    return true;                           // keep the node alive
  }
}
registerProcessor("peak-meter", PeakMeter);

The rules of the audio thread

• Never allocate in process(). Creating arrays or objects every 2.7 ms feeds the garbage collector, and a collection pause on the audio thread is a dropout. Allocate buffers once in the constructor and reuse them.
• Never block or wait. No locks, no await, no network, no console.log in the hot path. The thread cannot wait for anything.
• Keep the cost predictable. If processing one block takes 1 ms, you have used 37% of the deadline for one node. With 24 tracks you have no budget left.
• Talk to the main thread rarely. port.postMessage copies data and allocates; it is fine 20 to 60 times a second for meters, not for every block of audio.

A ring buffer is a fixed array used in a circle. The writer puts samples at the write index and moves it forward, wrapping to 0 at the end, and the reader takes samples from the read index up to the write index. Each index is written by only one thread, so no lock is needed: Atomics makes sure each thread sees the other's index update in the right order.

Size it for the longest stall you expect on the consumer. One second of stereo float audio at 48 kHz is 384 KB, cheap insurance.

Cross-origin isolation

After the Spectre attacks became public in 2018, browsers turned SharedArrayBuffer off. Today they expose it only on pages that are cross-origin isolated, which you check with self.crossOriginIsolated. The page must be served with two headers, COOP (Cross-Origin-Opener-Policy) and COEP (Cross-Origin-Embedder-Policy), explained in the security-headers lesson.

Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corp
# or, in Chrome and Firefox (not Safari): Cross-Origin-Embedder-Policy: credentialless

The cost: every cross-origin resource the page loads (sample packs on a CDN, avatars, fonts, analytics) must send Cross-Origin-Resource-Policy or CORS headers, or it fails to load. Popups opened to other origins lose their link back to the page, which can break OAuth sign-in popups. A common layout is to isolate only the studio route and keep the marketing site and sign-in pages normal.

Latency and recording a take that lines up

A singer presses Record, hears the song in headphones and sings along perfectly in time. Playback of the take is 40 ms late. Nothing is broken: the take is late by exactly the time sound spent travelling.

The song left the engine, waited in the output buffers, and reached the ears after the output latency. The voice reached the microphone, went through the converter and the input buffers, and reached our code after the input latency. The sum is the round-trip latency, and every recorded sample arrives that late.

Where the milliseconds go

The latencyHint option of the AudioContext constructor trades latency for stability. "interactive", the default, asks for the smallest buffers that do not glitch; "balanced" and "playback" ask for bigger buffers, which save power and survive heavy load but add delay. You can also pass a number of seconds.

A studio wants "interactive" while recording or playing a MIDI keyboard, and can offer a 'safe mode' with "playback" for slow machines that crackle. The context must be recreated to change it, so do it only when the transport is stopped.

Capturing the microphone

navigator.mediaDevices.getUserMedia({ audio: ... }) asks permission and returns a stream. Its defaults are tuned for video calls, and they damage music: echo cancellation removes anything that sounds like the speakers, noise suppression eats quiet guitar notes and reverb tails, and automatic gain control pumps the level up and down. Turn all three off, ask for the user's chosen device, and ask for stereo when the device has two inputs.

const stream = await navigator.mediaDevices.getUserMedia({
  audio: {
    deviceId: { exact: settings.inputDeviceId },
    echoCancellation: false,
    noiseSuppression: false,
    autoGainControl: false,
    channelCount: { ideal: 2 },
  },
});
const mic = new MediaStreamAudioSourceNode(ctx, { mediaStream: stream });
mic.connect(recorderNode); // an AudioWorkletNode that writes PCM to a ring buffer

Lining up the take

The recorder stamps the first captured block with its audio clock time. That tells us where the samples arrived, not when the singer sang them. To correct, shift the take earlier by the round-trip latency: keep the clip's start at the record position and set offsetSec = roundTrip. Playback then skips the first few milliseconds of the file, and every sample lands where it was sung.

The number comes from a one-time loopback calibration. The app plays a few short clicks and records them back (through a cable from output to input, or the laptop microphone next to its speaker), finds each click's peak in the recording, and takes the median distance. Store the result per pair of input and output device, and run it again when the devices change.

Monitoring means hearing yourself while recording. Through the browser, monitoring adds the full round trip, so a singer hears their own voice 30 ms late, which feels like a slap-back echo. Offer it as a switch, off by default, and suggest the audio interface's direct monitoring when one is connected. Never monitor through laptop speakers: the microphone picks them up and the loop howls.

The recording state machine

Recording has more states than a boolean can hold, and the bugs live in the transitions: a device unplugged mid-take, permission revoked, the tab hidden. Model it as a state machine, as the state-machines lesson describes, and drive the UI from the state: the record button, the count-in display and the disabled transport.

The most important rule is in the recording state: the worker writes each chunk of about 1 second to disk as it arrives, in the Origin Private File System (OPFS, covered in the storage chapter) or IndexedDB. If the tab crashes in minute 4 of a take, at most the last second is lost, and on reload the app finds an unfinished take and offers to keep it.

MIDI input, instruments and the piano roll

MIDI is the protocol, over 40 years old, that keyboards and drum pads speak. It carries no sound, only events: 'note 60 (middle C) on, velocity 100', 'note 60 off', 'controller 64 (sustain pedal) on'. An instrument track turns those events into sound, and the project stores them as notes in a clip.

Web MIDI: support and permission

navigator.requestMIDIAccess() returns the connected inputs and outputs. Support is uneven: Chrome and Edge have it, Firefox has it since version 108 (the user approves a small site permission add-on), and Safari on macOS and iOS does not support it at all. It needs a secure context (HTTPS).

Chrome used to prompt only for SysEx (system exclusive messages, which can read and rewrite a device's presets and firmware). From Chrome 124 it rolled out a prompt for any MIDI access, and asking with { sysex: true } still needs its own stronger grant, so request SysEx only if you edit device presets. All of this means the design must treat MIDI hardware as an enhancement.

• Feature detect 'requestMIDIAccess' in navigator; if it is missing, hide the MIDI settings and say why in one line.
• Ask for access only when the user opens the MIDI settings or arms an instrument track, never on page load: a prompt with no context gets denied.
• Listen to statechange on the access object so a keyboard plugged in mid-session appears without a reload.
• Always offer two fallbacks that work everywhere: the computer keyboard (the A to K row plays a white-key octave, Z and X shift octaves) and an on-screen keyboard you can click or tap.

access.inputs.forEach((input) => {
  input.onmidimessage = (e) => {
    const [status, note, velocity] = e.data;
    const type = status & 0xf0;
    if (type === 0x90 && velocity > 0) instrument.noteOn(note, velocity / 127);
    else if (type === 0x80 || (type === 0x90 && velocity === 0)) instrument.noteOff(note);
    if (recording) takeNotes.push({ note, velocity, at: e.timeStamp, type });
  };
});

Two timing details. For live playing, noteOn starts the voice at ctx.currentTime, as soon as possible, because the player is listening. For recording, the event's timeStamp is on the performance.now() clock, not the audio clock.

Convert it with the pair from ctx.getOutputTimestamp(), which maps a performance.now() time to the audio position the player was hearing at that moment, then to song ticks with the tempo map. Notes then land where the player meant them, not where the main thread happened to process them.

The piano roll

The piano roll is a grid: pitch on the vertical axis (a keyboard drawn down the left edge), time in ticks on the horizontal axis, notes as rectangles whose length is their duration and whose shade is their velocity. It is a canvas editor, much like the ones in the Figma and Excalidraw guides: draw the grid and notes on a canvas, hit-test pointer events against note rectangles, and keep the selection in the editor's own state.

• Snap and quantize. Clicks snap to the grid value (a sixteenth is 240 ticks at 960 PPQ). Quantize moves recorded notes toward the grid by a strength percentage, so a human feel survives.
• Draw only what is visible. A clip can hold thousands of notes. Keep notes sorted by start tick and binary search the visible range; the drawing cost then depends on the screen, not the song.
• Audition on edit. Dragging a note up plays its new pitch briefly, through the same instrument the track uses.
• Edits are commands. Moving 20 notes is one command with the before and after values, so undo restores them in one step, as the undo and redo guide describes.

The project model: a small document pointing at big files

Everything so far reads from one source of truth: the project. Get its shape right and saving, undo, collaboration and export all become simple. The key split is between the arrangement, which is small, edited constantly and must sync, and the audio, which is huge, written once and never changed.

type Tick = number;                          // 960 per quarter note
type Project = {
  id: string; version: number; sampleRate: 48000 | 44100;
  tempoMap: { tick: Tick; bpm: number }[];
  timeSignature: { tick: Tick; beats: number; unit: number }[];
  tracks: Track[];                            // ordered top to bottom
  buses: Bus[]; master: Strip; loop?: { start: Tick; end: Tick };
};
type Track = {
  id: string; name: string; kind: "audio" | "instrument";
  strip: Strip; instrument?: { preset: string; params: Record<string, number> };
  clips: Clip[]; automation: AutomationLane[]; frozenAssetId?: string;
};
type Strip = { volumeDb: number; pan: number; mute: boolean; solo: boolean;
  inserts: { id: string; type: string; params: Record<string, number>; bypass: boolean }[];
  sends: { busId: string; db: number }[] };
type AudioClip = { id: string; type: "audio"; assetId: string; start: Tick;
  offsetSec: number; durationSec: number; gainDb: number; fadeIn: number; fadeOut: number };
type MidiClip = { id: string; type: "midi"; start: Tick; length: Tick;
  notes: { pitch: number; start: Tick; length: Tick; velocity: number }[] };
type Clip = AudioClip | MidiClip;
type AutomationLane = { target: string; points: { tick: Tick; value: number; curve: "linear" | "hold" }[] };
type Asset = { id: string; sampleRate: number; channels: number; frames: number;
  original: { mime: string; bytes: number }; peaksReady: boolean };

• Clip times mix units on purpose. A clip's start is in ticks, so it moves with the music when the tempo changes. Its offsetSec and durationSec are in seconds of the source audio, because a recorded guitar does not speed up when you change the tempo. (Time-stretching audio to follow tempo is a separate, expensive feature.)
• Automation lanes are breakpoints. A lane targets one parameter, like track.t3.volumeDb, with points in ticks. The scheduler turns the points inside its window into linearRampToValueAtTime and setValueAtTime calls on the matching AudioParam.
• Assets are content addressed. The id is a SHA-256 hash of the original file. Importing the same loop twice stores it once, and an upload can skip files the server already has.
• Ids, not indexes. Every track, clip and note has a stable id, so undo commands and collaborators can point at them even after reordering.

The sizes explain where each piece lives. The document is small enough to keep whole in memory and save on every edit. The audio is too big for memory, so it stays on disk and is decoded only near the playhead.

Drawing waveforms on a long timeline

A clip on an audio track shows its waveform, so the user can see where the drum hits and the words are. The naive version loops over the decoded samples on every frame and draws a line through them. A 5 minute stereo clip at 48 kHz has 14.4 million samples per channel.

Zoomed out to fit a 1,200 pixel timeline, each pixel covers 12,000 samples, and reading all of them on every scroll frame costs tens of milliseconds per track and blocks the UI. It also needs the whole clip decoded in memory, 115 MB, just to draw a picture.

What a pixel column actually shows is the loudest and quietest sample in its time range: a vertical line from the minimum to the maximum. So we precompute peaks: split the audio into buckets of N samples and keep only the min and max of each bucket. Drawing then reads one pair per pixel, not thousands of samples.

For that 5 minute stereo clip, the finest level, at 256 samples per bucket, has 14,400,000 / 256 = 56,250 buckets per channel. Storing min and max as one signed byte each gives 56,250 x 2 channels x 2 bytes = 225 KB. The coarser levels add a third, so the pyramid is about 300 KB, against 115 MB of decoded float audio: about 0.3 percent. One signed byte per value gives 256 heights, more than a track row has pixels, so Int8Array is enough for display.

Rendering and virtualizing the timeline

• Map time to pixels with one number. The view keeps secondsPerPixel and scrollSec. A tick converts to seconds with the tempo map, then to x with (sec - scrollSec) / secondsPerPixel.
• Virtualize in both directions. Only track rows in the viewport are drawn, as in the virtualized list guide. Inside a row, only clips that overlap the visible time range are drawn, found with a binary search over clips sorted by start.
• Canvas per visible row. Each row is a canvas the width of the viewport. A redraw costs about one min and max line per pixel per row: 1,200 x 16 = about 19,200 lines, which takes a millisecond or two at most on a laptop (an estimate). Clip names, handles and selection outlines can be DOM on top, for hit testing and accessibility.
• Zoom around the pointer. When the user zooms with Ctrl and the wheel, keep the time under the pointer fixed: scrollSec = timeAtPointer - pointerX * newSecondsPerPixel. Clamp zoom between about 10 samples per pixel and the whole song on one screen.
• Keep the playhead separate. The playhead moves 60 times a second; draw it on its own layer (a thin absolutely positioned element moved with transform) so the waveform canvases do not repaint every frame.

Storing, uploading and exporting audio

Audio is the heaviest data most web apps ever hold. One minute of stereo audio at 48 kHz is 23 MB as 32-bit floats, 17 MB as 24-bit WAV, 11.5 MB as 16-bit WAV, about 5 to 6 MB as FLAC (lossless, an estimate that depends on the music) and under 1 MB as Opus at 128 kbps. Where each form lives decides memory use, load time and upload cost.

Local storage: OPFS, IndexedDB and quotas

• Project JSON and the edit queue go in IndexedDB, as the indexeddb lesson shows: small records, transactions, indexes by project.
• Audio assets and takes go in the Origin Private File System (OPFS), reached through navigator.storage.getDirectory(). In a worker, createSyncAccessHandle() gives fast, in-place writes, which suits appending recording chunks. IndexedDB Blobs also work and are the fallback.
• Quota is large but not unlimited. It varies by browser and free disk space. Read navigator.storage.estimate() before a long recording and warn when less than, say, 10 minutes of recording space is left.
• Ask for persistence. Call navigator.storage.persist() after the user saves their first project, so the browser does not evict the data under storage pressure. Browsers decide differently whether to grant it; treat the cloud copy as the real backup either way.

Uploading stems

A take is saved locally first, so the user never waits for the network to keep recording. In the background, an upload queue sends each new asset to object storage with a resumable, chunked upload, as the file-uploads lesson and the file uploader guide describe. Before uploading, the client asks the server whether the content hash already exists; a sample from the built-in library or a re-imported file skips the upload entirely. The project JSON can refer to an asset id before its upload finishes; collaborators see a 'still uploading' clip until it does.

Export: rendering faster than real time

Exporting by playing the song and recording the output takes as long as the song and records any glitch. Instead, use an OfflineAudioContext: the same graph API, but it renders into an AudioBuffer as fast as the CPU allows instead of to the speakers.

The export builds a fresh graph from the project, schedules every clip, note and automation point up front (there is no clock to race, so no look-ahead is needed), and calls startRendering(). Load every AudioWorklet module into the offline context too, since modules belong to one context.

async function exportMix(project: Project, assets: AssetCache) {
  const seconds = projectLengthSeconds(project) + 2;   // reverb tail
  const ctx = new OfflineAudioContext({
    numberOfChannels: 2, sampleRate: 48000, length: Math.ceil(seconds * 48000),
  });
  await ctx.audioWorklet.addModule("/worklets/effects.js");
  buildGraph(ctx, project);                // the same code the live engine uses
  scheduleEverything(ctx, project, assets);
  const mix = await ctx.startRendering();  // AudioBuffer, faster than real time
  return encodeInWorker(mix, { format: "wav", bitDepth: 24 });
}

Encoding happens in a worker. WAV is easy in plain JavaScript: a 44-byte header and the samples converted to 16 or 24-bit integers. MP3 needs an encoder compiled to WebAssembly, since browsers do not ship one; Opus can use the WebCodecs AudioEncoder where the browser supports it.

Watch memory: a 4 minute stereo render is a 92 MB AudioBuffer before encoding, because an offline context allocates its whole output up front. For very long projects, render a minute or two at a time in separate offline contexts and stream the encoded chunks into a file. Start each segment a few seconds early so reverbs and compressors settle, and drop that overlap.

Architecture, performance, collaboration and mobile

Now put the pieces together. The organising rule: the project store is the only source of truth, and everything else is a view of it. The React UI renders it, the audio engine plays it, the sync layer saves it. Each of those runs at its own pace and on its own thread where it can.

Why the boundary between the store and the engine matters: the engine must never read React state, and React must never hold audio nodes. If a fader drag re-rendered the whole track list, or the engine waited on a render, we would bring main thread jank straight back into the sound. The engine subscribes to the store, receives small change records such as 'clip c7 moved' or 'track t3 volume -4 dB', and applies them to its own graph.

Fast-moving values follow the opposite direction without the store. Meter levels arrive from worklets about 20 to 60 times a second and are drawn straight to a canvas in a requestAnimationFrame loop. A fader being dragged updates its AudioParam at once and commits one command to the store when the drag ends, so undo gets one step, not 200.

Interfaces

interface AudioEngine {
  unlock(): Promise<void>;                 // ctx.resume() from a user gesture
  load(project: Project): Promise<void>;   // build graph, decode near clips
  apply(changes: ProjectChange[]): void;   // from the store, after each command
  play(fromTick: Tick): void;
  stop(): void;
  setLoop(range: { start: Tick; end: Tick } | null): void;
  armRecord(trackId: string, deviceId: string): Promise<void>;
  position(): { tick: Tick; seconds: number };   // read in rAF; latency corrected
  meters: EventTarget;                     // "levels" events, about 30 per second
  freeze(trackId: string): Promise<string>;      // returns the new assetId
}

GET  /projects/:id                -> { project, version, assets: [{ id, url, peaksUrl }] }
POST /projects/:id/ops             { baseVersion, ops: [...] } -> { version } | 409 { missing ops }
WS   /projects/:id/live            server -> { type: "ops", version, ops } | { type: "presence", ... }
HEAD /assets/:sha256               -> 200 exists | 404 upload needed
POST /assets                       { sha256, bytes, mime } -> { uploadUrl, chunkSize }
PUT  {uploadUrl}  Content-Range: bytes 0-8388607/52428800   (resumable chunks)

The audio CPU budget and freezing tracks

Every node in the live graph costs time on the audio thread, every 2.67 ms. Native gains and panners are almost free. A ConvolverNode reverb with a 3 second impulse response is one of the most expensive nodes; a dozen of them can use a laptop's whole budget (an estimate; it depends on the machine). So reverbs live on a shared bus that tracks send to, not as an insert on every track.

Browsers do not offer a standard audio CPU meter, so estimate it yourself. On load, render 5 seconds of the project in an OfflineAudioContext and time it: if 5 seconds of music take 2 seconds to render, the graph needs about 40% of real time on this machine. It is a rough proxy, since the offline render does not run on the real-time audio thread, so show it as a CPU meter and warn early, above about 70%. The runtime-performance lesson covers keeping the main thread free, which protects the scheduler.

When the budget runs out, freeze a track: render it with its instrument and effects through an OfflineAudioContext into a new asset, then play that one buffer instead of the whole chain. A synth with 4 effects becomes one source node. The project keeps the original clips and sets frozenAssetId; unfreezing throws the render away. Desktop DAWs such as Ableton Live and Logic Pro offer the same freeze feature.

Collaboration, and why not live jamming

Collaborative editing works like the Google Docs and Figma guides: each command becomes a small operation ('move clip c7 to tick 15,360') sent to the server with the version it was based on, and remote operations are applied to the local store and then to the engine.

Most edits touch different clips, so a per-object last-writer-wins rule resolves the rare conflicts. A CRDT library (a data type that merges concurrent edits on its own, as the crdts lesson explains) is the heavier alternative. Audio assets are uploaded once and referenced by hash, so a collaborator downloads a take only when they open the project.

Playing music together live over the internet is a different problem. In clapping experiments at Stanford, pairs of players slowed down more and more as the delay between them grew, and about 25 ms is the limit usually quoted for playing comfortably together (sound in air travels about 8 m in that time). Light in optical fibre covers about 200 km per ms, so London to New York, about 5,600 km in a straight line, costs about 28 ms one way before any router, jitter buffer or audio buffer.

Add 10 to 30 ms of audio input and output latency at each end, and the total is several times over the limit. That is why browser studios like these sync edits, not live sound.

Accessibility

• Transport by keyboard. Space plays and stops, R records, L toggles the loop, Home returns to the start. Shortcuts are listed in a help dialog and do not fire while focus is in a text field, as the keyboard-navigation lesson explains.
• Faders and knobs are sliders. Each uses role="slider" (or a native range input) with aria-valuenow, aria-valuemin, aria-valuemax, and aria-valuetext such as "-6 decibels" or "pan 30 percent left", because a raw gain of 0.5 means nothing to a listener. Arrow keys change by 0.5 dB, Page Up and Page Down by 6 dB. See the aria-patterns lesson.
• The track list is real DOM. Track headers are a list of groups, each with a labelled name, mute, solo and arm buttons using aria-pressed.
• A text alternative for the canvas. The timeline canvas is invisible to screen readers, so offer a clip list view: a table of clips per track with start bar, length and name, where clips can be moved by editing numbers.
• Announce state, not levels. A polite live region says "Recording, track 3" or "Input clipping"; meters are not announced. Respect reduced motion for decorative animation, but the playhead stays, since it carries meaning.

Mobile limits

Phones are much tighter. Memory per tab is smaller, so the decoded audio budget must drop to a few hundred MB or less. And there is no Web MIDI on iOS at all.

On iOS, Web Audio is muted by the ring and silent switch by default. Recent Safari versions add the Audio Session API: setting navigator.audioSession.type = "playback" before playing marks the page as a music app and lets sound play with the switch on. The API is still Safari only and behaviour has changed between releases, so feature detect it and test on real devices. Safari can also put the context into an interrupted state during a phone call, so listen for statechange and show a tap-to-resume button.

Running it in the interview

This question is huge, and the trap is spending 20 minutes on the mixer UI. Interviewers want to see that you know where the real-time parts are. A 60 minute plan:

1. Minutes 0 to 7: scope. Ask about audio versus MIDI, recording, project size, collaboration and target browsers. Say early that live jamming is out of scope and why, in one sentence.
2. Minutes 7 to 12: the two clocks. Name the core difficulty before drawing boxes: the main thread is unreliable, the audio clock is exact, and the audio thread has a 2.7 ms deadline.
3. Minutes 12 to 25: architecture. Draw the project store, UI, audio engine, worklets, media worker, asset cache and sync layer. Then the audio graph: sources, channel strips, buses, master.
4. Minutes 25 to 40: two deep dives. The look-ahead scheduler with tick-to-seconds conversion, and recording with latency compensation. Let the interviewer pick if they have a preference.
5. Minutes 40 to 52: data and storage. The project JSON with clips pointing at assets, peaks for waveforms, OPFS and uploads, export with an OfflineAudioContext.
6. Minutes 52 to 60: budgets and wrap-up. CPU budget and freezing, accessibility of faders, mobile limits, and what you would build first.

Say the numbers out loud. '128 frames at 48 kHz is 2.67 ms', '25 ms timer, 100 ms look-ahead', '23 MB per stereo minute decoded', '256 samples per peak bucket'. Numbers show you have reasoned about the system rather than recited a component list.

Follow-up questions to expect