Design a log viewer for millions of lines

Requirements: what to ask before drawing anything

A log viewer shows lines of text that a program printed, in the order it printed them, usually with a timestamp and a level such as INFO or ERROR. It looks like the simplest UI there is: a scrolling box of monospace text.

It becomes a hard problem because of size and speed. One CI build can print 2 million lines, and one busy service can print thousands of lines per second. People open these panes when something is broken and they are in a hurry, so a frozen tab is the worst possible moment to fail.

This is a component question. You are designing the pane that two products embed: a CI product (one finished or running build log, a single file) and an observability product (logs from many services, queried by time range). The design has to serve both, so the first job is to find out how they differ.

Two related guides cover the general parts, and this one builds on them instead of repeating them. The virtualized-list guide covers windowing math, rows of unknown height, and browser height limits. The live-comments guide covers turning a fast stream into calm renders and a live region that doesn't flood a screen reader. This guide spends its time on what is specific to logs: the data volume, text search over lines that were never drawn, and the tools around the lines.

Questions that change the design

Functional requirements

• Open a log of up to 2 million lines and scroll anywhere in it with a normal scrollbar.
• Follow a live stream: new lines appear at the bottom and the view stays pinned there until the reader scrolls up.
• Search by substring or regular expression, see the match count, and step to the next and previous match.
• Filter by level, service, and time range, and combine filters with search.
• Jump to a timestamp or a line number, link to a line or a range (#L120-L180), and select lines to share.
• Show ANSI colors from build tools, pretty-print a JSON line on expand, and toggle timestamps between local time and UTC.
• Copy selected lines as plain text and download the full log.

Non-functional requirements, with numbers

• First lines on screen within 1 second of opening a build log on a fast connection, before the whole file has arrived.
• Scrolling at 60 frames per second on a 4-core laptop: under 8 ms of script per frame, so style, layout, and paint fit in the rest of the 16.7 ms frame.
• Live tail of 2,000 lines per second with no frame over 50 ms, the threshold Chrome uses for long tasks and long animation frames.
• Memory: stay under about 300 MB of JavaScript heap for the pane in the worst case, an estimate that keeps a typical laptop tab alive. Past that, evict old lines and say so.
• Search over 2 million lines in under 1 second for a substring, with the first counts visible in under 100 ms.
• Security: log text is untrusted. A line that contains HTML or escape codes must never run script or inject markup.
• Accessibility: every action by keyboard, a screen reader hears match counts and new errors, and the live region never floods it.

Out of scope

• The log pipeline itself: collection, indexing, and retention on the server. We design the API we need from it, not the storage engine.
• Charts of log volume over time (a histogram above the pane is common; the analytics-dashboard guide covers charts).
• Alerting and saved queries. They live in the product around the pane.

What makes a log viewer hard

A list of 100,000 products and a log of 2 million lines both need virtualization, but they are hard in different ways. A product list is short on text and long on images. A log is nothing but text, a lot of it, arriving fast, and the reader treats it as a document: they search it, select it, link to it, and copy it. Five traits drive every later decision.

1. Huge and flat. A 2 million line log is about 250 MB of text if lines average 125 bytes. That is far too many lines for the DOM, and even too many to keep as 2 million separate JavaScript objects. Where the bytes live is the first design decision, not a detail.
2. Live and fast. A running build or a service tail appends lines while the reader looks at them. At 2,000 lines per second, anything you do per line (a state update, a parse, a DOM node) is done 2,000 times a second. You must batch.
3. Read as a document. People expect Ctrl+F to find text, a drag to select lines, Ctrl+C to copy them, and a link to point at line 1,234. Virtualization removes the lines from the page, so the browser's own find, select, and copy only see the 40 or so rows that exist. Each of those features has to be rebuilt in our code.
4. Untrusted text. A log line is whatever a program printed. It can contain <script>, ANSI escape codes, 50,000 character JSON, or a regex-hostile string. Rendering it, parsing it, and searching it must all be safe against hostile input.
5. Read under pressure. People open logs during an incident. A pane that freezes for 3 seconds when they type a regex, or jumps away from the line they were reading when new lines arrive, costs real time when it matters most.

The good news: logs have one trait that makes them easier than a general list. Every line is one row of monospace text. If we choose not to wrap long lines, every row has the same height, and the whole windowing problem from the virtualized-list guide (measuring, estimating, correcting offsets) shrinks to one division: the first visible line is Math.floor(scrollTop / lineHeight). The rendering chapter shows how to keep that simplicity when the reader turns wrapping on.

The numbers to keep in your head

Two of those rows deserve a second look. 33 lines per frame means the tail must never do work per line on the main thread: one update per frame, carrying all 33. And 40 million pixels is past what browsers allow for one element's height: depending on engine and version, the limit sits somewhere between about 16 and 34 million pixels, so a naive spacer div silently stops growing. The rendering chapter handles that with scaled scrolling.

Where the lines live: a compact buffer, not an array of objects

Before rendering, decide what a line is in memory. The naive answer comes straight from a React tutorial: fetch the log, split it on newlines, parse each line into an object like { id, ts, level, text }, and keep an array of them in state. For a 2,000 line log this is perfect. For 2 million lines it fails in three ways at once.

• Memory. Every object carries a header and a pointer to its hidden class (the engine's internal description of the object's shape), every string carries its own header, and a millisecond timestamp is too big for V8's small-integer format, so it usually lives in a separate boxed heap number. A rough estimate is 50 to 100 bytes per line on top of the text, more for every extra parsed field, so 2 million lines cost about 400 to 500 MB of heap. On a 4 GB laptop, a tab that size is at real risk of being killed.
• Garbage collection. 2 million live objects give every major garbage collection 2 million objects to mark. V8 does much of that marking on background threads, but the main thread still pays for some of it, and pauses long enough to drop frames can show up while the reader scrolls.
• Parse time. text.split("\n") on 250 MB, then a JSON.parse or regex per line, is several seconds of main thread work. Nothing paints until it ends.

The fix is to stop thinking of a line as an object. A log is a stream of bytes with newlines in it. Keep the bytes as bytes, and keep a separate index of where each line starts. A line becomes a number (its index), and we only turn it into a string when it is about to be drawn or copied.

How the line store works

Bytes arrive from the network in pieces that end anywhere, often in the middle of a line or even in the middle of a multi-byte UTF-8 character. The store keeps the unfinished tail of each piece as a carry-over and prepends it to the next piece, so every stored chunk ends exactly on a newline. Each line then lives inside one chunk, and decoding a line never has to stitch two chunks together.

const NEWLINE = 10;
class LineStore {
  chunks: Uint8Array[] = [];
  chunkFirstLine: number[] = []; // line number that starts each chunk
  starts = new Uint32Array(1 << 16); // byte offset of each line inside its chunk
  count = 0; // lines stored
  firstLine = 0; // absolute number of the oldest kept line (grows on eviction)
  private decoder = new TextDecoder();

  appendChunk(bytes: Uint8Array) {   // bytes end on a newline
    this.chunkFirstLine.push(this.firstLine + this.count);
    this.chunks.push(bytes);
    for (let i = 0, start = 0; i < bytes.length; i++) {
      if (bytes[i] !== NEWLINE) continue;
      if (this.count === this.starts.length) this.grow();
      this.starts[this.count++] = start;
      start = i + 1;
    }
  }

  line(n: number): string {          // n is an absolute line number
    const c = upperBound(this.chunkFirstLine, n) - 1; // binary search: the chunk holding n
    const chunk = this.chunks[c];
    const i = n - this.firstLine;      // index into the columns
    const nextChunk = this.chunkFirstLine[c + 1] ?? this.firstLine + this.count;
    const end = n + 1 < nextChunk ? this.starts[i + 1] - 1 : chunk.length - 1; // drop the newline
    return this.decoder.decode(chunk.subarray(this.starts[i], end));
  }
  // grow() doubles the typed arrays; eviction is covered in the tail chapter.
}

Line numbers are absolute: line 1,234 is always line 1,234, even after the oldest lines are evicted. firstLine tracks how many have been dropped. That keeps deep links, selections, and search results valid across eviction, which the tail chapter relies on.

The memory budget, as estimates

Even the compact layout spends about 280 MB, so the store needs a cap. We cap at 256 MB of bytes on desktop and 64 MB when navigator.deviceMemory reports 4 GB or less (that property only exists in Chromium browsers, so default to the lower cap when it is missing on mobile). Past the cap, a live tail evicts its oldest chunk, and a finished build log switches to paging: it keeps the chunks around the reader and fetches others again with HTTP Range requests when needed.

The live tail: streaming, batching, and following

A running build or a kubectl logs -f style tail adds lines while the reader watches. The naive version opens a socket and, for every message, calls setLines([...lines, line]). At 2,000 lines per second that is 2,000 renders per second, each copying an array that keeps growing, and the tab locks up within a minute. The fixes are about where the stream lands, how often the UI hears about it, and how the view decides whether to follow.

Getting the bytes: which transport

Batching: one update per frame, not one per line

The worker appends bytes to the store as they arrive; that is cheap and off the main thread. What must be rationed is how often the main thread hears about it. The worker posts a small message after each append: the new total line count and, if the view is following, the decoded text of the last 100 lines so the bottom can render without a round trip.

The main thread does not render on each message. It records the latest one and schedules a single requestAnimationFrame callback, which commits whatever arrived since the last frame in one state update.

let pending: TailUpdate | null = null;
let frame = 0;

worker.onmessage = (event: MessageEvent<TailUpdate>) => {
  // Keep only the newest update; it already includes everything before it.
  pending = event.data;
  if (!frame) frame = requestAnimationFrame(flush);
};

function flush() {
  frame = 0;
  if (!pending) return;
  const update = pending;
  pending = null;
  viewStore.setTotal(update.total, update.tail); // one state change, one render
  // While following, the viewport's layout effect sets scrollTop to total * lineHeight.
}

At 2,000 lines per second this turns about 2,000 events into 60 renders, each handling about 33 lines. When the tab is in the background, requestAnimationFrame stops firing, so no renders happen at all. The worker keeps appending, and the first frame after the reader returns jumps to the new total. The live-comments guide uses the same idea with a slower 150 ms tick, because a chat row costs more to render than a line of text.

Following, pausing, and the new lines button

Auto-scroll is the feature people notice when it is wrong. If the view keeps jumping to the bottom while someone is reading an error 300 lines up, the pane is unusable. The rule, the same one the live-comments and llm-chat-ui guides use: follow only while the reader is at the bottom. The moment they scroll up, stop following and count what they are missing.

• Following: after each flush, set scrollTop to the bottom. The view is "at the bottom" if it is within 1 line height (20 px) of it, which absorbs rounding on zoomed or high-DPI screens.
• Paused: entered when a scroll event finds the view more than 20 px from the bottom and the scroll was not ours. We tell them apart by remembering the scrollTop we set; any other value came from the reader (wheel, keyboard, scrollbar drag, or find-and-jump).
• The button: while paused, show "312 new lines" pinned above the bottom edge. Clicking it, pressing End, or scrolling back to the bottom resumes following.
• Never pause on our own changes: eviction and a growing scroll height also move things. Only a reader-initiated scroll may change the follow state.

The cap: evicting the oldest lines

A service tail never ends. Once the store reaches its cap, the worker drops the oldest chunk (about 8,000 lines per 1 MB chunk) and raises firstLine. The columns must not be copied on every eviction, so each typed array works as a ring buffer: a fixed-size array whose start index moves forward, so dropping the oldest 8,000 entries is one addition, not a copy of 2 million.

Two things must then happen in the UI. First, a notice at the top of the pane says what was lost and how to get it back: "Showing the latest 2,000,000 lines. 240,000 older lines were dropped. Download the full log." Second, if the reader is paused in the middle, the lines above them just vanished, so the content would jump up by the dropped height. Subtract dropped * lineHeight from scrollTop in the same frame that applies the new total, and the lines under their eyes stay put.

One case needs a policy: the reader is paused on the very chunk that is next to go. Evicting it would delete what they are reading. We allow a soft overflow of 25 percent past the cap while a paused reader sits in the oldest chunk. If the hard limit is reached anyway, we evict, keep their line number, and show "The lines you were reading were dropped. Reload them", which fetches that range from the server.

Rendering: fixed heights, long lines, and what is inside a line

The fixed line height from the hard-parts chapter does most of the work here, so the general windowing machinery in the virtualized-list guide (measuring, estimating, correcting) isn't needed. With 20 px rows, the rows to draw run from the first visible line minus 20 rows of overscan to that line plus 40 visible rows plus 20 more. The visible block sits inside one tall spacer and moves with a single transform: translateY(...), as the performance chapter explains.

A row has four parts, in fixed-width columns so they line up: a line number gutter (a link, for deep links), the timestamp, a level badge, and the message. The message uses white-space: pre so spaces and alignment from the program survive. Tabs are expanded to spaces when the line is decoded, so a tab is always the same width and character counts stay meaningful.

When the log is taller than the browser allows

2 million lines at 20 px is 40 million pixels, past every browser's maximum element height, so a spacer that tall is silently clipped and the last lines become unreachable. The virtualized-list guide measures those limits and compares three fixes; a log viewer takes its scaled scroll space. Cap the spacer at 10 million pixels, safely under the smallest limit, and map scroll positions proportionally, so one pixel of scrollbar is about 0.2 lines.

The cost is that the browser's own wheel and arrow-key scrolling now moves four times as far, so the pane applies those deltas 1:1 in script, exactly as that guide recommends. Dragging the scrollbar still reaches any line, and jumps land exactly. Below the cap, the scale is exactly 1 and nothing changes.

Long lines: wrap, scroll, or truncate

Some lines are huge: a minified stack trace, a 50,000 character JSON payload, a progress bar printed with carriage returns. The choice here decides whether the fixed-height model survives.

The arithmetic only matches the browser if the browser also wraps at the exact column. Normal wrapping breaks at spaces, and overflow-wrap: anywhere breaks inside a word only when the word alone can't fit, so rows end at different columns. line-break: anywhere allows a break after any character, and white-space: break-spaces stops trailing spaces from hanging past the edge, so every full row holds exactly cols characters; a test that compares computed and measured rows on sample lines keeps that promise honest.

With wrapping on, the line at a given pixel offset is found by binary search over the prefix sums (rowsBefore[n]). The worker keeps them in a Uint32Array and ships the slice for the visible region to the main thread.

Inside a line: ANSI colors, JSON, and timestamps

Build tools color their output with ANSI escape codes: invisible byte sequences such as ESC[31m (switch to red) and ESC[0m (reset) that terminals interpret. Shown raw, a line reads [31merror[0m: test failed; shown well, "error" is red. Service logs, on the other hand, are often JSON objects on one line, and every line has a timestamp that different readers want in different zones. All three turn stored bytes into what the row shows, and all three must be cheap, because they run for every row that scrolls into view.

The codes that matter for logs are SGR codes (Select Graphic Rendition, the ones ending in m), with parameters separated by semicolons. 1 is bold and 0 resets. 30 to 37 pick one of 8 basic foreground colors and 90 to 97 their bright versions, while 38;5;n picks one of 256 palette colors and 38;2;r;g;b a 24-bit color. The 40s, 100s, and 48 do the same for the background.

ANSI colors without HTML injection

The tempting shortcut is an npm package that converts ANSI to an HTML string, then dangerouslySetInnerHTML. Now a log line containing <img src=x onerror=...> runs script in your product's origin, with the reader's session. Log text is attacker-controlled: anyone who can make your service print a string (a username, a URL path, a header) can write into the log. The xss lesson covers the general rule; here it means the parser must output data, never markup.

type Token = { text: string; fg?: string; bold?: boolean }; // fg: a theme class or "rgb(...)"
const CSI = /\x1b\[([0-9;?]*)([@-~])/g; // ESC [ parameters, then one final byte

function parseAnsi(line: string): Token[] {
  const tokens: Token[] = [];
  let style: Omit<Token, "text"> = {}; // styles reset at each line start
  let last = 0;
  for (const m of line.matchAll(CSI)) {
    if (m.index > last) tokens.push({ text: line.slice(last, m.index), ...style });
    last = m.index + m[0].length;
    if (m[2] !== "m") continue; // cursor moves, screen clears: drop them
    const p = (m[1] || "0").split(";").map(Number);
    for (let i = 0; i < p.length; i++) {
      const c = p[i];
      if (c === 0) style = {};
      else if (c === 1) style = { ...style, bold: true };
      else if (c === 22) style = { ...style, bold: false };
      else if (c >= 30 && c <= 37) style = { ...style, fg: BASIC[c - 30] };      // "red", ...
      else if (c >= 90 && c <= 97) style = { ...style, fg: BRIGHT[c - 90] };     // "bright-red", ...
      else if (c === 39) style = { ...style, fg: undefined };                     // default color
      else if (c === 38 && p[i + 1] === 5) { style = { ...style, fg: palette(p[i + 2]) }; i += 2; }
      else if (c === 38 && p[i + 1] === 2) { style = { ...style, fg: rgb(p[i + 2], p[i + 3], p[i + 4]) }; i += 4; }
      // 40 to 47, 100 to 107, and 48 set the background the same way.
    }
  }
  if (last < line.length) tokens.push({ text: line.slice(last), ...style });
  return tokens;
}

Two details matter for logs. First, carriage returns: progress bars print \r and redraw the same line many times. A terminal shows only the last redraw, so the decoder first drops the \r of a Windows \r\n line ending, then keeps only the text after the last remaining \r.

Second, state across lines: in a terminal, a color set on one line stays on until reset. Carrying it over would mean parsing from the start of the log to draw line 1,500,000. We reset at every line start instead. If a product needs carry-over, the worker can record the color state at the start of each chunk, so any line is at most one chunk of parsing away.

JSON lines: pretty-print on expand

A structured log line like {"ts":"...","level":"error","service":"checkout","msg":"card declined","order":{...}} is unreadable as one 600 character line. The row shows the extracted fields in their columns and the msg as the message; the worker pulls msg out when it decodes a window, which costs microseconds per line. A chevron expands the row into a pretty-printed, collapsible tree. That tree is built only on expand, never during scroll, and for lines over 100 KB the parse runs in the worker so a giant payload can't block a frame.

An expanded row breaks the fixed height for that one line. Rather than give up the simple math, the view keeps a small sorted list of expanded lines and their extra heights. The offset of line n becomes n * 20 plus the extra heights of expanded lines above it. With a handful of expanded rows, a linear pass over that list is cheaper than any tree.

Timestamps: local time, UTC, or relative

The worker parses each line's timestamp once, at ingest, into milliseconds since the epoch, stored in a Float64Array. The row formats it when drawn, with one cached Intl.DateTimeFormat per mode. Toggling between local time and UTC changes the formatter (timeZone: "UTC"), not the data, so it is instant even for 2 million lines.

Show milliseconds (fractionalSecondDigits: 3), because many lines share a second. Label the column with the zone ("UTC" or "UTC+05:30"), since a timestamp without a zone is the source of many wrong incident timelines. For build logs, a third mode shows time since the step started (+00:01:23.456), which answers "what was slow" faster than wall-clock time.

Search and filters: finding text that was never rendered

The reader presses Ctrl+F and types timeout. The browser's find bar searches the DOM, and the DOM holds about 80 lines, so it reports "0 of 0" while the log contains 312 timeouts. This is a common complaint about virtualized logs, and the pdf-viewer guide meets the same wall with pages that aren't rendered. The only fix is to search our own data and build the find experience ourselves.

Taking over Ctrl+F, carefully

If our search box exists but Ctrl+F still opens the browser's bar, readers get the useless "0 of 0" first. So the pane intercepts Ctrl+F (Cmd+F on macOS) and focuses its own search box, with three limits. First, only when focus is inside the pane, or the log is the whole page (a dedicated build log route); where the log is one widget among others, the rest of the page keeps native find.

Second, pressing the shortcut again while our box is already focused falls through to the browser, so the native bar is always one more keypress away. Third, the box's placeholder says what it searches: "Search 2,000,000 lines".

How the scan works

Decoding a 1 MB chunk to a string with TextDecoder takes about a millisecond or less, and String.prototype.indexOf is heavily optimized in every engine. So for each chunk the worker decodes it once, finds every match with indexOf (on lowercased copies for case-insensitive search), and maps each match position to a line number by walking forward through the newlines in step with the matches. A substring scan of 250 MB this way takes a few hundred milliseconds on a recent laptop (an estimate; measure on the slowest machine you support), inside our 1 second target.

• Streaming counts: every 50 ms the worker posts { searchId, scannedLines, matches }, so the box shows "1,204 matches, 40% scanned" almost at once and the count climbs.
• Cancellation: each keystroke (debounced by 150 ms) starts a search with a new searchId. A loop in a worker can't receive messages until it yields, so the scan yields to the event loop after every 8 chunks; on resuming it checks whether a newer search arrived and stops if so. Results with an old searchId are ignored on the main thread too.
• Results: a Uint32Array of matching line numbers, sorted because the scan runs in order, transferred to the main thread without a copy. 50,000 matches is 200 KB.
• Live tail: new chunks are scanned as they arrive and their matches appended, so the count stays true while the log grows.

Next, previous, and scrolling a match into view

The current match is an index into the sorted results. Next and previous move that index and scroll the line to the middle of the viewport (scrollTop = line * 20 - viewportHeight / 2, through the scale factor if scaled scrolling is on), so there is context above and below.

If the reader scrolled away since the last jump, Enter goes to the first match after the top visible line, found by binary search over the results, not to the old index. That small rule is what makes search feel like it follows you. Navigating also pauses follow mode, since it is a reader-initiated scroll.

Highlighting happens only for the rows on screen. For each of the 80 rendered rows the view needs the match ranges inside that line. For a substring, the main thread finds them itself, since indexOf over 80 short strings costs microseconds. For a regex it asks the worker, because running a user's regex on the main thread is the one thing we promised never to do (the regex section below explains why).

The ranges split the row's ANSI tokens at match boundaries, and each matching piece is wrapped in a mark element, with the current match styled differently. The CSS Custom Highlight API (CSS.highlights with Range objects) is an alternative that styles ranges without splitting any nodes.

Regex search without hanging the tab

Readers want regular expressions: error|fatal, user_id=\d{6}, took (\d{4,})ms. JavaScript's RegExp is a backtracking engine: when part of a pattern fails, it goes back and tries other ways to split the input. For most patterns that is fast, but for some it is exponential.

The pattern (a+)+$ against a line of 30 as followed by a b has about a billion ways to split the as, and the engine tries them before giving up; in a quick test in Node 22 on a laptop, 28 as already took over 3 seconds. This is ReDoS, regular expression denial of service. In a log viewer there doesn't even need to be an attacker: a reader types a sloppy pattern, and one unlucky line in the log does the rest.

Worse, there is no way to interrupt a running exec. The cooperative cancellation above yields between chunks, but a single regex call on a single line never yields. If that runs in the store worker, the store worker is stuck: no tail, no line requests, nothing renders new rows. If it runs on the main thread, the tab is frozen.

The watchdog in detail

const BUDGET_MS = 1000;
let regexWorker = spawnRegexWorker();
let watchdog = 0;

function spawnRegexWorker() {
  const worker = new Worker(new URL("./regex.worker.ts", import.meta.url), { type: "module" });
  worker.onmessage = ({ data }) => {
    if (data.type === "progress") armWatchdog(data.searchId); // re-arm after every chunk
    if (data.type === "done") clearTimeout(watchdog);
  };
  const { port1, port2 } = new MessageChannel(); // chunks flow store worker -> regex worker
  worker.postMessage({ type: "port" }, [port1]);
  storeWorker.postMessage({ type: "regex-port" }, [port2]);
  return worker;
}

function armWatchdog(searchId: number) {
  clearTimeout(watchdog);
  watchdog = window.setTimeout(() => {
    regexWorker.terminate(); // the only way to stop a running exec
    regexWorker = spawnRegexWorker(); // fresh worker, fresh channel
    storeWorker.postMessage({ type: "cancel", searchId }); // stop feeding chunks
    ui.searchError(searchId, "This pattern took too long on one line. Try a simpler one.");
  }, BUDGET_MS);
}

The budget is per chunk, not per search: a complex but correct pattern over 250 chunks may take a few seconds in total and should be allowed to, as long as each chunk finishes within a second. A healthy pattern scans a 1 MB chunk in a few milliseconds, so a full second of silence is a strong signal. Terminating the worker frees its memory, and the fresh worker is ready for the next query. The error names the problem and keeps the matches found so far, so the reader learns something and loses nothing.

The same disposable worker answers highlight requests for the visible rows when the query is a regex, with the same watchdog. And invalid patterns (new RegExp("(") throws a SyntaxError) are caught on the main thread before anything starts, with the engine's message shown under the box.

Filters: an index of visible lines

Search finds lines and leaves the rest in place. Filters hide lines: show only ERROR and WARN, only the checkout service, only the last 15 minutes, or only lines that match the search. The naive version is lines.filter(...) into a new array of objects, re-run on every change. With our compact store there are no objects to filter, and that turns out to be an advantage.

At ingest the worker already extracts a few fields per line into columns: typed arrays with one entry per line. level is a Uint8Array (0 for trace up to 5 for fatal), service is a Uint16Array of ids into a small dictionary of service names (usually a few hundred at most), and ts is the Float64Array of timestamps. Testing a line against a filter is then a few array reads, with no decoding and no strings.

Keeping the reader's place and the tail

• Keep the place: before applying a new filter, note the line number at the top of the viewport. After, binary search the new index for the first visible line at or after it and scroll there. Without this, turning on "errors only" throws the reader to some unrelated spot.
• Combine with search: "Show only matching lines" intersects the filter index with the sorted search results, a linear merge of two sorted arrays.
• Live tail: each appended batch is tested against the active filter and matching line numbers are pushed onto the index. The "N new lines" button counts only lines that pass the filter.
• Eviction: dropping the oldest chunk drops a prefix of the index too, found by binary search for firstLine.
• Counts per value: the filter menu shows "ERROR (1,204)". The worker computes these counts in the same pass, so the reader sees how many lines each choice will show before choosing.
• In the URL: filters live in the query string (?level=error,warn&service=checkout), so a shared link reproduces the view. The url-as-state idea applies directly.

For service logs that page from the server, the same filters are sent as query parameters, and the server returns only matching lines. The client-side index still exists over what has been loaded, so toggling a filter on lines already in memory is instant while the server fetches more.

Jumping: to a time, to a line, and links to share

During an incident the question is rarely "what is on line 1,482,003". It is "what happened at 14:32:05" (because a graph spiked then) or "look at the lines my colleague linked". Both are jumps into a log too big to scroll through by hand.

Jump to a timestamp with binary search

The worker already holds every line's timestamp in a Float64Array. In a single build log, timestamps only go up, so finding the first line at or after 14:32:05 is a binary search: about 21 comparisons for 2 million lines, under a microsecond. No scan, no decoding. The view scrolls that line to the top third of the viewport and briefly highlights it.

function lineAtOrAfter(ts: Float64Array, count: number, target: number): number {
  let lo = 0;
  let hi = count; // answer is in [lo, hi]
  while (lo < hi) {
    const mid = (lo + hi) >>> 1;
    if (ts[mid] < target) lo = mid + 1;
    else hi = mid;
  }
  return lo; // === count means "after the last line"
}

Two real-world wrinkles. Lines without a timestamp (a stack trace's continuation lines) inherit the timestamp of the line before them at ingest, so the array has no holes. Merged service logs can also be slightly out of order, because several machines with different clocks write into one stream; the server sorts each page by time, so out-of-order lines differ only by the clock skew between hosts, usually milliseconds, and binary search still lands close.

If the target is outside what is loaded, the jump becomes a server request for the page around that time (GET /logs?around=...), which the interface section defines.

Deep links to a line or a range

Clicking a line number selects it and puts #L1234 in the URL; Shift-clicking another extends it to #L1234-L1260. GitHub's file view made this format familiar, and the code-review-diff guide uses the same anchors for diff lines. The URL changes with history.replaceState, so selecting ten lines doesn't add ten entries to the back button. The line numbers are the absolute ones from the store, so the link survives filters, eviction, and reloads.

Opening a link is where the stream complicates things. The page loads, the hash says line 1,234, and the store has 0 lines so far. The view shows the first lines as they arrive and a small banner, "Going to line 1,234", and jumps as soon as the store's count passes it. Follow mode stays off, because the reader came for line 1,234, not the bottom.

If the line is beyond what the store will hold, or was evicted, the view requests that range from the server. If the log is shorter than the link (the build was retried and printed less), say so instead of silently landing on the last line.

Selecting, copying, and downloading lines that aren't there

A reader drags from line 100 down to line 900 to copy a failing test's output. With native browser selection and a virtualized list, three things go wrong. As they drag past the bottom, the rows near line 100 scroll out and are removed, and the selection's start point disappears with them, so it collapses or jumps.

Ctrl+A selects the page, which holds only the 80 rendered rows of the log. And Ctrl+C copies only what is selected in the DOM, so even a lucky selection holds at most a screenful. Copy has to be rebuilt on top of our own data, like search.

Making Ctrl+C copy lines that aren't rendered

The copy event lets us replace what goes to the clipboard: call event.preventDefault() and event.clipboardData.setData("text/plain", text). The catch is that setData must happen synchronously, inside the event, and our text lives in the worker, one message away. So whenever the line selection changes, the view asks the worker for its text right away (debounced by 100 ms) and keeps the result ready. When Ctrl+C arrives, the text is already on the main thread.

• What gets copied: the message text with ANSI codes stripped, lines joined by \n. A setting adds timestamps and line numbers, which is what people want when pasting into an incident document.
• A size limit: prefetching is fine up to about 20,000 lines (around 2.5 MB of text). Past that, the copy handler shows "Selection too large to copy. Download it instead" with a button, rather than freezing the tab building a 250 MB string.
• A Copy button: for the toolbar button there is no copy event, so it uses the async Clipboard API, navigator.clipboard.writeText(text), inside the click handler. It needs a secure context (HTTPS), and some browsers also require a user gesture, which a button click provides.
• Select all: with focus in the pane, Ctrl+A selects all lines in the line selection model (not the DOM), and the status bar says "2,000,000 lines selected" so the next step, copy or download, is a choice the reader makes knowingly.

Downloading: from the server, not from the tab

The naive download builds a Blob from everything in memory and clicks a link to it. That doubles memory at the worst time (250 MB of store plus 250 MB of Blob), and it only contains what the tab holds, which after eviction is not the full log.

The full log already exists on the server as a file. The Download button is a plain link to GET /builds/123/logs/raw, answered with Content-Disposition: attachment and gzip compression, so the browser streams it straight to disk without the page touching a byte. Repetitive log text usually compresses several times over, so the transfer is much smaller than the file.

Downloading a filtered view or a selection is the one case for a client-built file, since only the client knows the filter. Under 50 MB, the worker joins the lines and returns a Blob, which becomes an object URL for a temporary link (revoked with URL.revokeObjectURL once the download has started; the memory-leaks lesson explains why). Above that, the same filters are sent to the server's raw endpoint as query parameters.

Architecture, data model, and interfaces

Every decision so far pushed work off the main thread. Put together, the component splits into three places that run code: the main thread draws and handles input, a store worker owns the bytes and everything computed from them, and a disposable regex worker runs patterns we don't trust. The main thread never holds more than the lines near the viewport.

Why these boundaries? The store worker sits where the bytes are biggest: it streams from the network straight into its chunks, so 250 MB never crosses a thread boundary. The main thread asks it for line windows: "lines 48,000 to 50,000, decoded, with tokens and match ranges". A 2,000 line window is about 250 KB of strings and takes a few milliseconds to produce.

The view keeps the current window plus one on each side, and when scrolling nears the edge it requests the next one ahead of time. With a fixed line height, the view knows exactly which lines it will need, so prefetching is accurate and a blank row is rare. When one does appear (a scrollbar drag across a million lines), it shows a dim placeholder for a frame or two, which is far better than a frozen tab.

When the product already has a WebSocket on the main thread, tail bytes arrive there instead. The main thread sets the socket's binaryType to "arraybuffer", doesn't decode anything, and posts each frame's buffer to the store worker as a transferable (worker.postMessage(msg, [buffer])). Transferring moves ownership instead of copying, so a 64 KB batch crosses in microseconds. The main thread's buffer becomes detached (its byteLength drops to 0), which guarantees we don't keep two copies.

Data model: what lives where

The data splits by size and by who needs it. The big, per-line data lives only in the store worker, as typed arrays. The main thread holds small, per-screen data, and anything a shared link must reproduce lives in the URL. Nothing per-line ever lives in React state.

// Store worker (per-line, typed arrays)
type LineStoreState = {
  chunks: Uint8Array[];      // ~1 MB each, end on a newline
  starts: Uint32Array;       // byte offset of each line in its chunk
  ts: Float64Array;          // epoch ms, inherited for continuation lines
  level: Uint8Array;         // 0 trace .. 5 fatal, 255 unknown
  service: Uint16Array;      // id into services[]
  services: string[];        // dictionary, usually under a few hundred
  firstLine: number;         // absolute number of the oldest kept line
  count: number;             // lines kept
  visible: Uint32Array | null; // filter index, null when no filter
};

// Main thread (per-screen)
type LineWindow = { from: number; lines: { n: number; tokens: Token[]; ts: number; level: number }[] };
type ViewState = {
  total: number;             // lines in the current view (filtered or not)
  follow: "following" | "paused";
  unseen: number;            // for the "N new lines" button
  windows: LineWindow[];     // current plus neighbours, about 6,000 lines
  search: { id: number; query: string; regex: boolean; matches: Uint32Array; current: number; scanned: number };
  selection: { anchor: number; focus: number } | null; // absolute line numbers
  expanded: Map<number, number>; // line -> extra height in px
};

Note what is missing: there is no Line object type anywhere. A line is a number. Each structure holds one aspect of all lines (offsets, timestamps, levels), which is called a columnar layout. It is what makes the filter pass fast (it reads two small arrays, not 2 million objects) and what keeps memory near 1 byte per character.

Interface: props, server endpoints, and worker messages

Three interfaces meet in this component: the props the products use, the server API it calls, and the message protocol between the main thread and the store worker. Each should be small and boring.

type LogSource =
  | { kind: "file"; rawUrl: string; streamUrl?: string } // build log
  | { kind: "query"; queryUrl: string; tailUrl?: string; filters: LogFilters }; // service logs

type LogViewerProps = {
  source: LogSource;
  maxBytes?: number;                 // store cap, default 256 MB (64 MB on low-memory devices)
  lineHeight?: number;               // default 20
  initialLine?: number;              // otherwise read from #L1234
  timeZone?: "local" | "utc" | "relative";
  onLineSelect?: (range: { from: number; to: number }) => void;
  renderLineActions?: (line: number) => React.ReactNode; // e.g. "Open trace"
};

For a build log, the server needs only a static file that supports Range requests, plus a sparse line index: the byte offset of every 10,000th line. With it, the client turns "lines 900,000 to 910,000" into one byte range without reading what is before it. The live part uses the same URL with a resume offset.

GET /builds/123/logs/raw
Range: bytes=112000000-113999999
-> 206 Partial Content, Content-Range: bytes 112000000-113999999/251234567

GET /builds/123/logs/index
-> 200 { "every": 10000, "offsets": [0, 1251004, 2499870, ...] }

GET /builds/123/logs/stream?from=251234567
-> 200, body stays open while the build runs; ends when the build ends

GET /builds/123/logs/raw?download=1
-> 200, Content-Disposition: attachment; filename="build-123.log", Content-Encoding: gzip

GET /logs/query?service=checkout&level=error,warn&q=timeout
    &from=2026-03-02T14:00:00Z&to=2026-03-02T15:00:00Z&limit=5000&cursor=eyJ0cyI6...
-> 200 { "lines": [{ "id": "17e9a..", "ts": 1772460725123, "level": "error",
          "service": "checkout", "text": "..." }], "next": "eyJ0cyI6...", "approxTotal": 48210 }

GET /logs/tail?service=checkout&level=error,warn
Accept: text/event-stream
-> id: 17e9b..
   data: {"lines":[...]}            (one event per 100 ms batch)

Paging uses an opaque cursor (the last timestamp plus a tiebreak id), not a page number, because new lines keep arriving and page numbers would shift under the reader. approxTotal is labelled approximate on purpose: an exact count over a large time range is expensive for log backends, and "about 48,000" is enough to size a scrollbar. Errors use one shape, { "error": { "code": "range_not_satisfiable", "message": "..." } }, and the pane shows a retry for transient codes.

type ToWorker =
  | { type: "open"; source: LogSource; maxBytes: number }
  | { type: "window"; reqId: number; from: number; to: number; highlight?: Query }
  | { type: "filter"; reqId: number; filters: LogFilters }
  | { type: "search"; searchId: number; query: Query }
  | { type: "text"; reqId: number; from: number; to: number; withTimestamps: boolean }
  | { type: "append"; bytes: ArrayBuffer }; // transferred, from a main-thread socket

type FromWorker =
  | { type: "total"; total: number; firstLine: number; dropped: number; tail?: LineWindow }
  | { type: "window"; reqId: number; window: LineWindow }
  | { type: "search-progress"; searchId: number; scanned: number; matches: number }
  | { type: "search-done"; searchId: number; matches: Uint32Array } // transferred
  | { type: "text"; reqId: number; text: string }
  | { type: "error"; code: "network" | "too-large" | "decode"; message: string };

Performance and accessibility: a busy main thread and a quiet screen reader

The hardest moment for the main thread is a reader scrolling while the tail runs at full speed and a search is counting. Here is where each millisecond of a 16.7 ms frame goes in that moment, as estimates for a 4-core laptop. The runtime-performance lesson explains how to measure these in the Performance panel.

Avoiding layout thrash

Layout thrash is reading a layout value (like scrollTop or offsetHeight) right after changing the DOM, which forces the browser to compute layout immediately, many times per frame. A log view is at risk because it both reads scroll position and writes rows. The rules that keep it to one layout per frame:

• Read scrollTop once in the scroll handler, store it, and do all window math from that number. With fixed heights, no row ever needs to be measured.
• Measure the character width once, when the font loads (document.fonts.ready), with one probe element. It feeds the wrap math and never changes until the font size does.
• Move the visible block with one transform: translateY(...) and give the scroll container contain: strict, so changes inside it don't invalidate layout outside it.
• Reuse row nodes. Key rows by their slot in the window (0 to 79), not by line number, so scrolling by one line updates 80 text nodes instead of unmounting and mounting rows. Line numbers are still shown and used for links; they just aren't React keys.
• When following, set scrollTop after the render that added lines, in the same frame, and never read scrollHeight in a loop. The total height is known: lines times 20.

Knowing it stays fast

Ship with measurement, because the worst logs only exist in production. Record time to first line, the largest log size opened, store bytes at eviction, search duration by log size, regex watchdog kills, and long frames. In Chromium, PerformanceObserver with the long-animation-frame entry type reports frames over 50 ms with the scripts that caused them, which points straight at the culprit. A dashboard of "p95 search time for logs over 1 million lines" tells you when a change made things worse.

Accessibility: keyboard, screen readers, and a quiet live region

A log viewer is used by engineers who rely on keyboards and screen readers like anyone else, and virtualization takes away much of what assistive technology normally gets for free: the reader can't browse lines that don't exist. The keyboard-navigation and screen-readers lessons cover the general patterns; three things are specific to logs.

The tail: role=log without a flood

ARIA has a role made for this: role="log", a live region where new content is added at the end, with an implicit aria-live="polite". The live-comments guide shows why that fails on a fast stream: the speech queue floods, and at 2,000 lines per second nothing else can be heard. We use its answer. The rows container keeps role="log" with an aria-label (the role requires an accessible name) but sets aria-live="off", so appended lines are never read automatically.

A separate, visually hidden status element announces summaries instead, at most every few seconds: "12 new error lines" while paused, or nothing at all for info lines. The announcement policy is configurable, with errors only by default, because errors are what someone watching a tail is waiting for.

Keyboard: lines, matches, and selection

Focus uses one tab stop for the whole line area (a roving focus, not 2 million tab stops). The focused line is a number in ViewState, and the rendered row for it gets tabIndex={0} and real focus. When it scrolls out and its node is reused for another line, focus moves to the container and returns to the row when it renders again, so the screen reader doesn't land on the wrong line.

Each row is labelled for screen readers as "Line 48,311, 14:32:05.123, error, card declined". The number and the level go first, because they are what the listener uses to orient.

Announcing search

The search box gets aria-describedby pointing at its count, and the same polite status region announces the result once the scan settles ("312 matches"), not on every progress update. Moving between matches announces "Match 4 of 312, line 48,311" followed by the line's text. Errors from the regex watchdog are announced with the same polite region, since they follow the reader's own action.

Running it in the interview

In 60 minutes you can't cover every chapter in depth, and you shouldn't try. The interviewer is looking for whether you see that this is a data problem before it is a rendering problem, and whether you know what virtualization breaks. A good plan for the time:

Draw the wireframe first, because it names the features (toolbar, gutter, new lines button, dropped lines notice) you will refer to for the rest of the hour. Then draw the three-thread architecture before any detail. Most weak answers spend 20 minutes on windowing math; say that fixed heights make it one division, name the virtualization pieces you would reuse, and move on to what is specific to logs.

Follow-up questions to expect

• "What if the log is 10 GB?" The client never holds it. The store becomes a sliding window over server ranges, using the sparse line index; search moves to the server or to a background scan that streams through ranges without keeping them. The UI doesn't change, which is the point of the line-number abstraction.
• "Why not SharedArrayBuffer so both threads can read the bytes?" It would let the main thread decode lines without a message. But it requires cross-origin isolation (Cross-Origin-Opener-Policy and Cross-Origin-Embedder-Policy headers), which breaks many third-party embeds in a large product. Line windows with prefetch are fast enough, so we don't pay that cost.
• "How do you test this?" A synthetic log generator (2 million lines, ANSI codes, long lines, JSON, a catastrophic regex input) in unit tests for the store and parser, and a performance test in CI that fails if a 2 million line search takes over 1 second or any frame during a simulated tail exceeds 50 ms.
• "Multiple tabs open on the same tail?" Each tab has its own store and stream; that is acceptable for a log pane. A SharedWorker could share one stream across tabs, but support on mobile browsers is recent and still incomplete, so it is an optimization, not the base design.
• "What about mobile?" A lower cap (64 MB), wrapping on by default because sideways scrolling is painful on a narrow screen, and server-side search by default for service logs.