cf.completefrontend
My progress
SYSTEM DESIGN · #114

Design an IoT dashboard for thousands of sensors

An operator console for a factory, a building or a fleet: tens of thousands of sensors report every few seconds, and the browser must show live charts, fleet health and alarms without ever receiving the raw firehose. The hard parts are server-side aggregation and screen-driven subscriptions, pixel-aware downsampling that keeps spikes and gaps, live charts on canvas with ring buffers, honest handling of stale, late and missing data, and alarms that do not flap or storm.

The question

Design the web dashboard for an industrial IoT platform. A customer has a few sites, each with buildings, floors and zones, and between 5,000 and 50,000 sensors that report temperature, pressure, power, vibration and similar values every 1 to 10 seconds.

Operators need a live overview of fleet health, drill-down charts for any sensor over any time range, and alarms they can acknowledge. Some screens run all day on wall displays in a control room. Walk me through the data, how it reaches the browser, how you draw it, and how you deal with missing, late and alarming data.

Real-life examples: AWS IoT SiteWise Monitor, Grafana for industrial telemetry, ThingsBoard, Samsara fleet dashboards, Smart building consoles, Factory floor SCADA web views

Requirements: which IoT dashboard are we building?

IoT (Internet of Things) means physical devices with sensors that send readings over the network. A sensor measures one thing (a temperature probe, a power meter, an accelerometer on a motor). A device or asset is the physical thing that carries one or more sensors, such as a chiller with ten sensors. A gateway is a small box on site that collects readings from many devices over local protocols and forwards them to the cloud.

The dashboard is the web app operators use to watch all of this. You have seen versions of it in AWS IoT SiteWise Monitor, Grafana boards built on industrial data, ThingsBoard, Samsara fleet dashboards, and smart-building consoles that show every floor of an office tower.

Several other guides cover nearby ground, and we link them instead of repeating them. The analytics-dashboard guide covers the widget grid, one query per widget, filters and URL state. The stock-ticker guide covers flashing cells and throttled rendering of fast prices, and the live-leaderboard guide covers ranked lists that reorder while people read them.

The log-viewer guide covers streaming append-only text, and the google-maps guide covers tiles and marker clustering. The data-table guide covers big tables, the tree-view guide covers lazy, keyboard-friendly trees, and the notifications-center guide covers an alert inbox. This guide focuses on what is special about telemetry from many devices: time series, volume, gaps, staleness and alarms.

Questions to ask first

QuestionWhy it changes the designWhat this guide assumes
How many sensors, and how often do they report?Sets the volume. 500 sensors could stream raw; 50,000 cannot.20,000 sensors per customer on average, up to 50,000; one sample every 1 to 10 s, typical 5 s.
How far back can users look?Decides whether we need precomputed rollups.Raw data for 30 days, 1 minute rollups for 1 year, 1 hour and 1 day rollups for 5 years.
How fresh must the live view be?Decides push vs polling and the batching window.A new reading should appear within 2 s of the server receiving it.
Who evaluates alarms, and who must see them?Alarms must fire even when no browser is open.Rules run on the server; every operator on a site sees the same alarm state.
Are there wall displays?Always-on screens change memory and reconnect rules.Yes: control room screens that run for weeks.
Do users write to devices?Commands to machines bring safety rules.Out of scope: the dashboard is read and acknowledge only.

Functional requirements

  • Fleet overview: a site tree (site, building, floor, zone, device) with roll-up health, a device list with status (online, stale, offline, alarming), and a map or floor plan.
  • Device detail: live charts for each sensor of a device, with a time range picker from the last 15 minutes to the last 5 years.
  • Live tiles: current value, unit and a small trend line (a sparkline) for many sensors on one screen.
  • Alarms: a list of active alarms, alarm markers on charts, acknowledge with a note, synced across operators in real time.
  • Gaps and staleness: missing data shows as missing, and every value says how old it is.

Non-functional requirements

  • Live latency: server receive to pixel under 2 s at the 95th percentile.
  • Chart load: any time range on one sensor draws in under 1 s, because the server never sends more points than the chart has pixels for.
  • Main thread: live updates cost under 100 ms of main-thread work per second (10%), so scrolling and typing stay smooth.
  • Long running: a wall display runs for 30 days with flat memory and reconnects on its own after any outage.
  • Accessible: every chart has a text alternative, alarm states never rely on colour alone, and the site tree works with a keyboard.

Out of scope: device provisioning, firmware updates, sending commands to machines, and the rule editor for alarms. Each one is a product of its own. We also skip the dashboard builder, since analytics-dashboard covers widget layout.

Check your understanding · Exercise 1

Which answers would change the architecture of this dashboard the most? Select all that apply.

What makes it hard: the shape and the size of telemetry

Before any boxes and arrows, look at the data. Every later decision comes from two facts: telemetry is a huge number of small time series, and almost none of it can be sent to the browser as it arrives.

Wireframe of the operator console for Building 2. A header has a Live status pill, a range picker set to 1 hour, and the site time 14:02 IST. On the left, a site tree: Pune site, Building 2 with 3 alarming devices, Floor 3 with 40 offline, Zone B, Floor 4. In the middle, four live tiles: chiller CH-7 at 6.2 degrees, air handler AHU-2 at 21.4 degrees, pump P-3 stale for 4 minutes and faded, and cold room CR-2 alarming at 9.8 degrees. On the right, an alarm panel with 4 alarms, showing CR-2 high and gateway GW-3 down first, and an Ack button. Below the tiles, a detail chart for CR-2 over 1 hour with a dashed threshold line at 8 degrees and a break in the line labelled no data. A footer reads Pune: 6,000 devices, 3 alarming, 41 offline.Plant console: Building 2● LiveRange: 1 hsite time 14:02 ISTPune siteBuilding 2 ▲3Floor 3 ⊘40Zone BFloor 4site treeCH-7 6.2°AHU-2 21.4°P-3 stale 4mCR-2 ▲9.8°Alarms (4)▲ CR-2 high⊘ GW-3 downAckCR-2 cold room temp, last 1 hno dataPune: 6,000 devices, 3 alarming, 41 offline
Every part answers is anything wrong, and where: roll-up badges in the tree, tiles with their age, alarms with an acknowledge action, and a chart that shows the threshold and the hole in the data instead of hiding it.

The shape: one time series per sensor per metric

A time series is a list of (timestamp, value) pairs for one thing. Here the thing is a sensor and a metric, such as chiller-07 / supply_temp. A single reading is a sample. Samples carry more than a number, and each extra field matters to the screen:

  • Unit: degC, kPa, kW, mm/s. The unit belongs to the series, not to each sample, and the UI converts it for display (Celsius vs Fahrenheit) without touching the stored value.
  • Quality: industrial systems tag each sample as good, uncertain or bad. AWS IoT SiteWise uses GOOD, BAD and UNCERTAIN; OPC UA (the common factory protocol) has the same idea. A bad sample must not be drawn like a good one.
  • Two timestamps: when the device measured it and when the server received it. They differ, sometimes by hours (chapter 7).
  • Type: most series are numbers, but some are states (pump running or stopped, door open or closed) and draw as coloured bands, not lines.

Sampling is irregular. Some devices report on a fixed clock. Many report by exception: they send a value only when it moves more than a set amount (a deadband), so a steady temperature may send nothing for ten minutes.

For a report-by-exception device, silence means "unchanged". For a clocked device, silence means "something is wrong". The UI has to know which kind of device it is looking at.

The size: do the volume math out loud

The 80 bytes per JSON sample is an estimate for a message like an id, a timestamp, a value and a quality flag.
QuantityMathResult
Ingest rate20,000 sensors x 1 sample per 5 s4,000 samples/s
Ingest at the top end50,000 sensors x 1 sample per 1 s50,000 samples/s
Per day4,000 x 86,400 sabout 346 million samples
As JSON to one browser4,000 x about 80 bytesabout 320 KB/s, 28 GB a day
One sensor, 1 day raw at 5 s86,400 / 517,280 points
One sensor, 1 year raw17,280 x 365about 6.3 million points
One sensor, 5 years raw17,280 x 1,825about 31.5 million points

Now compare that with what a screen can show. A chart 1,200 CSS pixels wide can show at most about 2,400 meaningful points (a low and a high per pixel column). A device list shows maybe 30 rows at a time, and a floor plan a few hundred icons.

People read a changing number at most a few times per second. So one screen needs a few thousand numbers to draw its charts once, then a few hundred changed values per second. The fleet produces 4,000 to 50,000 new samples every second, for every customer, forever.

Four boxes left to right. Devices: 20,000 sensors, one sample every 5 seconds. Ingest: 4,000 samples per second, about 346 million a day, stored raw and as rollups. BFF: sends only screen-sized aggregates. Browser: about 300 visible series in one batch per second, a few KB. A wedge under the boxes narrows from left to right, and a note says a 1,200 pixel chart can show about 2,400 points and a list shows about 30 rows.Devices20,000 sensorsIngest4,000 samples/sBFFaggregatesBrowser300 series, 1/s1 sample / 5 s each346 M a daymin, max per bucketa few KB/sa 1,200 px chart shows about 2,400 points
The fleet produces thousands of samples per second forever; one screen uses a few thousand numbers to draw, then a few hundred changed values per second. Everything between the devices and the browser exists to close that gap without hiding spikes or gaps.

That gap is the whole design. The rest of the guide answers the three questions it raises: which numbers the browser needs right now (chapter 3), how to turn millions of samples into a few thousand without lying (chapter 4), and how to draw the live part cheaply (chapter 5).

Three more traits make telemetry harder than a stock ticker. Data goes missing (devices lose power, gateways lose the network). Data arrives late and out of order (a gateway uploads an hour of buffered readings after it reconnects). And the important events are rare spikes inside long boring stretches, which is exactly what naive averaging erases.

Check your understanding · Exercise 2

An operator opens a 1-year chart of one vibration sensor that samples every 5 s, in a chart 1,000 px wide. Which statements are true? Select all that apply.

Never stream the firehose: aggregate on the server, subscribe to the screen

The naive design opens a socket and forwards every sample from the ingest pipeline to every browser. It works in the demo with 200 simulated sensors. With 20,000 real ones, each browser receives 4,000 messages per second, parses about 320 KB of JSON per second, and runs a store update for each one.

Even at 0.05 ms per message, that is 200 ms of main-thread work every second, before a single pixel is drawn. Ten operators means ten copies of the whole firehose leaving the data centre.

The fix has two halves. The server reduces data before it leaves (aggregation), and the browser asks only for what the operator can see right now (a subscription tied to the screen).

Decision What should the live channel send to each browser?

Everything, filter in the browser

The server pushes every sample for the customer.

The client drops what it does not show.

Pros

  • Simplest server.
  • Any screen can show anything instantly.

Cons

  • Bandwidth and CPU grow with fleet size, not with screen size.
  • Leaks data for sites the user may not be allowed to see unless the server filters anyway.

Poll a summary endpoint every few seconds

Each widget calls GET /latest?ids=... on a timer.

Pros

  • Plain HTTP, easy to cache and debug.
  • No socket state on the server.

Cons

  • Up to one full interval of extra delay.
  • Many widgets times many operators means many requests that mostly return "no change".

Screen-driven subscriptions over one socket

The client tells the server which series and which resolution it is showing.

The server sends a batch of changes for only those series every 500 ms to 1 s.

Pros

  • Traffic follows the screen: 300 visible sensors cost the same with 5,000 or 50,000 in the fleet.
  • One place to enforce permissions per series.

Cons

  • The server keeps per-connection subscription state.
  • The client must keep the subscription in sync as the view changes.

What we pick: Screen-driven subscriptions. The cost then depends on what one human can see, which is small and bounded, instead of on fleet size, which grows every quarter. Polling stays as the fallback and as the way to load history, but the live tail comes over one socket.

What a subscription names

A subscription is not just a list of ids. It names three things, because each one changes what the server sends:

  • Which series: the sensors and metrics in view. For a list, that is the visible rows plus a small overscan. For a map or floor plan, it is a bounding box or a zone id, and the server resolves it to sensors.
  • What form: a fleet tile needs only the latest value and status. A sparkline needs one value per 10 s bucket. A detail chart needs per-bucket min and max at the chart's resolution.
  • How often: a wall display summary may be fine with 5 s batches; an open detail chart wants 1 s.
ts
type Subscribe = {
  op: "sub";
  key: string; // "list:visible", "chart:chiller-07", ...
  series: string[]; // or { zone: "b2-f3" } resolved on the server
  form: "latest" | "spark-10s" | "minmax";
  bucketMs?: number; // for minmax: chart range / plot width
  everyMs: 500 | 1000 | 5000;
};

type Unsubscribe = { op: "unsub"; key: string };
Each screen region owns one keyed subscription. Replacing a key replaces the whole set, so the client never has to compute adds and removes.

Keeping it in sync as the view moves

The view changes all the time: the operator scrolls the device list, pans the floor plan, opens a chart, changes its range from 1 hour to 7 days. Sending a new subscription on every scroll event would flood the server. Three rules keep it calm:

  1. Debounce the change by about 150 to 250 ms after scrolling or panning stops (the debounce-throttle lesson covers the mechanics). Rows that fly past during a fling never subscribe.
  2. Keep a grace period: when a series leaves the screen, keep it subscribed for 5 to 10 s. Scrolling back a little then shows fresh values at once.
  3. Fill on subscribe: the server answers a new subscription with the current value of each series straight away, so a newly visible row never shows a blank until the next batch.
Sequence diagram with four columns: List, Sub manager, Socket worker and BFF. The list tells the sub manager it settled on rows 120 to 150, 200 milliseconds after scrolling stopped. The sub manager sends a keyed subscription for list visible to the socket worker, which sends sub with 31 series to the BFF. The BFF replies with handles and a snapshot of current values, and the worker hands the first values to the list, which draws them. Then the BFF sends 1 second batches. Rows that left the screen are unsubscribed after a 10 second grace period.ListSub managerSocket workerBFFsettled: 31 rowssub list:visiblesub, 31 serieshandles + snapshotfirst values drawn1 s batchesunsub old rows
The subscription follows where the user settles, not every frame of the fling. The snapshot on subscribe means new rows never sit blank waiting for a slow sensor to change.

Some data must flow no matter what is on screen. Alarm changes and fleet-level counts (how many devices are offline per site) go on a small, always-on channel. They are cheap because they are already rolled up on the server, and the operator must never miss them because the right row was scrolled away.

The fleet lab below runs 300 simulated sensors that each report every 2 s, standing for a fleet of 300, 3,000 or 30,000. Switch between subscribing to everything and subscribing to what is visible, then between rendering on every message and once per frame. Watch messages per second, renders per second and the share of a simulated main-thread budget grow or stay flat as the fleet grows.

Then take gateway GW-3 offline and compare a flat list of alarms with alarms grouped by their parent. In the flat list, the two real alarms are pushed out of view by offline rows, which is the alarm storm of chapter 8 in miniature.

Fleet
Second 1. A grid of 300 sensors standing for a fleet of 300; the top 4 rows are visible on screen. Every sensor is subscribed, one render per message. 150 messages per second, 150 renders per second, 30% of the main thread, 12 KB per second. The alarm list has 2 entries.second 1: dashed box is what the screen showsmessages/s: 150renders/s: 150main thread: 30% of budgetHigh temp: sensor 23High vibration: sensor 187warm sent, red alarm, hatched offline, faded unsubscribed2 alarm rows, 2 screen reader announcements
30% of the main thread, 12 KB/s on the wire. Try a bigger fleet: this cost grows with it.

Check your understanding · Exercise 3

An operator scrolls quickly through a list of 20,000 devices, then stops. Which behaviours belong in a good subscription layer? Select all that apply.

Downsampling: fewer points, same picture

Now the history chart. The operator opens a pump and picks "last 30 days". At one sample per 5 s that is 518,400 raw points for one line. The naive chart fetches them all: about 10 MB of JSON (at about 20 bytes per time and value pair), a second or more of parsing, and an SVG path with half a million segments.

Worse, most of that work is invisible. The plot area is perhaps 1,200 px wide, so about 430 samples land in each pixel column.

The rule that fixes this is pixel-aware downsampling: ask the server for about one bucket per pixel column, where a bucket is a slice of time. Bucket size = time range / plot width, so for 30 days on 1,200 px each bucket covers 36 minutes. Grafana works the same way: its "max data points" query option defaults to the width of the panel in pixels. The open question is what one bucket should contain.

Decision How should the server reduce each bucket of raw samples?

Average per bucket

Return the mean of all samples in each bucket.

Pros

  • One number per bucket, the smallest payload.
  • Smooth, easy to read trends.

Cons

  • Hides spikes: a 10 s pressure spike to 900 kPa inside a 36 minute bucket at 400 kPa moves the average by about 2 kPa.
  • Operators look for exactly those spikes.

Min and max per bucket

Return the lowest and highest value of each bucket (optionally also first and last).

Draw a vertical span per pixel column, or a line through min and max in time order.

Pros

  • Every spike and dip survives, by construction.
  • Merges exactly: the min of a week is the min of its daily mins, so rollups compose.

Cons

  • Two to four numbers per bucket.
  • Noisy data looks like a thick band, which is honest but can surprise users.

LTTB (Largest-Triangle-Three-Buckets)

Keep the first and last point. In each bucket, keep the one real point that makes the largest triangle with the point kept before it and the average of the next bucket.

Pros

  • Returns real samples, so tooltips show true values and times.
  • Looks very close to the full line at a fixed point budget.

Cons

  • Keeps one point per bucket, so in a noisy bucket with both a spike and a dip, one of them can be lost.
  • Needs the next bucket, so it cannot be precomputed per bucket or merged across rollups.

What we pick: Min and max per bucket for every chart where a missed spike costs money, which is most of an industrial dashboard. They are cheap to precompute, they compose across rollups, and they never hide an excursion. We also store sum and count so the UI can draw a mean line on top. LTTB is a good choice for presentation charts and for client-side thinning of a raw window, where it is applied once to real samples.

LTTB comes from Sveinn Steinarsson's 2013 master's thesis at the University of Iceland, "Downsampling Time Series for Visual Representation". It is popular because it keeps the shape of a line well with very few points. It is a visual method, not a statistical one, and that is why min and max are the safer default for alarms and maintenance.

Min and max per pixel column is also well studied. The M4 method (Jugel and others, VLDB 2014) keeps the first, last, minimum and maximum sample of each pixel column, and the paper shows that a line chart drawn from those four points matches the chart drawn from all the raw data. Our buckets store min and max as values; add first and last when lines between columns must be exact.

The lab below draws 12 hours of one sensor (1,440 samples, one every 30 s) with two spikes, a dip and an hour with no data. Change the chart width and the method, and watch which extremes survive. At 60 columns, spike B and dip C share a bucket, so LTTB keeps only one of them. Then turn on "connect across missing data" to see a line invent an hour of readings.

Chart width
Method
12 hours of one sensor, 1,440 samples with 120 missing, drawn average at 60 pixel columns: 56 points. Spike A at 96 lost, spike B at 82 lost, dip C at 12 lost. The missing hour is shown as a gap.no dataA: lostB: lostC: lostpoints sent: 56raw: 1,440 (120 missing)per column: 0.9spike A (96): lostspike B (82): lostdip C (12): lostgap: drawn as a gap, the line breaks
Averages smooth the spikes into the noise: the operator would never know the pump hit 96.

Rollups: a pyramid in time

Computing min and max over 6.3 million raw rows for a 1-year chart on every request is slow. So the pipeline precomputes rollups: tables of per-bucket aggregates at fixed resolutions, written as data arrives. AWS IoT SiteWise does this as a service: its aggregates API takes a resolution of 1m, 15m, 1h or 1d and returns average, count, minimum, maximum, sum and standard deviation. It works like map tiles: each zoom level has its own precomputed layer, and the client reads the one that matches its zoom.

A pyramid of five layers for one sensor. At the bottom, raw samples, 17,280 a day, used for a 1 hour range. Above it, 1 minute rollups, 1,440 a day, for a 1 day range. Then 15 minute rollups, 96 a day, for 30 days. Then 1 hour rollups, 24 a day, for 1 year. At the top, 1 day rollups, 1 a day, for 5 years. Each bucket stores min, max, sum and count.1 d: 1 a day5 years1 h: 24 a day1 year15 m: 96 a day30 days1 m: 1,440 a day1 dayraw: 17,280 a dayraw serves ranges up to about 1 houreach bucket: min, max, sum, count
Like map tiles, each zoom level has its own precomputed layer. The client sends its width; the server picks the coarsest layer whose bucket still fits range / width and merges whole groups of neighbours.
Pick the coarsest rollup whose bucket is no bigger than range / width, then merge whole groups of neighbours on the server, rounding the bucket up to a multiple of the rollup. The client gets at most one bucket per pixel column, and every merged bucket is exact.
Range on a 1,200 px chartIdeal bucket (range / width)Read fromRows readBuckets sent
1 hour3 sraw (5 s samples)720720 raw samples
1 day72 s1 m rollup, merged 2 at a time (2 min)1,440720
30 days36 min15 m rollup, merged 3 at a time (45 min)2,880960
1 yearabout 7.3 h1 h rollup, merged 8 at a time (8 h)8,7601,095
5 yearsabout 36.5 h1 d rollup, merged 2 at a time (2 d)1,825913

Merging is where the choice of aggregates matters. The min of three 15 minute buckets is the min of their mins, and the same goes for max, sum and count. The average of three averages is wrong when the buckets have different counts (one had a gap), so we store sum and count and divide at the end. Merged buckets must also be whole multiples of the rollup: a 36 minute bucket would cut a 15 minute bucket in half, and nobody knows which half held its max.

ts
type Bucket = { t: number; min: number; max: number; sum: number; n: number };

// Merge fine buckets into coarser ones of size `stepMs`, a whole multiple of the
// fine size. Empty buckets stay out: the client sees a missing bucket and draws a gap.
function merge(fine: Bucket[], stepMs: number): Bucket[] {
  const out: Bucket[] = [];
  for (const b of fine) {
    if (b.n === 0) continue;
    const t = Math.floor(b.t / stepMs) * stepMs;
    const last = out[out.length - 1];
    if (last && last.t === t) {
      last.min = Math.min(last.min, b.min);
      last.max = Math.max(last.max, b.max);
      last.sum += b.sum;
      last.n += b.n;
    } else {
      out.push({ ...b, t });
    }
  }
  return out;
}
Rollups compose because min, max, sum and count all merge exactly. The mean is sum / n, computed last.

Two details make the chart honest. First, an empty bucket is not a zero: the server leaves it out (or sends null), and the chart breaks the line there. Second, LTTB in its original form assumes no gaps; the reference implementation lists gaps as a known limitation. If you use it, run it on each unbroken segment separately.

Check your understanding · Exercise 4

A compressor's discharge pressure briefly hit 950 kPa for 8 s last Tuesday. Which ways of drawing a 30-day chart at 1,200 px will still show that spike? Select all that apply.

Live charts: ring buffers, one draw per frame, canvas

History is one request. The live tail is a stream: every second a batch arrives with new buckets for the series on screen. The naive live chart pushes each new point into a JavaScript array, calls setState, and lets a React SVG chart re-render.

On a screen with 100 sparklines and 8 detail charts, that is hundreds of re-renders a second, arrays that grow forever, and an SVG DOM with tens of thousands of nodes. The tab stutters within minutes and runs out of memory within days.

The stock-ticker guide covers the general rule of coalescing updates into one render per animation frame, so here we focus on what charts of telemetry add: a fixed time window, many lines, and memory that must stay flat.

A ring buffer per window

A live chart shows a sliding window, such as the last 15 minutes. Old points fall off the left as new ones arrive on the right. A ring buffer is a fixed-size array with a write index that wraps around: writing point 901 into a 900-slot buffer overwrites slot 0.

Memory is allocated once and never grows. With a typed array (Float32Array, Float64Array), the values sit in one flat block of memory, with no object per point for the garbage collector to track.

ts
class Ring {
  private v: Float32Array;
  private head = 0; // next slot to write
  private size = 0;
  constructor(readonly capacity: number) {
    this.v = new Float32Array(capacity).fill(NaN); // NaN = no data
  }
  push(value: number) {
    this.v[this.head] = value;
    this.head = (this.head + 1) % this.capacity;
    this.size = Math.min(this.size + 1, this.capacity);
  }
  // Oldest to newest, without allocating: the chart reads through this.
  forEach(fn: (value: number, i: number) => void) {
    const start = (this.head - this.size + this.capacity) % this.capacity;
    for (let i = 0; i < this.size; i++) fn(this.v[(start + i) % this.capacity], i);
  }
}
Values only: the server sends buckets on a fixed grid (every 10 s, say), so every series shares one time axis and the slot index is the time. A missing bucket is written as NaN, which the chart draws as a gap.
A ring buffer with 10 slots, numbered 0 to 9, holding 21.7, 21.9, 22.1, 20.4, 20.6, 20.9, NaN, 21.0, 21.2 and 21.4. Slot 2 holds the newest value. The head points at slot 3, which holds the oldest value and is overwritten next. Slot 6 holds NaN, a missing bucket that the chart draws as a gap. Reading goes from slot 3 to 9, then wraps to 0 to 2. As a Float32Array it uses 40 bytes, allocated once.21.721.922.120.420.620.9NaN21.021.221.40123456789newestgaphead: next write, oldest valueread order: slots 3 to 9, then 0 to 2Float32Array(10): 40 bytes, allocated once
The buffer never grows: each new bucket overwrites the oldest. A missing bucket is NaN, so the chart can break the line exactly there.

Do the memory math before choosing the type. A Float64Array uses 8 bytes per value, a Float32Array 4. Float32 keeps about 7 significant digits, which is plenty for a temperature of 21.37 or a power reading of 1,284.6 kW. It is not enough for epoch timestamps in milliseconds (13 digits, rounded to steps of about 2 minutes), which is one more reason to share a single time axis instead of storing a time per point.

Typed arrays have an exact size. Arrays of small objects cost far more per point and give the garbage collector millions of things to trace.
ScreenSeries x pointsFloat64ArrayFloat32Array
Device page, 1 h at 1 s20 x 3,600 = 72,000576 KB288 KB
Tile grid, 1 h sparklines at 10 s300 x 360 = 108,000864 KB432 KB
Wall display, 24 h at 10 s2,000 x 8,640 = 17.3 millionabout 138 MBabout 69 MB
Same, as JS objects {t, v}17.3 million objectsoften several hundred MB (estimate)n/a

Rendering: when and with what

When: socket batches write into ring buffers and mark the chart dirty. A single requestAnimationFrame loop redraws dirty charts, at most once per frame. If the tab is hidden, the visibilitychange event fires and the loop stops; browsers also pause requestAnimationFrame in background tabs. Buffers keep filling, so the chart is current the moment the tab comes back.

Decision What should draw the charts?

SVG, one element per line or point

A React chart library builds path and circle elements.

Pros

  • Crisp, styleable with CSS, each point can be focusable.
  • Great for a few charts with a few hundred points.

Cons

  • Each update rebuilds or diffs DOM nodes; thousands of points means thousands of nodes.
  • 100 sparklines updating each second keeps layout and paint busy.

Canvas 2D (for example uPlot)

Draw lines into a bitmap with CanvasRenderingContext2D; one element per chart, no DOM per point.

Pros

  • uPlot's README reports about 50 KB minified and a chart with 166,650 points created in 25 ms from cold start, on the author's hardware.
  • Redraw cost depends on points in view, not on DOM size.

Cons

  • Accessibility and hit-testing are your job (chapter 11).
  • Text and tooltips need a DOM layer on top.

WebGL (ECharts GL, Plotly scattergl, custom)

Upload points to the GPU and draw them with shaders.

Pros

  • Millions of points per frame.
  • Good for scatter plots and spectrograms of vibration data.

Cons

  • Larger bundles and slower start.
  • Browsers cap the number of live WebGL contexts per page, so 100 sparklines cannot each have their own.

What we pick: Canvas 2D for everything on this dashboard, with uPlot (or a similar small canvas library) for detail charts. Thanks to downsampling, no chart ever holds more than a few thousand points, which is well inside what canvas handles in a few milliseconds. WebGL is kept for one special view, the vibration spectrogram, not for ordinary lines.

Sparklines in one canvas. A tile grid with 100 sparklines should not mean 100 canvases and 100 chart instances. Draw them all into one canvas that sits behind the grid, each in its own rectangle, measured from the tile positions once per layout change. One draw call loop per frame paints only the tiles whose data changed. The tiles themselves stay plain DOM, with the current value as real text.

Workers. Parsing binary batches and running LTTB on a raw window are pure computations, so they can run in a Web Worker and hand back typed arrays as transferable objects (moved, not copied). With OffscreenCanvas, a worker can even draw the charts itself. That helps when the main thread is busy with a large tree or table. Start with parsing in a worker, measure, and move drawing only if the profile says so.

Check your understanding · Exercise 5

A wall display shows 2,000 series with a 24-hour live window at 10 s buckets. Which choices keep it healthy for weeks? Select all that apply.

The fleet view: status, lists, maps and the site tree

The first screen an operator sees answers one question: is anything wrong, and where? That needs a status for every device and a way to roll statuses up to sites and buildings. The naive version downloads all 20,000 devices with their latest readings, computes status in the browser, and filters and sorts in memory.

That is a 5 to 10 MB payload on every load. The status logic is duplicated (and drifts) between the server's alarm engine and the browser, and the numbers in the header disagree with what another operator sees.

One status, computed in one place

Thresholds depend on device type: a 1 s vibration sensor is stale after a few seconds, a 15 minute deadband meter only after an hour. Every status has a shape and a word, not only a colour.
StatusRule (example thresholds)Shown as
OnlineLast sample within the expected interval x 3Filled circle, "Online"
StaleNo sample for 3 to 10 intervalsHollow circle, "Stale 2 min"
OfflineNo sample for 10+ intervals, or the gateway reported it disconnectedCrossed circle, "Offline since 09:42"
AlarmingAt least one firing alarm (chapter 8)Triangle with severity, "High: 2 alarms"
UnknownNever reported, or decommissionedDashed circle, "No data yet"

The server computes status, because it already sees every sample and every alarm, and because all operators must agree. The browser shows it. The one thing the client adds is a local clock tick: if the last-seen time crosses the stale threshold while no message arrives, the client can show "stale" itself rather than wait for the server to say so. That matters most when the connection to the server is what broke.

The site tree and roll-up health

Industrial customers think in places: site, building, floor, zone, device. The left panel is a tree (the tree-view guide covers lazy loading and keyboard behaviour), and each node shows a roll-up: how many of its devices are in each state, plus the worst state as its badge.

Decision Where should roll-up counts for the site tree be computed?

In the browser, from every device's status

Load all devices, group them by ancestors, count.

Pros

  • No extra server work.
  • Instant re-grouping by any field.

Cons

  • Needs every device on every client, which is the payload we are trying to avoid.
  • Every status change must reach every client to keep counts right.

On the server, as live counters per tree node

When a device changes status, the server adjusts the counts of its zone, floor, building and site.

Clients subscribe to the counts of the nodes they have expanded.

Pros

  • A 40-node expanded tree costs 40 small counters, whatever the fleet size.
  • Every operator sees the same numbers.

Cons

  • Counters must be rebuilt after restarts and kept correct when devices move between zones.

What we pick: Server-side counters. The tree is the place where fleet size hurts most, and the counts are a pure function of state the server already owns. The client stores them per node id in a normalized map and patches them from the always-on channel.

A site tree with roll-up counts. Pune site: 3 alarming and 41 offline of 6,000. Its children: Building 1 with 1 offline of 3,000, and Building 2, marked bad, with 3 alarming and 40 offline of 3,000. Building 2's children: Floor 3, marked bad, with 2 alarming and 40 offline of 800, and Floor 4 with 1 alarming of 700. Floor 3's child Zone B has 2 alarming and 40 offline of 120. A note says when a device goes offline, the server adds one to its zone, floor, building and site.Pune site▲3 ⊘41 / 6,000Building 1⊘1 / 3,000Building 2▲3 ⊘40 / 3,000Floor 3▲2 ⊘40 / 800Floor 4▲1 / 700Zone B▲2 ⊘40 / 120device goes offline:+1 on zone, floor,building and site
The tree badge is a counter the server keeps per node, so an expanded tree of 40 nodes costs 40 small numbers, not 6,000 device records. The worst state inside picks the badge; the counts say how bad.

The device list and the map

The device list is a virtualized table (see the virtualization lesson and the data-table guide). What is specific here is that sorting and filtering happen on the server: "offline devices in Building 2, oldest first" is a query, and the client asks for pages by cursor. Live updates only patch status and value cells of rows that are subscribed.

If a live change would move a row (a device goes offline while the list is sorted by status), do not reorder under the user's cursor. Show "3 devices changed status, refresh order" and let them apply it. The live-leaderboard guide covers the other choice, animating live reorders, for lists where movement is the point.

The map and the floor plan reuse what the google-maps guide teaches about clustering. The telemetry twist: a cluster's badge shows the worst status inside it and the count per status, so a red triangle on a cluster of 400 tells you to zoom in there. For floor plans, the zone polygons are the natural clusters, and the roll-up counters from the tree already give their colours.

Check your understanding · Exercise 6

Building 2 has 3,000 devices. A floor's network switch fails and 120 devices stop reporting. What should the operator see? Select all that apply.

Gaps, stale values, clock skew and late data

Telemetry is never complete. Devices reboot, gateways lose the uplink, a sensor's battery dies. A dashboard that hides this is dangerous: an operator who sees a calm flat line at 4 degrees in a cold room assumes the cold room is fine, when in fact the probe died at 4 degrees six hours ago. This chapter is about telling the truth about time.

No data is not zero

Three different things can look alike on a chart, and each needs its own drawing:

  • A real zero: the power meter reads 0 kW because the line is stopped. Draw it as a value at zero.
  • A gap: no samples arrived for a clocked device. Break the line, and optionally shade the gap with a label such as "no data 02:10 to 03:40".
  • Unchanged: a deadband device sent nothing because nothing moved. Draw a step line that holds the last value, but only up to that device's stale threshold; after that it becomes a gap.

Chart libraries default to different things here, so be explicit. Grafana's time series panel has a "connect null values" setting with never, always, and a threshold option, which is the same choice in product form.

In our data, a missing bucket is null on the wire and NaN in the ring buffer, and the renderer never draws a segment that touches one. For raw points, we break the line when two neighbours are further apart than about 2 to 3 times the expected interval, which is what the threshold option does.

Whose clock?

Each sample has a device time (when it was measured) and a receive time (when the server got it). Device clocks drift, reset to 1970 after a power loss, or run in the wrong time zone. Receive times are trustworthy but include network delay and, after an outage, hours of buffering.

Decision Which timestamp should charts and staleness use?

Device time everywhere

Plot and compute staleness from the time the device reports.

Pros

  • Buffered data lands where it happened, so a backfilled hour looks right.

Cons

  • A device with a clock 10 minutes fast looks fresh even when silent.
  • A clock reset puts points in 1970 or in the future.

Receive time everywhere

Ignore device time; the server stamps each sample on arrival.

Pros

  • One trusted clock, simple staleness.

Cons

  • A one-hour backfill lands as one huge spike at reconnect time.
  • Network jitter moves points around.

Device time for plotting, receive time for liveness

Plot by device time when it is within a sane window of receive time (for example within 5 minutes in the future); otherwise plot by receive time and flag the sample.

Compute last-seen and staleness only from receive time.

Pros

  • Backfills land in the right place, and a wrong clock cannot fake liveness.
  • The server can show a "clock skew" warning per device.

Cons

  • Two timestamps per sample on the server, and rules to explain to users.

What we pick: Device time for the chart, receive time for liveness, with the server correcting or flagging clocks that are clearly wrong. Each question then gets the clock that answers it truthfully: when did it happen, and when did we last hear from it.

Late, out-of-order and backfilled samples

A gateway loses its uplink at 02:10, keeps buffering, and reconnects at 03:40. It then uploads 90 minutes of readings for 400 devices in a few seconds. Those samples belong to buckets that the live stream already closed and sent as empty. Three things must happen:

  1. The pipeline writes the samples by device time and recomputes the affected rollup buckets (1 m, 15 m, 1 h, 1 d).
  2. It sends each subscribed client a small revision message: series ids plus the time range that changed, not the data itself.
  3. Each client checks whether that range is inside a chart it is showing. If so, it refetches just that range at its current resolution and replaces those buckets. If not, it marks its history cache for that range as stale.
A timeline from 02:00 to 04:00. Data flows until 02:10, then the gateway loses its uplink and the live stream closes the buckets from 02:10 to 03:40 as empty. At 03:40 it reconnects. Below, four steps: the backfill uploads 90 minutes, the rollups are recomputed for 1 minute to 1 day, a revision message names the series and the range 02:10 to 03:40, and the chart refetches that range. At the bottom, the repaired timeline has data for the whole two hours.live stream, as it happeneduplink down: buckets closed empty02:0003:4004:00Backfill90 min uploadRollupsredo 1 m to 1 dRevision02:10 to 03:40Chartrefetch rangeafter the revisionwhole range has data, by device timeslightly late samples: wait about 2 s before closing a bucket
The live stream cannot wait 90 minutes, so it closes buckets as empty. When the buffered data arrives, a small revision message tells clients which range changed, and only charts showing that range refetch it.

Smaller lateness is handled by waiting a little before closing a bucket. If the live stream sends a 10 s bucket 2 s after it ends, most slightly late samples still make it in. Anything later arrives as a revision. This is the same trade as a watermark in stream processing: wait longer for more complete buckets, or less for fresher ones.

Think first

A cold room probe reports by exception with a 0.5 degree deadband. It has sent nothing for 40 minutes. Its last value was 3.8 degrees. What should the tile and the chart show?

Check your understanding · Exercise 7

A gateway reconnects after 90 minutes and uploads its buffer. Which client behaviours are correct? Select all that apply.

Alarms: evaluated on the server, calm on the screen

An alarm (industrial software often says alarm; cloud tools say alert) is a rule over a series, such as "supply temperature above 8 degrees", plus the state that rule is in right now. The naive design checks thresholds in the browser as values arrive and turns tiles red.

It looks fine in a demo, and it is wrong in four ways. No alarm fires when no browser is open at 3 a.m. Two operators with different subscriptions see different alarms. A value hovering at 8.0 flips the tile red and back many times a minute, and nobody can acknowledge a state that only exists in one tab.

Decision Where should alarm rules run?

In each browser

The client compares incoming values with thresholds it downloaded.

Pros

  • No server work, instant feedback.

Cons

  • Only works while a tab is open and subscribed to that series.
  • Clients disagree; there is no shared state to acknowledge, escalate or audit.

On the server, next to ingest

A rules engine evaluates every sample (or every bucket) and keeps one state per alarm.

State changes are stored and pushed to every client on the always-on channel.

Pros

  • Fires with no one watching, and can page or email.
  • One truth for every operator, with a history for audits.

Cons

  • Rule changes need a deploy or a rule editor.
  • Adds a hop of latency (usually well under a second).

What we pick: On the server. Alarms are a safety feature with an audit trail, so they cannot depend on which tabs happen to be open. The browser renders alarm state, sends acknowledgements, and draws alarms on charts. The only thing it computes is local display, such as a threshold line.

States, hysteresis and debounce

An alarm is a small state machine (see the state-machines lesson). Two tricks keep it from flapping, which means switching on and off many times as a value hovers near the limit:

  • Debounce with a pending state: the condition must hold for a duration (say 60 s) before the alarm fires. Grafana alert rules call this the pending period, and Prometheus has the same idea with its for clause.
  • Hysteresis: use a different limit to clear than to fire. Fire above 8.0 degrees, clear only below 7.5. A value bouncing between 7.9 and 8.1 then fires once and stays firing, instead of flapping.
Alarm state machine. OK goes to Pending when the value rises above 8.0. Pending goes back to OK if the value drops below 8.0 before 60 seconds pass, or to Firing if it holds for 60 seconds. Firing goes to Resolved only when the value drops below 7.5. Resolved goes back to OK when it is acknowledged or auto-closed. A note says hysteresis fires above 8.0 and clears only below 7.5, and acknowledged is a flag an operator sets on Firing or Resolved.OKnormalPendingwait 60 sFiringshown, notifiedResolvedneeds ack?above 8.0back below 8.060 sbelow 7.5acked, or auto-closehysteresis: fire above 8.0, clear only below 7.5acknowledged: a flag an operator sets on Firing or Resolved
The pending period filters short excursions; the gap between 8.0 and 7.5 stops a value hovering at the limit from flapping. Acknowledgement is a separate flag, so an alarm can be firing and acknowledged at once.

Acknowledged is a separate flag, not a separate condition: an operator says "I see it, I am on it". The alarm can be firing and acknowledged, or it can clear while still unacknowledged, which many plants require an operator to confirm. The ISA-18.2 standard for alarm management in process plants describes a similar lifecycle, with states such as unacknowledged, acknowledged, and returned to normal but unacknowledged. It calls the two tricks above deadband and on-delay.

Alarm storms

A gateway serving 400 devices loses power. Every device goes offline, and every "device offline" rule fires: 400 alarms in a second. Worse, sensors on those devices may also fire "value missing" rules.

The operator's list fills with 1,000 rows, and the one alarm that matters (a real high-pressure alarm in another building) scrolls out of view. This is an alarm storm, which alarm management guides call an alarm flood. Guidance based on ISA-18.2 often treats more than 10 new alarms in 10 minutes for one operator as a flood.

  • Group by parent: the server knows the topology (device belongs to gateway). When a parent is down, children's connectivity alarms are folded under one parent alarm: "Gateway GW-14 offline, 400 devices affected".
  • Suppress consequences: rules can declare dependencies, so "value missing" does not fire for a device whose gateway is down.
  • Rate-limit announcements in the client: the screen reader hears "1 critical alarm: Gateway GW-14 offline, 400 devices", not 400 messages.

Acknowledging, in sync across operators

Acknowledge is a write, and two operators may click it at the same moment. The client sends the alarm id with the version it saw. The server accepts the first, rejects the second with the current state, and broadcasts the change. The UI shows "Acknowledged by Sam, 14:02" in every open console within about a second.

Optimistic UI is fine here: show acknowledged at once. But the rollback path must be clear. If the server says the alarm already cleared or someone else acknowledged it, show who and when instead of an error.

Sequence diagram with four columns: Operator, Alarm panel, API and Console B, where Sam works. The operator clicks Ack with a note, and the panel shows Acked as pending at once. A moment earlier, Sam's console posted an ack with version 7. The panel also posts its ack with version 7. The API handles Sam's first and answers 200 with version 8, then replies to the panel with 409: Sam already acknowledged it, version 8. The panel replaces its optimistic state with Acked by Sam at 14:02, and the API broadcasts alarm version 8 to every console.OperatorAlarm panelAPIConsole Bclick Ack + noteAcked (pending)Sam: ack, v7POST ack, v7200, v8409: Sam, v8by Sam, 14:02alarm v8 to all
This is the failure path: two acks race with the same version, the first wins, and the second gets 409 with the current state. The optimistic state is replaced by the truth, and the user sees who acted instead of an error.

On charts, alarms become context: a dashed threshold line, a shaded band for each firing interval, and a marker with the acknowledgement note. These come from the alarm history API for the visible range, so a chart of last Tuesday shows last Tuesday's alarms.

Check your understanding · Exercise 8

A tank level hovers between 89.8% and 90.3% for ten minutes. The rule is "above 90% is high". Which setups stop the alarm from flapping? Select all that apply.

Transport: one socket, binary batches, resume after drops

We have decided what crosses the wire (screen-driven aggregates) and how often (every 0.5 to 1 s). Now the pipe itself. The choosing-realtime and websockets lessons cover the transports in general; here the questions are who the browser talks to, how compact a batch is, and what happens after the connection drops.

Decision Should the browser connect to the MQTT broker directly, or to a backend-for-frontend?

MQTT over WebSockets, straight to the broker

Devices already publish to an MQTT broker. The MQTT specification defines a WebSocket transport (subprotocol mqtt, binary frames), and brokers such as HiveMQ and EMQX support it, so a browser MQTT client can subscribe to topics like site/b2/+/temp.

Pros

  • No extra service: the broker already does fan-out.
  • Lowest possible latency for raw values.

Cons

  • Topics carry raw samples, so the browser gets the firehose for every topic it subscribes to, with no rollups or min and max.
  • Per-user permissions must be expressed as broker topic rules, and one broad wildcard can leak a whole site.
  • Alarms, counts and history live elsewhere, so the client still needs a second backend.

WebSocket to a backend-for-frontend (BFF)

A service built for this UI holds the subscriptions from chapter 3, reads from the stream and the rollups, checks permissions, and sends compact batches.

MQTT stays between devices, gateways and the cloud.

Pros

  • Aggregation, permissions, alarms and counts all come over one socket shaped for the screen.
  • The device protocol can change without touching the browser.

Cons

  • One more service to run and scale (each holds thousands of sockets).
  • Adds a small hop of latency.

What we pick: A BFF. The browser should never see device topics, for the same reason it should never see the raw firehose: the screen needs aggregates, permissions and alarm state, and the broker has none of those. Direct MQTT is fine for a lab tool watching ten devices. The backend-for-frontend lesson covers the pattern itself.

Compact batches

Every 1 s the BFF sends one frame per connection with the changes for subscribed series. Three tricks make it small:

  • Handles instead of ids: when the client subscribes, the server gives each series a small integer handle. A frame then says "handle 17" (2 bytes) instead of "chiller-07/supply_temp" (about 25 bytes).
  • Deltas only: a series that did not change in this batch is left out. In a calm plant, most are.
  • Binary frames: a typed layout parsed with a DataView, no JSON parser and no strings to allocate.
ts
// Frame: [u32 seq][u32 bucketStartSec][u16 count] then count x [u16 handle][f32 value][u8 flags]
function parse(buf: ArrayBuffer, rings: Map<number, Ring>) {
  const view = new DataView(buf);
  const seq = view.getUint32(0);
  const count = view.getUint16(8);
  let at = 10;
  for (let i = 0; i < count; i++) {
    const handle = view.getUint16(at);
    const value = view.getFloat32(at + 2);
    const flags = view.getUint8(at + 6); // bit 0: gap, bit 1: bad quality
    // A real client also keeps `flags` in a parallel Uint8Array ring, so bad samples draw differently.
    rings.get(handle)?.push(flags & 1 ? NaN : value);
    at += 7;
  }
  return seq;
}
7 bytes per changed series. A batch of 300 changes is about 2.1 KB, against about 15 KB as JSON (an estimate at 50 bytes per JSON entry).

WebSocket compression (permessage-deflate) helps JSON a lot and binary a little, at a CPU cost on both ends. With binary deltas it is often not worth it; measure with real data. Batching every 1 s also means we never send more frames than the screen can use: 1 frame per second per connection, whatever the ingest rate.

Dropping, resuming, filling the gap

Connections drop: laptops sleep, Wi-Fi roams, the BFF deploys. The connection is a state machine with backoff, and the important part is what happens to data missed while it was down.

Connection state machine. Connecting goes to Live when the socket opens. Live goes to Reconnecting when the socket drops or the watchdog sees 2 minutes without a frame. Reconnecting retries with backoff and goes to Resuming when the socket opens and subscriptions are sent with resumeFrom. Resuming goes back to Live if the server replays the missed frames, or to Backfilling if the server says too-old. Backfilling fetches the missed range over HTTP and goes to Live when caught up. A note says every frame has a sequence number, values show their age when not Live, and backoff with jitter is capped at 30 seconds and never gives up.Connectingfirst snapshotLive1 s framesReconnectingbackoff + jitterBackfillingHTTP gap fillResumingresumeFrom seqopendropopenreplay oktoo-oldcaught upevery frame has a seq; values show their age when not Livebackoff with jitter, capped at 30 s, never gives up
Resume is cheap when the gap is short (replay from the BFF's 60 s buffer) and correct when it is long (HTTP backfill of visible ranges). Either way the user sees ageing values and a banner, never silently frozen numbers.
  1. Each frame carries a sequence number. The client remembers the last one it applied and the bucket time it covered.
  2. On reconnect, the client re-sends its subscriptions with resumeFrom: lastSeq. If the BFF still holds those frames (a short replay buffer, say 60 s), it replays them and the gap never shows.
  3. If the gap is older than the buffer, the BFF answers resume: "too-old". The client then fetches the missed range over HTTP for the visible series only, writes it into the rings, and joins the live stream.
  4. Retries use exponential backoff with jitter (see retries-and-backoff), capped at about 30 s, and never give up. During the outage, every value shows its age and the header says "Live updates paused, reconnecting".
Sequence diagram with four columns: Charts, Socket worker, BFF and History API. The socket drops, and the worker tells the charts that live updates are paused, so values show their age. It retries at 1, 2 and 4 seconds, which fail. Then it opens and sends its subscriptions with resumeFrom 812. The BFF answers too-old. The worker asks the History API for the missed range of the visible series and gets buckets back. It writes them into the rings, and the charts go live again.ChartsSocket workerBFFHistory APIpaused, values ageretry 1, 2, 4 sresumeFrom 812too-oldGET missed rangebucketsrings filled, live
The failure path is visible at once: values age and a banner says live updates are paused. Recovery asks for exactly what the screen needs, the missed range of the visible series, instead of a replay of everything.

Check your understanding · Exercise 9

A control room laptop sleeps for 20 minutes and wakes up. The BFF keeps a 60 s replay buffer. What should the client do? Select all that apply.

Architecture, data model and interface

With the big decisions made, we can name the parts and what each one owns, then where each piece of state lives and the API between browser and server. The guiding idea is that each boundary sits where the data gets smaller: the pipeline turns millions of samples into rollups, the BFF turns rollups into screen-sized batches, and the client turns batches into pixels.

Architecture. The browser zone has UI with the tree, list and tiles; a Sub manager that knows what is visible; a Render loop that paints canvases on animation frames; a Worker zone with the Socket client that parses frames and resumes; a Series store with ring buffers and history; an Entity store with devices, nodes and alarms; and an HTTP client for history and acks. UI sends visible ids to the Sub manager, which sends subscriptions to the Socket client. The socket writes values into the Series store and alarms into the Entity store. The Render loop reads the Series store and paints the UI. The server zone has the BFF for socket and HTTP, the Alarm engine, and Rollups storage fed by ingest.BrowserServerUItree, list, tilesSub managerwhat is visibleRender looprAF, canvasWorkerSocket clientparse, resumeSeries storerings, historyEntity storedevices, alarmsHTTP clienthistory, ackssubalarmsvaluesdrawpainthistoryBFFsocket + HTTPAlarm enginerules, statesRollupsfed by ingest
Each boundary sits where data gets smaller. The worker absorbs parsing bursts, the series store keeps fast numbers out of React, and the BFF is the only thing the browser talks to.
PartLives inResponsibility
Ingest and rollupsServerStore raw samples by device time, maintain 1 m, 15 m, 1 h and 1 d min, max, sum and count, emit revisions for late data.
Alarm engineServerEvaluate rules with pending periods and hysteresis, group by parent, keep alarm state and history.
BFFServerHold per-connection subscriptions, map series to handles, send 1 s binary batches, keep tree counters and a 60 s replay buffer, check permissions.
Socket clientBrowser, in a workerConnect, back off, resume, parse frames into typed arrays, post them to the main thread as transferables.
Subscription managerBrowser, main threadCollect what each screen region shows, debounce, keep grace periods, send keyed subscriptions.
Series storeBrowser, main threadRing buffers for live windows, a history cache keyed by series, range and resolution.
Entity storeBrowser, main threadNormalized devices, tree nodes with counts, alarms, by id.
Render loopBrowser, main threadOne requestAnimationFrame loop that redraws dirty canvases and patches dirty text cells.
UIBrowserSite tree, device list, map or floor plan, tile grid, detail charts, alarm panel.

Why the socket lives in a worker: parsing is the one piece of live work that scales with incoming data, and it needs nothing from the DOM. Moving it off the main thread keeps scroll and input smooth during a burst, such as the backfill after a gateway reconnects. Why the series store is not in React state: values change every second for hundreds of series, and the only consumer is a canvas. React renders the structure (which tiles, which charts); the render loop paints the numbers.

The data flow for the main screen, end to end, makes the reduction visible at each hop.

Three flows left to right. Live: Ingest sends samples to the BFF, which sends a binary batch every second to the Worker, which writes Float32 values into Rings, which the Canvas draws once per animation frame. History: Rollups give min and max buckets to the BFF, which returns at most one bucket per pixel column, as columns, to the Cache, which loads typed arrays into the Chart. Write: the Alarm panel posts an ack to the API, the Alarm engine stores version 8, and an alarm event reaches all consoles.liveIngestBFFWorkerRingsCanvassamples1 s binaryFloat32per framehistoryRollupsBFFCacheChartmin, max1 per pixeltyped arrayswriteAlarm panelAPIAlarm engineAll consolesPOST ackstate v8alarm event
At every arrow the data is smaller or more targeted than at the one before. History requests carry the chart width (width=1200), so the server, not the browser, decides how many points to send.

Writes are few and go over HTTP: acknowledging an alarm, saving a view, adding a note. They are small, need clear errors, and must be idempotent; a socket gives none of that for free. Their results come back to every operator through the alarm channel.

Check your understanding · Exercise 10

Which responsibilities belong in the browser in this design? Select all that apply.

Data model: where each piece of state lives

The client holds five kinds of state, and each has a different lifetime and update rate. Mixing them is the usual source of slow dashboards: putting live values in the same store as device names means every value tick re-renders every component that reads a name.

ts
type Status = "online" | "stale" | "offline" | "alarming" | "unknown";

type Device = { id: string; name: string; nodeId: string; gatewayId: string;
  type: string; staleAfterMs: number; status: Status; lastSeenAt: number };

type Series = { id: string; deviceId: string; metric: string;
  unit: string; kind: "number" | "state"; precision: number };

type TreeNode = { id: string; parentId: string | null; level: "site" | "building" | "floor" | "zone";
  name: string; timeZone: string; counts: Record<Status, number> };

type Alarm = { id: string; seriesId: string; parentAlarmId?: string;
  state: "pending" | "firing" | "resolved"; acked?: { by: string; at: number; note?: string };
  severity: "low" | "high" | "critical"; since: number; version: number };

// Not in the entity store: ring buffers (Map of handle to Ring) and history
// buckets keyed by `${seriesId}|${resolution}|${from}-${to}`.
Entities are normalized by id (see the normalization lesson). Fast numbers live outside them.
StateWhereUpdatesWhy there
Devices, tree nodes, alarmsNormalized entity storeA few per secondShared by tree, list, map and alarm panel; patched by id.
Live valuesRing buffers (typed arrays)Hundreds per secondRead only by canvases and text cells in the render loop.
History bucketsIn-memory cache with LRU evictionOn range changeReused when the user zooms back; invalidated by revisions.
Selected site, device, range, resolutionURLOn navigationShareable and survives reload (see url-as-state).
Panel sizes, unit preferenceLocal storageRarelyPer operator, per machine.
Six boxes for client state. Top row: URL with site, device and range; Entity store with devices, tree nodes and alarms by id; Local storage with units and panel sizes. Bottom row: Series meta with unit and precision; Ring buffers keyed by handle, holding Float32 values; History cache, an LRU with a byte cap. Arrows: the entity store links devices to series meta by device id; series meta maps to ring buffers by handle and to the history cache by series, resolution and range. A note says the range in the URL picks the resolution.URLsite, device, rangeEntity storedevices, nodes, alarmsLocal storageunits, panel sizesSeries metaunit, precisionRing buffershandle: Float32History cacheLRU, byte capdeviceIdby handleby series, resolution, rangerange in the URL picks the resolution
Entities change a few times a second and are read by many components; values change hundreds of times a second and are read only by the render loop. Keeping them apart is what keeps re-renders cheap.

Note the timeZone on tree nodes. A site has a time zone, and that is what shift reports and alarm times are read in (chapter 11). The precision on series says how many decimals are meaningful, so the UI never shows a temperature as 21.3700008392334.

Interface: HTTP for history and writes, one socket for live

The API splits by what each call does. Anything with a range, a page or a write is HTTP, because it is cacheable, retryable and easy to debug. Anything that changes on its own is pushed over the socket.

http
GET /api/series/chiller-07.supply_temp/buckets?from=2026-09-21T00:00Z&to=2026-10-21T00:00Z&width=1200

200 OK
Cache-Control: private, max-age=60
{
  "resolution": "15m", "bucketMs": 2700000, "unit": "degC",
  "t":   [1789948800000, 1789951500000, ...],
  "min": [5.1, 5.0, null, ...],
  "max": [6.4, 9.8, null, ...],
  "sum": [...], "n": [540, 539, 0, ...]
}
The client sends its plot width; the server picks the rollup and bucket (here 15 m rollups merged into 45 minute buckets, 960 of them). Columns instead of objects compress better and load straight into typed arrays. A null bucket is a gap.
CallPurposeNotes
GET /api/tree?parent=b2Children of a tree node with countsLazy, one level at a time.
GET /api/devices?node=b2&status=offline&sort=lastSeen&cursor=...Server-side filtered, sorted listCursor pagination; returns rows and a total.
GET /api/series/:id/bucketsHistory at the right resolutionAbove; also used for gap fill after reconnect.
GET /api/alarms?state=firing&node=b2Active alarms, grouped by parentChildren included as a count, expandable.
POST /api/alarms/:id/ackAcknowledgeBody: version and note. 409 with current state if someone else acked first.
WS sub and unsubKeyed subscriptionsFrom chapter 3; the reply maps series to handles and includes a snapshot.
WS binary frameLive batchSequence, bucket time, handle, value, flags.
WS alarm, counts, revisionAlways-on eventsSmall JSON; revisions carry series ids and a time range.

Errors follow one shape: { "code": "ALARM_ALREADY_ACKED", "message": ..., "current": {...} }, so the client can show who acted instead of a generic failure. Every live message carries the server time it was sent, which lets the client estimate its own clock offset and compute ages correctly even when the operator's laptop clock is off.

Check your understanding · Exercise 11

Why does the history endpoint take the chart's width in pixels? Select all that apply.

Performance, wall displays and accessibility

A dashboard that is fast for 5 minutes is easy. One that is fast after 30 days on a control room wall is the real target. We set budgets, measure them in the field, and design the wall display mode for failure. Then we make sure charts, alarms and times work for every operator, whatever their eyes, keyboard or time zone.

The live budget

200 ms at the 75th percentile is the "good" threshold for Interaction to Next Paint, one of the Core Web Vitals. The other targets are this design's own estimates.
BudgetTargetHow we stay inside it
Main-thread time for live updatesUnder 100 ms per second (10%)Parse in a worker, one draw per frame, only dirty canvases, text cells patched directly.
Single frame of live workUnder 8 ms (half of a 16.7 ms frame at 60 Hz)Draw at most a few thousand points per chart; spread big redraws over frames.
Live payloadUnder 10 KB/s per screenVisible series only, deltas, binary, 1 s batches.
History requestUnder 1 s for any rangeRollups and one bucket per pixel; at most a few thousand rows.
MemoryFlat over 30 daysFixed-size rings, LRU caches with a byte cap, no growing arrays.
Interaction to Next PaintUnder 200 ms at p75Live work never blocks input; long tree and list renders are virtualized.

Measure in the field, not just on a developer laptop. Report from each console: frames that took over 50 ms (the Long Animation Frames API in Chromium gives scripts and durations), the size of the ring and cache memory we allocated ourselves (we know it exactly, since typed arrays have fixed sizes), messages per second, and seconds since the last frame applied. The runtime-performance lesson covers the tools.

Wall display mode

A control room screen is a browser in kiosk mode that nobody touches for weeks. Everything that is a small leak in a normal tab becomes an outage there. The memory-leaks lesson covers finding leaks; this is how we design so they cannot pile up:

  • Bounded everything: rings are fixed size, the history cache has a byte cap with least-recently-used eviction, the alarm history list keeps the last 500 entries, and every event listener and subscription is removed when its view unmounts.
  • Reconnect forever: backoff caps at 30 s and never stops. A big "Disconnected for 4 min" banner and ageing values make the outage visible from across the room.
  • Keep the screen on: navigator.wakeLock.request("screen"), reacquired on visibilitychange, since the browser releases it when the page is hidden. The Screen Wake Lock API is Baseline since 2025 and needs HTTPS.
  • Planned reload: once a day, at a quiet hour in the site's time zone and only when no alarm is firing, the page reloads itself. This clears anything we did not foresee (browser memory growth, a stale bundle after a deploy). The URL holds the whole view, so the screen comes back exactly as it was.
  • Heartbeat watchdog: if no frame has been applied for 2 minutes while the socket claims to be open, treat the connection as dead and reconnect. Half-open TCP connections are common on long-lived networks.

Think first

Why reload the page on purpose once a day if we designed everything to be bounded?

Check your understanding · Exercise 12

A wall display's memory grows by about 40 MB a day. Which are likely causes in a dashboard like this? Select all that apply.

Accessibility: charts, alarms and the site tree

A canvas chart is a picture, and a colour-coded fleet map is a puzzle for anyone with colour blindness. Operators include people using screen readers, magnifiers and keyboards, and alarm consoles are safety tools, so this part is not optional.

Charts and states that do not depend on sight or colour

  • Text alternative per chart: next to every canvas, a short text summary updated at most every 10 s: "Supply temperature, 6.2 degrees Celsius, 30 s ago. Last hour: min 5.1, max 9.8 at 13:42, rising. 1 alarm firing." Computed from the same buckets the chart draws. A "View as table" button shows the buckets as a real table.
  • Not by colour only: each status has a shape, an icon and a word (chapter 6), and alarm bands on charts use a pattern as well as a colour. The visual-accessibility lesson covers contrast.
  • Announcements with limits: one polite live region for new critical alarms, summarised per batch ("3 new critical alarms in Building 2") and limited to one announcement every few seconds. Never announce value ticks.
  • Keyboard tree: the site tree follows the APG tree view pattern (arrow keys move and expand, typeahead jumps by name), and each node's name includes its roll-up ("Floor 3, 2 alarming, 1 offline"). The aria-patterns lesson and the tree-view guide go deeper.
  • Reduced motion: with prefers-reduced-motion, charts update without scrolling animation and alarm badges do not pulse.

Whose time zone, which unit

A remote operations centre in one city watches plants in three others. "The alarm fired at 02:00" must say whose 02:00. The default is site-local time for anything about a place (shift charts, alarm logs, daily rollups), because that is what people on site use. The operator can switch to their own time zone, and tooltips can show both.

Daily rollups are cut at site-local midnight on the server, so a "daily max" means the same day for everyone, including on days when daylight saving time makes a day 23 or 25 hours long.

Format with Intl.DateTimeFormat and Intl.NumberFormat using the operator's locale and the site's timeZone (see locale-formatting). Units are stored once per series in SI or the device's native unit and converted only for display, with a preference per operator (Celsius or Fahrenheit, bar or psi). Each value is formatted with the series' precision, so 21.37 does not become 21.3700008392334 after a trip through a Float32Array.

Check your understanding · Exercise 13

Which make a live chart usable for a screen reader user? Select all that apply.

Running it in the interview

In 60 minutes you cannot cover everything above. What the interviewer is checking is whether you spot the volume problem early and let it drive the design, and whether you are honest about missing and late data. A plan that works:

MinutesWhat you doWhat to say out loud
0 to 6Scope questions: sensor count, rate, retention, freshness, wall displays, writes."The answer to how many sensors decides whether the browser can ever see raw data."
6 to 12Volume math on the board, next to pixels on a screen."4,000 samples a second in, a few hundred changed values a second useful on one screen. The gap is the design."
12 to 22Architecture: ingest, rollups, alarm engine, BFF, socket worker, stores, render loop."Each boundary is where data gets smaller."
22 to 32Subscriptions and downsampling: min and max, rollup pyramid, width in the request."Averages hide the spike that broke the pump."
32 to 42Live rendering: ring buffers, rAF, canvas, memory math."2,000 series x 8,640 points x 4 bytes is 69 MB, allocated once."
42 to 52Gaps, clocks, late data; alarms with hysteresis and grouping."No data is not zero, and a gateway outage is one alarm, not 400."
52 to 60Transport details, wall displays, accessibility, questions.Pick what the interviewer reacts to.

Draw the volume funnel first, before any boxes. It is a small drawing that shows you understand the problem, and it gives you something to point at every time a later choice comes up. Then draw the architecture from right (devices) to left (pixels).

Follow-up questions to expect

Think first

An engineer wants a "raw mode" where the operator sees every sample of one sensor live at 100 Hz for commissioning. How would you support it?

Think first

The customer grows from 20,000 to 200,000 sensors. What changes in the frontend?

Think first

A site's internet link goes down. Operators on site still need the dashboard. What would you propose?

Summary

  • Do the volume math first: 4,000 to 50,000 samples per second come in, while one screen uses a few thousand numbers to draw and a few hundred changed values per second after that. Never stream the raw firehose to the browser.
  • Subscribe to the screen: visible series, at the form and resolution the view needs, debounced while scrolling, with a short grace period and an immediate snapshot.
  • Downsample to about one bucket per pixel. Use min and max (plus sum and count) so spikes survive and rollups merge; keep LTTB for presentation.
  • Precompute rollups at 1 m, 15 m, 1 h and 1 d, pick one by range / width like a tile pyramid in time, and merge whole groups of its buckets so the client gets at most one per pixel.
  • Live charts are ring buffers of typed arrays drawn on canvas once per frame, paused when hidden. Do the memory math: Float32 for values, one shared time axis.
  • No data is not zero: draw gaps, show the age of every value, use device time to plot and receive time for liveness, and repair late data with revisions.
  • Alarms live on the server with pending periods and hysteresis, grouped by parent to stop storms, acknowledged with versions and synced to every operator.
  • Wall displays need bounded memory, reconnect forever, a wake lock, a heartbeat watchdog and a planned daily reload.

Check your answer

Tick what your own design covered. Pick the level you're interviewing for: each level includes the ones before it.

Junior 0 of 3
Mid-level 0 of 4

References

Concepts behind it

Try next