cf.completefrontend
ALGORITHMS IN FRONTEND WORK · CORE

Running tasks with a concurrency limit

Run many async tasks with at most limit in flight at once, starting the next task as soon as one finishes.

Time O(n) bookkeepingSpace O(n + limit)5 practice problemsStep-through picture

The idea

Uploading 200 photos or prefetching 500 images at once would open hundreds of network requests. Browsers limit connections per server anyway, the server may reject the flood, and nothing finishes quickly. Running them one by one is safe but slow. The middle path is a concurrency limit: at most limit tasks run at the same time.

A neat way to build it is a pool of workers. Start limit workers. Each worker takes the next task index from a shared counter next, waits for that task to finish, and then takes another, until no tasks are left. Because each worker only takes a new task after finishing one, there are never more than limit tasks running.

To predict the timing, simulate it: each task has a duration. At time 0 the first limit tasks start. Whenever a task finishes, the worker that ran it starts the next task right then. The finish times follow from that rule, and a min-heap of finish times makes the simulation fast.

An everyday example

Checkout lanes at a shop

A shop has three checkout lanes and a single line of customers. The first three customers go straight to a lane. When any lane finishes, the next person in line walks to that lane. No lane ever serves two people at once, and nobody waits while a lane is free.

How the example maps to the code
In the exampleIn the code
The checkout lanesThe limit workers
The single line of customerstasks, taken in order
"Next, please"const index = next++
Serving one customerawait tasks[index]()
The shop closes when the line is emptyawait Promise.all(workers)

From the slow way to running tasks with a concurrency limit

  1. Start with the two obvious ways. Promise.all(tasks.map((task) => task())) starts everything at once: 200 uploads open 200 requests, the server may reject the flood, and over HTTP/1.1 the browser queues most of them anyway. A for loop with await runs one task at a time: safe, but the total time is the sum of every duration.
  2. The middle path is to run limit at a time. The first idea is batches: start limit tasks, wait for all of them, then start the next batch. But each batch waits for its slowest task while the other slots sit empty. In the trace below, batches of 3 would take 250 + 300 = 550 ms.
  3. The better rule is "as soon as a slot is free, fill it". Picture limit checkout lanes and one line of tasks: whenever a lane finishes, it calls the next task in line. So write a worker that loops: take the next task, wait for it, take another, until none are left. Start limit workers.
  4. The workers share one counter, next, the index of the first task nobody has taken. Each result goes into results[index], so the output keeps the input's order even though tasks finish in a different order. Waiting for every worker with Promise.all(workers) means every task is done. That pool of workers is running tasks with a concurrency limit.

How it works

  1. Make results with one slot per task and a shared counter next = 0.
  2. Define worker: while next is less than the number of tasks, take index = next++, await tasks[index](), and store the value in results[index].
  3. Start Math.min(limit, tasks.length) workers and keep their promises in workers.
  4. Wait for all workers with Promise.all(workers). Each worker only stops when no tasks are left.
  5. Return results, which is in the original task order, not the finishing order.

Watch it run

This picture runs the code below on a real input. Press Step to move one step at a time, or Play to watch. The highlighted line is the one that just ran. Change the input to see other cases.

Loading the picture…
See every step at onceThe same run, laid out side by side
Drawing every step…

The code

Point at a note to light up the lines it explains.

JavaScriptOpen in the code editor (opens in a new tab)
  1. async function runWithLimit(tasks, limit) {
  2. const results = new Array(tasks.length);
  3. let next = 0;
  4. async function worker() {
  5. while (next < tasks.length) {
  6. const index = next++;
  7. results[index] = await tasks[index]();
  8. }
  9. }
  10. const workers = [];
  11. for (let w = 0; w < Math.min(limit, tasks.length); w++) {
  12. workers.push(worker());
  13. }
  14. await Promise.all(workers);
  15. return results;
  16. }

How fast it is

TimeO(n) bookkeeping
Extra memoryO(n + limit)

Each task is handed out once and its result stored once. The wall-clock time depends on the durations: roughly the total work divided by limit, but never less than the longest task.

  • Best caseO(n)

    The bookkeeping is always one hand-out and one stored result per task. The wall-clock time is best when tasks have equal durations and n is a multiple of limit: then it's exactly the total work divided by limit.

  • Average caseO(n)

    Mixed durations, like the trace: the wall-clock time lands between total / limit and that plus the longest task.

  • Worst caseO(n)

    Still O(n) bookkeeping. The wall-clock worst case is a long task handed out last: [100, 100, 100, 100, 400] with limit = 2 ends at 600, while putting the 400 first would end at 400.

Common mistakes

  • Calling all the task functions first, as in tasks.map((t) => t()), and then limiting. The requests have already started; pass functions that start work when called.
  • Splitting tasks into fixed batches of limit and awaiting each batch. One slow task holds up the whole batch while other slots sit idle.
  • Pushing results in finishing order with results.push(value). The order then changes from run to run; store by index.
  • Letting one failure stop everything without meaning to. Decide whether to fail fast or collect errors per task.

When to use it

Look for these clues in a problem:

  • Many requests, uploads, or jobs must run, but only a few at a time.
  • The question gives task durations and a limit, and asks for finish times or the total time.
  • Results must stay in the original order even though tasks finish out of order.
  • It mentions a pool, a queue of jobs, rate limits, or Promise.all with a limit.

Check yourself

Answer each one in your head first, then open it.

Why can't two workers both take task 3?

JavaScript runs one piece of code at a time, and a worker only pauses at await. Reading next and adding one happen together, so the second worker always sees the new value.

Why do fixed batches take 550 ms in the trace's example while the pool takes 400 ms?

A batch waits for its slowest task: 250 ms for the first batch and 300 ms for the second. The pool starts a new task the moment any slot is free, so short tasks fill the gaps.

With limit = 1, how long does the trace's example take?

950 ms, the sum of all durations, because one worker runs every task in turn.

What is the shortest possible total time for any limit?

At least the longest single task, and at least the total work divided by limit, whichever is larger. For the trace that's max(300, 950 / 3), about 317 ms; the pool got 400.

Practice problems

Stuck on one? Tuto is in the playground too, and gives you hints, never the answer.

Not started

0 of 5 solved. Solve one to earn the bronze medal, half for silver, and all of them for gold.

Start practicing

easy

medium

hard

Go further

You don't need these to start practicing. Open one when you want more.

Why it works

The reasoning that makes it always right

At most limit tasks run at once, because there are only Math.min(limit, tasks.length) workers, and a worker takes a new task only after await gives back the result of its current one. No worker ever runs two tasks at the same time.

Each task runs exactly once, because of how JavaScript runs code: between two awaits, a worker runs without anyone else running. The check next < tasks.length, the read of next, and the ++ all happen in one such stretch, so two workers can never get the same index. A worker stops only when every index has been handed out, so no task is skipped.

The answer is complete and in order. Each result is stored at its own index, not pushed in finishing order. A worker's promise resolves only after its last task has finished, so when Promise.all(workers) resolves, every task is done and every slot of results is filled. And no slot sits idle while a task is waiting, because a worker takes the next task the moment its last one ends.

Going deeper

3 more ideas

Why next++ never gives a task to two workers

JavaScript runs one piece of code at a time. Between two awaits, a worker runs without being interrupted, so reading next and adding one happen together. Two workers can't both read the same value, so every index is taken exactly once.

Simulating it with a heap

To compute finish times for n tasks, keep a min-heap of the times when each worker becomes free, starting with limit zeros. For each task in order, take the earliest free time t, the task finishes at t + duration, and put that back. That is O(n log limit). With a small limit, scanning the workers' free times is simpler and just as fast.

Errors and cancellation

If a task throws, this version rejects the whole Promise.all while other workers keep going. Real code often catches each error and stores it in results, like Promise.allSettled. Passing an AbortSignal lets the caller stop the remaining tasks.

Work it by hand

Every step in a table, with why

Run 6 tasks with 3 workers. Short tasks free their worker early, so worker 2 runs three tasks while worker 0 runs one.

durations = [250, 100, 150, 300, 50, 100] (task `i` waits that many ms, then returns `ri`), limit = 3

StepTimeEventnextRunning (worker: task)Why
10worker 0 takes task 01w0: 0The first worker starts. index = next++ hands out task 0, which will end at 250.
20worker 1 takes task 12w0: 0, w1: 1Task 1 will end at 100.
30worker 2 takes task 23w0: 0, w1: 1, w2: 2Math.min(3, 6) is 3, so this is the last worker. All 3 slots are busy.
4100task 1 done, worker 1 takes task 34w0: 0, w1: 3, w2: 2results[1] is stored and worker 1 loops at once. Task 3 is the longest, 300 ms, ending at 400.
5150task 2 done, worker 2 takes task 45w0: 0, w1: 3, w2: 4Task 4 is short and will end at 200.
6200task 4 done, worker 2 takes task 56w0: 0, w1: 3, w2: 5next is now 6, equal to tasks.length: every task has been handed out.
7250task 0 done, worker 0 stops6w1: 3, w2: 5next < tasks.length is false, so worker 0's loop ends and its promise resolves.
8300task 5 done, worker 2 stops6w1: 3Worker 2 ran three tasks: 2, 4, and 5.
9400task 3 done, worker 1 stops6noneThe last worker finishes, so Promise.all(workers) resolves.

It returns [r0, r1, r2, r3, r4, r5] at 400 ms, in task order even though the finish order was 1, 2, 4, 0, 5, 3. One at a time would take 950 ms, and fixed batches of 3 would take 550 ms.

Compared with other ways

When to pick something else
ApproachTimeExtra memoryChoose it when
Running tasks with a concurrency limitO(n)O(n)Many requests or jobs that must not all run at once, where you want every slot busy until the work runs out.
Promise.all on everythingO(n)O(n)A small number of tasks, like 5 requests, where starting them all at once is fine and fastest.
Fixed batches of limitO(n)O(n)The API itself takes work in batches, or each batch must finish before the next can start. Otherwise slow tasks leave slots idle.
One at a time with await in a loopO(n)O(n)Each task depends on the one before, or the server allows only one request at a time. The total time is the sum of all durations.

Extra memory: O(n + limit): the results array plus one waiting promise per worker.

How the steps grow

1101001k10k100k1M116324864input size n
Steps as the input grows, for Running tasks with a concurrency limit and the other approaches, on a scale where each line up is 10 times more steps.
  • Running tasks with a concurrency limit, Promise.all on everything, Fixed batches of limit, One at a time with await in a loop O(n)
About how many steps for bigger inputs
Approachn = 10n = 1,000n = 1,000,000
Running tasks with a concurrency limit101,0001 million
Promise.all on everything101,0001 million
Fixed batches of limit101,0001 million
One at a time with await in a loop101,0001 million

Inputs that need care

6 tricky inputs
No tasks: tasks = []
No workers start, Promise.all([]) resolves at once, and it returns [].
limit larger than the number of tasks
Math.min(limit, tasks.length) starts one worker per task, so they all run at once and each worker stops after one task.
limit = 0 or a negative limit
No worker starts, so no task ever runs, and it returns an array of empty slots. Guard with Math.max(1, limit) or throw an error.
A fractional limit like 2.5
The loop runs for w = 0, 1, and 2, so 3 workers start. Round the limit down first if 2 was meant.
One task throws
That worker's promise rejects, so Promise.all(workers) rejects with the error right away. The other workers keep starting tasks in the background, and the failed worker takes no more. Catch errors per task if you need every result.
Passing promises instead of functions: tasks = urls.map((url) => fetch(url))
The requests already started when the array was built, so the limit does nothing, and tasks[index]() throws a TypeError because a promise isn't a function. Pass () => fetch(url) instead.

Variations and follow-ups

Twists interviewers add

A limiter you can add to at any time

Libraries like p-limit return a function: limit(() => fetch(url)). Each call goes into a queue, and whenever fewer than limit tasks are running, the next one in the queue starts. It's the same rule, but tasks can arrive while others run.

Collecting errors

Wrap each task in try/catch and store { status: "fulfilled", value } or { status: "rejected", reason } at results[index], like Promise.allSettled. One failed upload then doesn't hide the results of the others.

Priorities

Take tasks from a priority queue instead of in index order. An image loader can then start the images that are on screen before the ones far below.

Predicting the finish time

To compute when everything ends without running anything, keep a min-heap of the times each worker becomes free, starting with limit zeros. For each task in order, take the earliest time t and put back t + duration. That is O(n log limit); on the trace it gives 400.

In an interview

Spot it, explain it, answer follow-ups
  • Spot it from "at most N at a time", "limit concurrency", "a pool of workers", or "Promise.all but not all at once". If it says "at most N per second", that's a rate limit, which is a different tool.
  • Explain the worker loop and the shared next counter, then say why there is no race: JavaScript doesn't switch between workers in the middle of next++.
  • Test by hand by drawing a timeline like the trace, with durations that make one worker run several short tasks. Check the finish time and that results are in task order.
  • Expect follow-ups: "what if a task fails?" (fail fast, or collect each result like Promise.allSettled), "how would you cancel?" (pass an AbortSignal and stop taking tasks once it's aborted), and "when does everything finish?" (simulate with a min-heap).

Where frontend code uses it

The same idea in real UI code
  • Upload widgets send a few files at a time so progress stays smooth and the server isn't flooded.
  • Libraries like p-limit and p-queue implement this pattern, and image prefetchers and crawlers use it to cap open requests.
  • Browsers have their own limits: over HTTP/1.1 they open about 6 connections per host, and they queue extra requests the same way.

Related algorithms