The N+1 problem is a common way of sending far more requests than you need. You fetch a list of N items with one request, then each item fetches something related to it with a request of its own: 1 for the list, plus N more. A page of 50 blog posts that each show their author's name sends 51 requests, where 2 would do: one for the posts, and one for all their authors. It happens because the code that shows a post knows only about that post, so it asks for that one author. The name comes from databases, where it means one query for a list of rows, then one more query for each row.
A DataLoader fixes it by batching: merging many small requests into one. Every caller still asks for one key, such as a user ID, with load(key), and gets back a promise of that one value. The loader doesn't send anything straight away. It collects the keys loaded in the same tick, sends them in one request for many keys, then hands each caller its own result. Each piece of code stays as simple as if it were alone, and the loader does the merging. It's like a waiter who takes the whole table's order before going to the kitchen, rather than walking there and back for each person.
In plain words, the same tick means all the code that runs, one line after another, before control goes back to the event loop, which picks the next thing to run: a timer, a click, a response that has arrived. Loads made in a loop, or in a map whose promises go to Promise.all, all happen in the same tick, so they land in the same batch, and the loader sends it once that code has finished. A load made later, after awaiting a response or in a click handler, runs in a later tick and goes in a new batch. Here are three posts, by authors 7, 9, and 7, each loading its author:
| Step | What happens | Requests for authors |
|---|---|---|
| The code loops over the posts | It calls load(7), load(9), and load(7). Each call gets a promise back at once. | 0: the loader is collecting keys |
| That code finishes | The loader calls the batch function once, with [7, 9]. Two callers want 7, but it's asked for once. | 1: /api/users?ids=7,9 |
| The response arrives | The batch function fulfills with [user 7, user 9]. | 1 |
| Each caller gets its result | Both load(7) promises fulfill with user 7, and load(9) with user 9. | 1, where loading each author on its own sends 3 |
A click later calls load(12) | It's a new tick, so it's a new batch: [12]. | 2 |
You give a loader its batch function when you create it. The batch function gets an array of keys and must fulfill with an array of values, one per key, in the same order: the value at position 0 is for the key at position 0, and so on. That's how the loader knows which result belongs to which caller. Servers don't always answer that way: they may send back what they found in their own order, and leave out what they didn't find. So the batch function puts the results in key order first:
// Stands in for one request, /api/users?ids=7,9,12: the server sends back
// the users it found, in its own order, and leaves out the ones it didn't
const fetchUsers = async (ids) => [{ id: 9, name: "Grace" }, { id: 7, name: "Ada" }];
// The batch function: many keys in, one value per key out, in the keys' order
const batchUsers = async (ids) => {
const found = await fetchUsers(ids);
return ids.map((id) => found.find((user) => user.id === id) ?? null);
};
batchUsers([7, 9, 12]).then(console.log);
// [{ id: 7, name: "Ada" }, { id: 9, name: "Grace" }, null]
You meet loaders in GraphQL servers. There, each field of the response is filled in by its own function, a resolver, which runs once for every object that needs the field. A query for 50 posts and their authors runs the author resolver 50 times, each knowing only its own post: the classic N+1 problem. The usual fix is the dataloader package on npm, released by Facebook, which gave the pattern its name. It also caches each key's result for as long as the loader lives, so servers usually create new loaders for every incoming request.
Write createLoader(batchFn, options). batchFn(keys) takes an array of keys and returns a promise of an array of values, one per key, in the same order, like a function that fetches many users in one request. createLoader returns an object with one method, load(key), which returns a promise of the value for one key.
load doesn't call batchFn right away. Every load made while the current code runs joins one batch. Once that code has finished, and before any timer runs, batchFn is called once with the batch's keys, in the order they were first loaded. Each load then fulfills with the value at its key's position.
A load made after a batch has been sent starts a new batch, even while the first one is still in flight. Nothing is cached between batches, so loading a key again later asks for it again.
A batch asks for each key once: when a key is loaded twice in the same batch, both calls get its value. Keys are compared the way Map compares them.
If a value in the array is an Error, that key's load rejects with it, and the others still fulfill. If batchFn rejects or throws, every load in the batch rejects with its error. If it fulfills with anything but an array with one value per key, every load in the batch rejects with a TypeError.
options.maxBatchSize is optional. With it, a batch with more keys than that is split into several batchFn calls of at most maxBatchSize keys each, in order.
const users = createLoader(async (ids) => {
console.log("fetching", ids);
const found = await fetchJson("/api/users?ids=" + ids.join(","));
return ids.map((id) => found.find((user) => user.id === id) ?? new Error("No user " + id));
});
const [ada, grace, again] = await Promise.all([users.load(1), users.load(2), users.load(1)]);
// logs "fetching [1, 2]" once; ada and again are user 1, grace is user 2
await users.load(3); // a new batch: logs "fetching [3]"
await users.load(99); // logs "fetching [99]", then rejects with Error: No user 99
const batchFn = spy(async (keys) => keys.map((key) => key * 10));
const loader = createLoader(batchFn);
loader.load(1);
loader.load(2);
loader.load(3);
expect(batchFn).not.toHaveBeenCalled();
await flush();
expect(batchFn).toHaveBeenCalledTimes(1);
expect(batchFn).toHaveBeenCalledWith([1, 2, 3]);
useFakeTimers(); // timers only run when the test moves the clock, and it doesn't
const batchFn = spy(async (keys) => keys);
const loader = createLoader(batchFn);
loader.load("a");
loader.load("b");
await flush();
expect(batchFn).toHaveBeenCalledWith(["a", "b"]);
const loader = createLoader(async (keys) => keys.map((key) => "user " + key));
const results = await Promise.all([loader.load(3), loader.load(1), loader.load(2)]);
expect(results).toEqual(["user 3", "user 1", "user 2"]);
const clock = useFakeTimers();
const batchFn = spy((keys) => new Promise((resolve) => setTimeout(resolve, 100, keys)));
const loader = createLoader(batchFn);
const first = loader.load(1);
await flush();
const later = [loader.load(2), loader.load(1)]; // the first batch is still in flight
await flush();
expect(batchFn).toHaveBeenCalledTimes(2);
expect(batchFn.calls[1].args[0]).toEqual([2, 1]);
await clock.tickAsync(100);
await expect(first).resolves.toBe(1);
await expect(Promise.all(later)).resolves.toEqual([2, 1]);
const batchFn = spy(async (keys) => keys.map((key) => "user " + key));
const loader = createLoader(batchFn);
const results = await Promise.all([loader.load(1), loader.load(2), loader.load(1)]);
expect(batchFn).toHaveBeenCalledTimes(1);
expect(batchFn).toHaveBeenCalledWith([1, 2]);
expect(results).toEqual(["user 1", "user 2", "user 1"]);
const missing = new Error("No user 2");
const loader = createLoader(async (keys) => keys.map((key) => (key === 2 ? missing : "user " + key)));
const [one, two, three] = await Promise.allSettled([loader.load(1), loader.load(2), loader.load(3)]);
expect(one).toEqual({ status: "fulfilled", value: "user 1" });
expect(two.status).toBe("rejected");
expect(two.reason).toBe(missing);
expect(three).toEqual({ status: "fulfilled", value: "user 3" });
const error = new Error("database down");
const rejecting = createLoader(async () => {
throw error;
});
const throwing = createLoader(() => {
throw error;
});
const results = await Promise.allSettled([rejecting.load(1), rejecting.load(2), throwing.load(1), throwing.load(2)]);
for (const result of results) expect(result.reason).toBe(error);
const tooFew = createLoader(async (keys) => keys.slice(1));
const notArray = createLoader(async () => "user 1");
const results = await Promise.allSettled([tooFew.load(1), tooFew.load(2), notArray.load(1)]);
for (const result of results) expect(result.reason).toBeInstanceOf(TypeError);
const batchFn = spy(async (keys) => keys.map((key) => key * 10));
const loader = createLoader(batchFn, { maxBatchSize: 2 });
const results = await Promise.all([1, 2, 3, 4, 5].map((key) => loader.load(key)));
expect(batchFn.calls.map((call) => call.args[0])).toEqual([[1, 2], [3, 4], [5]]);
expect(results).toEqual([10, 20, 30, 40, 50]);
Run tests runs every test on your code, here in your browser. The problem is solved when they all pass.
Ctrl + Enter runs the tests (⌘ + Enter on a Mac).
I'm here to help with DataLoader-style batch loader. I give hints, never the solution, so you get to solve it yourself. Ask for a hint, select code to ask about it, or press Ask Tuto why on a failing test.
Your first 3 answers need no account. for free to keep asking.