Split a 120,000-page marketplace into sitemaps of 50,000 URLs with generateSitemaps
Leaf and Kettle's marketplace lists teas from partner farms, each at /market/<id>, and there are about 120,000 listings. One sitemap may hold at most 50,000 URLs, so app/market/sitemap.ts must become several: /market/sitemap/0.xml, /market/sitemap/1.xml, and so on. Read the listings with @/lib/market.
Export generateSitemaps, which returns one { id } for every 50,000 listings or part of them, numbered from 0. It asks the database for the count only, with countListings(), and loads no listings.
The default export gets { id }, and returns that sitemap's entries: url, the listing's full URL on SITE_URL, and lastModified, its updatedAt. Sitemap 0 has the first 50,000 listings in id order, sitemap 1 the next 50,000, and the last one whatever is left; no listing is in two sitemaps. Each sitemap asks getListings for its own listings, once, and asks the database nothing else.
await generateSitemaps();
// [{ id: 0 }, { id: 1 }, { id: 2 }]
await sitemap({ id: Promise.resolve("1") });
// [
// { url: "https://leafandkettle.example/market/50001", lastModified: … },
// …
// { url: "https://leafandkettle.example/market/100000", lastModified: … },
// ]
app/market/sitemap.tsimport type { MetadataRoute } from "next";
import { SITE_URL, countListings, getListings } from "@/lib/market";
export default async function sitemap(): Promise<MetadataRoute.Sitemap> {
const listings = await getListings(0, await countListings());
return listings.map((listing) => ({
url: `${SITE_URL}/market/${listing.id}`,
lastModified: listing.updatedAt,
}));
}
lib/market.ts (read-only)export const SITE_URL = "https://leafandkettle.example";
/** The most URLs one sitemap may list. */
export const URLS_PER_SITEMAP = 50_000;
export type Listing = { id: number; updatedAt: Date };
/** The marketplace's database: every listing in id order, and every query it was asked. The tests change it. */
export const db: { listings: Listing[]; queries: string[] } = {
listings: Array.from({ length: 120_000 }, (_, index) => ({
id: index + 1,
updatedAt: new Date(Date.UTC(2026, 9, 1 + (index % 30))),
})),
queries: [],
};
/** How many listings there are, without loading any. */
export async function countListings(): Promise<number> {
db.queries.push("count");
return db.listings.length;
}
/** At most `limit` listings, in id order, after skipping the first `offset`. */
export async function getListings(offset: number, limit: number): Promise<Listing[]> {
db.queries.push(`offset ${offset}, limit ${limit}`);
return db.listings.slice(offset, offset + limit);
}
- generateSitemaps lists one sitemap for every 50,000 listings or part of them, numbered from 0
- generateSitemaps counts the listings without loading any
- sitemap 0 lists the first 50,000 listings by their full URL and date
- reads its id from a promise of a string, as Next.js passes it
- the last sitemap has the rest, and every listing is in exactly one sitemap
- each sitemap asks the database only for its own listings
The answer
Write it yourself first. Then compare it with a reference solution and the checks it passes.
Checking your sign-in…