Skip to the code

Rate-limit the newsletter sign-up

Bots have found Leaf and Kettle's newsletter form and post hundreds of fake sign-ups a minute. Add a rate limit to the POST handler in app/api/newsletter/route.ts, which takes { "email": "..." }, adds it with subscribe(email) from @/lib/newsletter, and answers { subscribed: email }, or 400 with { error: "Enter a valid email" } for an email it can't use. The starter already does that part.

Each client may make 5 requests per window of 60 seconds. A client is the first address in the X-Forwarded-For header, trimmed, so 203.0.113.7, 10.0.0.1 is 203.0.113.7; requests without that header all count as one client. A client's window starts with its first request, and the first request 60 seconds or more after that starts a new window, with a fresh count.

Check the limit before you read the body, so every request counts, a bad one too. A request within the limit gets its usual answer, plus the headers X-RateLimit-Limit: 5 and X-RateLimit-Remaining, how many requests the client has left in this window: 4 after its first, 0 after its fifth.

A request over the limit gets 429 Too Many Requests with { error: "Too many sign-ups, try again later", retryAfter }, where retryAfter is the number of seconds until the window ends, rounded up. Send the same number in a Retry-After header, along with X-RateLimit-Limit: 5 and X-RateLimit-Remaining: 0. A refused request isn't counted and doesn't move the window.

POST /api/newsletter    X-Forwarded-For: 203.0.113.7
(its 6th request, 20 seconds after its 1st)
→ 429 { error: "Too many sign-ups, try again later", retryAfter: 40 }
  Retry-After: 40    X-RateLimit-Limit: 5    X-RateLimit-Remaining: 0

The app's other files

lib/newsletter.ts, read-only. Import it as @/lib/newsletter.

/** Everyone signed up so far. */
export const subscribers: string[] = [];
 
/** Adds an email to the newsletter, once. */
export async function subscribe(email: string) {
  if (!subscribers.includes(email)) subscribers.push(email);
}

What the tests check

  1. lets five requests through, counting down X-RateLimit-Remaining
    useFakeTimers();
    for (let n = 1; n <= 5; n++) {
      const email = "reader" + n + "@example.com";
      const response = await POST(
        request("/api/newsletter", { method: "POST", headers: { "content-type": "application/json", "x-forwarded-for": "203.0.113.7" }, body: JSON.stringify({ email }) }),
      );
      expect(response.status).toBe(200);
      expect(await response.json()).toEqual({ subscribed: email });
      expect(response.headers.get("x-ratelimit-limit")).toBe("5");
      expect(response.headers.get("x-ratelimit-remaining")).toBe(String(5 - n));
    }
    expect(subscribers.length).toBe(5);
    
  2. answers the sixth with 429 and how long to wait, rounded up
    const clock = useFakeTimers();
    const send = (email) =>
      POST(request("/api/newsletter", { method: "POST", headers: { "content-type": "application/json", "x-forwarded-for": "203.0.113.7" }, body: JSON.stringify({ email }) }));
    for (let n = 1; n <= 5; n++) await send("reader" + n + "@example.com");
    clock.tick(20_000);
    const response = await send("late@example.com");
    expect(response.status).toBe(429);
    expect(await response.json()).toEqual({ error: "Too many sign-ups, try again later", retryAfter: 40 });
    expect(response.headers.get("retry-after")).toBe("40");
    expect(response.headers.get("x-ratelimit-limit")).toBe("5");
    expect(response.headers.get("x-ratelimit-remaining")).toBe("0");
    clock.tick(39_500);
    expect((await send("later@example.com")).headers.get("retry-after")).toBe("1");
    expect(subscribers).not.toContain("late@example.com");
    expect(subscribers).not.toContain("later@example.com");
    
  3. starts a new window 60 seconds after the client's first request
    const clock = useFakeTimers();
    const send = (email) =>
      POST(request("/api/newsletter", { method: "POST", headers: { "content-type": "application/json", "x-forwarded-for": "203.0.113.7" }, body: JSON.stringify({ email }) }));
    await send("a@example.com");
    clock.tick(30_000);
    for (const email of ["b@example.com", "c@example.com", "d@example.com", "e@example.com"]) await send(email);
    clock.tick(29_999);
    expect((await send("f@example.com")).status).toBe(429);
    clock.tick(1);
    const fresh = await send("g@example.com");
    expect(fresh.status).toBe(200);
    expect(fresh.headers.get("x-ratelimit-remaining")).toBe("4");
    
  4. refused requests don't push the window back
    const clock = useFakeTimers();
    const send = (email) =>
      POST(request("/api/newsletter", { method: "POST", headers: { "content-type": "application/json", "x-forwarded-for": "203.0.113.7" }, body: JSON.stringify({ email }) }));
    for (let n = 1; n <= 5; n++) await send("reader" + n + "@example.com");
    for (let second = 10; second <= 50; second += 10) {
      clock.tick(10_000);
      const response = await send("bot" + second + "@example.com");
      expect(response.status).toBe(429);
      expect(response.headers.get("retry-after")).toBe(String(60 - second));
    }
    clock.tick(10_000);
    const response = await send("real@example.com");
    expect(response.status).toBe(200);
    expect(response.headers.get("x-ratelimit-remaining")).toBe("4");
    
  5. counts each client by the first address in X-Forwarded-For
    useFakeTimers();
    const from = (forwardedFor, email) => {
      const headers = { "content-type": "application/json" };
      if (forwardedFor !== null) headers["x-forwarded-for"] = forwardedFor;
      return POST(request("/api/newsletter", { method: "POST", headers, body: JSON.stringify({ email }) }));
    };
    for (let n = 1; n <= 5; n++) await from("203.0.113.7", "reader" + n + "@example.com");
    expect((await from("203.0.113.7, 10.0.0.1", "proxied@example.com")).status).toBe(429);
    const other = await from("198.51.100.2", "other@example.com");
    expect(other.status).toBe(200);
    expect(other.headers.get("x-ratelimit-remaining")).toBe("4");
    expect((await from("198.51.100.2 , 10.0.0.1", "other2@example.com")).headers.get("x-ratelimit-remaining")).toBe("3");
    expect((await from(null, "nobody@example.com")).headers.get("x-ratelimit-remaining")).toBe("4");
    expect((await from(null, "nobody2@example.com")).headers.get("x-ratelimit-remaining")).toBe("3");
    
  6. checks the limit before the body, and counts bad requests too
    useFakeTimers();
    const send = (body) =>
      POST(request("/api/newsletter", { method: "POST", headers: { "content-type": "application/json", "x-forwarded-for": "203.0.113.7" }, body }));
    for (let n = 1; n <= 5; n++) {
      const response = await send(n % 2 ? "not json" : JSON.stringify({ email: "no-at-sign" }));
      expect(response.status).toBe(400);
      expect(await response.json()).toEqual({ error: "Enter a valid email" });
      expect(response.headers.get("x-ratelimit-remaining")).toBe(String(5 - n));
    }
    const response = await send(JSON.stringify({ email: "real@example.com" }));
    expect(response.status).toBe(429);
    expect(subscribers).toEqual([]);
    

Learn more

app/api/newsletter/route.tsSaved in this browser

Run tests runs every test on your code, here in your browser. The problem is solved when they all pass.

Ctrl + Enter runs the tests (⌘ + Enter on a Mac).