completefrontend

Predict the outputMedium

An old public/robots.txt is still there after you add app/robots.ts. What does next build print, and what does /robots.txt answer?

Leaf and Kettle moved to the App Router and wrote app/robots.ts, which keeps crawlers out of /checkout and names the new sitemap. Nobody deleted the old site's public/robots.txt. Does the build complain, and which file does a crawler get?

app/layout.tsx
export const metadata = { title: "Leaf and Kettle" };
 
export default function RootLayout({ children }: LayoutProps<"/">) {
  return (
    <html lang="en">
      <body>{children}</body>
    </html>
  );
}
app/page.tsx
export default function HomePage() {
  return (
    <main>
      <h1>Leaf and Kettle</h1>
      <p>Loose-leaf teas, packed by hand.</p>
    </main>
  );
}
app/robots.ts
import type { MetadataRoute } from "next";
 
export default function robots(): MetadataRoute.Robots {
  return {
    rules: { userAgent: "*", allow: "/", disallow: "/checkout" },
    sitemap: "https://leafandkettle.example/sitemap.xml",
  };
}
public/robots.txt
# Leaf and Kettle, old site
User-agent: *
Disallow: /basket

It's a project create-next-app made (Next.js 16.4.3, TypeScript, the App Router, and Cache Components on) with these files, built with next build and run with next start. What happens:

  1. You run next build.
  2. You start the app with next start and ask for /robots.txt.

The answer

Work out what Next.js does at each step: what the build prints, what each request answers, and what the page shows. Then check.

Checking your sign-in…