An old public/robots.txt is still there after you add app/robots.ts. What does next build print, and what does /robots.txt answer?
Leaf and Kettle moved to the App Router and wrote app/robots.ts, which keeps crawlers out of /checkout and names the new sitemap. Nobody deleted the old site's public/robots.txt. Does the build complain, and which file does a crawler get?
app/layout.tsxexport const metadata = { title: "Leaf and Kettle" };
export default function RootLayout({ children }: LayoutProps<"/">) {
return (
<html lang="en">
<body>{children}</body>
</html>
);
}
app/page.tsxexport default function HomePage() {
return (
<main>
<h1>Leaf and Kettle</h1>
<p>Loose-leaf teas, packed by hand.</p>
</main>
);
}
app/robots.tsimport type { MetadataRoute } from "next";
export default function robots(): MetadataRoute.Robots {
return {
rules: { userAgent: "*", allow: "/", disallow: "/checkout" },
sitemap: "https://leafandkettle.example/sitemap.xml",
};
}
public/robots.txt# Leaf and Kettle, old site
User-agent: *
Disallow: /basket
It's a project create-next-app made (Next.js 16.4.3, TypeScript, the App Router, and Cache Components on) with these files, built with next build and run with next start. What happens:
- You run
next build. - You start the app with
next startand ask for/robots.txt.
The answer
Work out what Next.js does at each step: what the build prints, what each request answers, and what the page shows. Then check.
Checking your sign-in…