Deviopath › Next.js

robots.ts

Easy · SEO

A file app/robots.ts can default-export a function that returns a MetadataRoute.Robots object. Next.js turns it into /robots.txt, which tells search engines what they may crawl.

Write robots(isProduction, baseUrl):

- In production, return { rules: { userAgent: "*", allow: "/", disallow: "/admin" }, sitemap: baseUrl + "/sitemap.xml" }.

- Otherwise (preview or staging sites that should not appear in Google), return { rules: { userAgent: "*", disallow: "/" } } with no allow and no sitemap keys.

Simplified: the real function takes no arguments and usually reads these values from environment variables.

Next: sitemap.ts