/sitemap.xml and /robots.txt
What a crawler is pointed at, and what it is kept out of.
Response
/sitemap.xml lists the default version pages, per locale, with hreflang
alternates. /robots.txt is generated beside it and points at the sitemap.
Notes
Only the default version is listed. Older versions stay reachable and carry
canonical plus noindex — a search result that lands a reader on last year
documentation is the failure this pair exists to prevent, and the
version banner is its visible
half.
A version marked eol leaves the sitemap entirely.
Was this page helpful?