The page https://docs.freebsd.org/robots.txt no longer exists.
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/doc/commit/?id=7808bd27abab51b4602005cdcd593ac18e59256c commit 7808bd27abab51b4602005cdcd593ac18e59256c Author: Sergio Carlavilla Delgado <carlavilla@FreeBSD.org> AuthorDate: 2021-01-26 19:12:37 +0000 Commit: Sergio Carlavilla Delgado <carlavilla@FreeBSD.org> CommitDate: 2021-01-26 19:17:05 +0000 Enable robots.txt in docs.freebsd.org Enable again robots.txt for docs.freebsd.org PR: 253031 Submitted by: wosch@ documentation/config/_default/config.toml | 1 + documentation/themes/beastie/layouts/robots.txt (new) | 19 +++++++++++++++++++ 2 files changed, 20 insertions(+)
Fixed. Thanks for the report.