Skip to content
goflag
Documentation
Rule catalogue

robots.blocks-page

A page asking to be indexed must not be disallowed by robots.txt

errorsite-widevendor-spec

Why it matters

The quiet version of the site-wide block. A Disallow added years ago for a reason that made sense then, still shadowing a section that has since been given pages asking to be found. Nothing in a browser shows it: the page loads perfectly for you and is never fetched by a crawler.

What the finding looks like

The message goflag prints, with example values substituted.

error robots.blocks-page Page declares <meta name="robots" content="index"> but robots.txt line 4 disallows /blog for *. robots.txt wins: the page is never fetched, so the tag asking for it is never read.

How to fix it

The catalogue carries no snippet for a site rule: a cross-page rule builds its remedy from what the crawl found, so goflag attaches the fix to the finding itself rather than to the rule. Run it and read the finding.

Says who?

This rule is vendor-spec, and it cites the documents below. Nothing here is goflag’s opinion of what a good page looks like — if you disagree with a finding, this is what you are disagreeing with.