robots.txt blocks Googlebot
high Crawlability robots_blocks_google
What we check
We evaluate your robots.txt for the Googlebot user agent against the homepage path.
Why it matters
If Googlebot may not fetch the homepage, the site cannot be crawled normally and pages fall out of results over time. This is a common leftover from a staging site.
How to fix it
Remove the Disallow: / that applies to Googlebot or *. Keep blocks narrow (specific folders), never site-wide on production.
How to confirm the fix
Paste the file into the robots.txt tester with the Googlebot agent and path /. Glossary: Crawlability.
In the API response
When this check fails, output.technical_issues contains an item with "id": "robots_blocks_google", the affected URLs as evidence and the fix. If it is among the highest-priority problems it also appears in actions (as finding_id) and in plan.
Related checks
Run the full audit on your site. One call checks every item in the check reference and ranks the fixes.
Open the playground Endpoint details