Generate or test your robots.txt
Two modes: generate a robots.txt from scratch, or paste the one you already have and check whether a specific path is allowed or blocked.
What you type here never leaves your browser.
Quick presets
Your robots.txt
User-agent: *
How to use it
In Generate, the code below updates as you build your rules; copy it or download it once it's ready. In Test, paste your current file, give me a path and a user-agent, and I'll tell you which rule wins and why, following the same precedence order crawlers actually use.
Limitations
I follow the standard rules (the longest, most specific match wins), but every crawler can interpret the finer points slightly differently. This gives you the usual reading, not an absolute guarantee of how every search engine behaves.
One wrong line in robots.txt can stop Google crawling whole sections of your site without anyone noticing. That's why it's one of the first things I look at in a technical review.
Ask for a free review of your site.
If you suspect Google isn't seeing pages it should, tell me and I'll look into it.