Robots.txt Tester
Check whether search engines and bots are allowed to crawl your URLs.
| URL | Result | Matching rule | Group |
|---|
● Everything runs in your browser. Your content and URLs are never sent to our servers.
How it works
The tester follows Google's robots.txt specification (RFC 9309): the crawler uses the group whose User-agent matches it most specifically (or *), then the rule with the longest matching path wins; if an Allow and a Disallow match with the same length, Allow wins. * matches any sequence of characters and $ anchors the end. Lines the parser doesn't understand are reported.