robots.txt Tester
Paste a robots.txt, pick a path and User-agent, and see whether a crawler is allowed — with the exact rule that won.
Matching rules
Paste a robots.txt and a path to see which Allow / Disallow lines apply.
Groups & sitemaps
How it works
Paste the contents of a robots.txt file (or tap an example). Enter the path a crawler would request, such as /admin or /blog/post, and optionally a User-agent name like Googlebot. The tool never fetches a remote robots.txt — that would be a risk to other servers — so paste only.
Matching follows the rules search engines commonly use: pick the most specific matching User-agent group (falling back to *), then among that group's Allow and Disallow lines choose the longest match. If two rules are the same length, Allow wins. * matches any characters and $ anchors the end of the path. An empty Disallow: means allow everything.
Crawl-delay is listed as a note because it is non-standard (Google ignores it; some bots honour it). Sitemap: lines are listed so you can see what the file advertises. For a kid-friendly tour of what crawlers actually do with these signs, see Wirebook's What are bots doing on the web? lesson.
Questions
Is my robots.txt uploaded?
No. Parsing and matching run entirely in your browser. The page never fetches a URL for you, so it cannot be used to probe other sites.
Which crawler's rules does this follow?
The common Google/Bing model: most specific User-agent group, then longest Allow/Disallow, with * and $ wildcards. Individual bots still differ around Crawl-delay and edge cases.
What does an empty Disallow mean?
Disallow: with nothing after it means 'allow all' for that User-agent group. It is how many sites write an allow-everything file.
Why can't I paste a URL to fetch robots.txt?
Fetching arbitrary URLs from the server would let strangers use this site to scan other people's servers. Paste the file contents instead.