WorkbenchA useful little toolbox.

Robots.txt tester

Test a URL against pasted robots.txt rules and see the deciding line.

Processed on your device. No file upload.

How to use this tool

  1. Enter your values or try the example.
  2. Run the tool and check the result.
  3. Save the result where a download is available.

Supported features & limits

Paste robots.txt and enter a full HTTP(S) URL plus crawler token. Uses robots-parser 3.0.1 with wildcard and end-anchor support. Does not fetch the live file or prove indexing, server availability or WAF access. Maximum 100 KB.

What happens to my files?

Your browser reads the file locally. File contents are not sent to our server. Closing or resetting this page clears the working result. Downloaded copies remain on your device.

Maintained by Roman Zmazhenko. Updated 1 November 2026.