Robots.txt Generator & Validator
Create crawler rules, add sitemap directives, and validate robots.txt syntax locally before publishing the file at your website root.
Validation results
- Generate or paste robots.txt content, then validate it.
Crawler rule generation
Build User-agent, Allow, Disallow, Sitemap, and optional Host directives without memorizing robots.txt formatting.
Syntax validation
Check common mistakes such as missing user-agents, relative sitemap URLs, malformed paths, duplicate rules, and unsupported directive names.
Private local processing
The generator and validator run in your browser. Robots.txt content is not uploaded to Convertr servers.
Frequently asked questions
Is my robots.txt file uploaded for validation?
No. Generation and validation run in your browser with JavaScript. The text you enter is not sent to Convertr servers.
Where should I place robots.txt?
Place robots.txt at the root of the host it applies to, such as https://example.com/robots.txt.
Does robots.txt remove pages from Google?
No. Robots.txt controls crawler access. Use noindex directives on the page or remove the page if you need deindexing.