Robots.txt Generator
Build a robots.txt file with a simple form-based editor.
Rule 1
robots.txt
User-agent: *
Build a valid robots.txt file visually — no syntax memorization required. Add rules for specific bots, set allow/disallow paths, add crawl-delay, and include your sitemap URL. Copy or download the finished file.
Key Features
Multi-rule builder
Add multiple rule blocks, each with its own User-agent, Allow paths, Disallow paths, and optional Crawl-delay.
Bot shortcuts
Quick-insert buttons for common bots: all bots (*), Googlebot, Bingbot, Baiduspider, and more — no need to type exact user-agent strings.
Path presets
One-click insertion of commonly-disallowed paths like /admin/, /wp-admin/, /cgi-bin/, and /private/ to save time on standard configurations.
Download as robots.txt
Download the generated file named robots.txt ready to upload to your web server root.
How to Use
- 1Click Quick-add for common bots or type a custom User-agent.
- 2Add Allow and Disallow paths for that rule.
- 3Click Add Rule to create additional rule blocks.
- 4Enter your Sitemap URL, then Copy or Download the result.