Robots.txt Generator

Generate robots.txt files to control how search engines crawl your website. Includes templates for common scenarios and support for multiple user agents.

Features

Preset templatesMultiple user agentsAllow/Disallow rulesCrawl delay settingsSitemap referencesCommon path patterns

Robots.txt Generator

Quick Templates

Robot Rules

No rules configured yet

Add a rule or select a template to get started

Sitemaps

How to Use

• Select a template or build custom rules
• Choose which bots to target with User-agent
• Add Allow/Disallow rules for specific paths
• Optionally add crawl delay and sitemap locations
• Download and upload to your website root as robots.txt
• Test using Google's robots.txt Tester in Search Console