Skip to main content
SwiftTools

Robots.txt Generator

Free robots.txt generator. Build a valid robots.txt with allow/disallow rules and a sitemap reference. Clean syntax, no guesswork.

Allow all paths by default
robots.txt
User-agent: *
Allow: /
Disallow: /admin/
Disallow: /api/
Crawl-delay: 5
Sitemap: https://example.com/sitemap.xml

Processed locally in your browser. Nothing you type is uploaded to a server.

About Robots.txt Generator

Generate a clean robots.txt for your site. Allow or disallow paths, point crawlers to your sitemap, and copy the result straight into your web server root.

How to use it

  1. 1Add the paths to disallow.
  2. 2Add your sitemap URL.
  3. 3Click Generate and copy the file.

Examples

Basic

Input
disallow /admin/
Result
Disallow: /admin/

With sitemap

Input
disallow /private/ + sitemap
Result
full robots.txt

Frequently asked questions

Does robots.txt block indexing?

It asks crawlers not to fetch; it is not a security measure.

Can I allow then disallow?

Yes - add both rules for granular control.

Where does the file go?

At your site root: https://yourdomain.com/robots.txt

Related tools