seo_generate_sitemap_and_robots
Generate standard-compliant sitemap.xml and robots.txt with directives for Googlebot, Bingbot, and AI search engines to fix missing or outdated configuration files.
Instructions
Generates production-ready, standard-compliant sitemap.xml and robots.txt files with crawler directives for Googlebot, Bingbot, and AI search engines (GPTBot, ClaudeBot, PerplexityBot).
USAGE GUIDELINES:
Use when a website is missing sitemap.xml or robots.txt, or needs clean, updated configuration files.
Do NOT use to audit existing files; use 'seo_audit_robots_and_sitemap' instead.
Do NOT use to crawl pages; use 'seo_audit_sitemap_multipage' instead.
BEHAVIORAL TRANSPARENCY:
Pure generation tool. Returns formatted XML and robots.txt file contents as text output. Does not write to disk directly unless copied by user.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| urls | No | List of relative or absolute URLs to register in sitemap.xml (e.g. ["/", "/about", "/pricing"]). | |
| targetUrl | Yes | Base website domain URL (e.g. "https://example.com"). | |
| disallowedPaths | No | URL path prefixes to disallow in robots.txt (e.g. ["/admin/", "/api/private/"]). |