Use it to discover, extract, and crawl XML sitemaps for any domain through MCP-connected AI assistants.
Discover sitemaps for a domain by checking robots.txt, common paths, and sitemap indexes (discover_sitemaps).
Extract URLs from a specific sitemap file, including nested sitemap indexes, with optional metadata like lastmod, changefreq, and priority (extract_sitemap).
Full crawl a domain: automatically discover all sitemaps and return a deduplicated URL list in one call (full_crawl).
Limit results with max_urls to control how many URLs are returned.
Integrate with MCP clients like Claude Desktop, Cursor, and Windsurf via stdio.
Monitor quotas through the meta.quota field to track remaining API requests.
Support SEO audits, content inventories, research, and agent workflows requiring complete or partial URL lists.
Enable continuous monitoring via the separate SitemapKit Monitoring service for new pages and webhook alerts.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@SitemapKit MCP Server Find all sitemaps for stripe.com"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Give AI agents a reliable XML sitemap crawler through the Model Context Protocol (MCP). The SitemapKit extraction API discovers sitemap files, parses nested sitemap indexes, and returns a deduplicated URL list for SEO audits, content inventories, research, and agent workflows.
Works with MCP-compatible clients including Claude Desktop, Cursor, Windsurf, and other AI assistants that support local stdio servers.
Tool | Description |
| Find all sitemap files for a domain (checks robots.txt, common paths, sitemap indexes) |
| Extract all URLs from a specific sitemap file |
| Discover + extract all URLs across all sitemaps in one call |
Related MCP server: Webpage MCP Server
Sign up at sitemapkit.com and grab your API key from app.sitemapkit.com/settings/api.
Add to ~/Library/Application Support/Claude/claude_desktop_config.json:
{
"mcpServers": {
"sitemapkit": {
"command": "npx",
"args": ["-y", "sitemapkit-mcp"],
"env": {
"SITEMAPKIT_API_KEY": "your-api-key-here"
}
}
}
}Add to .cursor/mcp.json in your project (or the global ~/.cursor/mcp.json):
{
"mcpServers": {
"sitemapkit": {
"command": "npx",
"args": ["-y", "sitemapkit-mcp"],
"env": {
"SITEMAPKIT_API_KEY": "your-api-key-here"
}
}
}
}Add to ~/.codeium/windsurf/mcp_config.json:
{
"mcpServers": {
"sitemapkit": {
"command": "npx",
"args": ["-y", "sitemapkit-mcp"],
"env": {
"SITEMAPKIT_API_KEY": "your-api-key-here"
}
}
}
}Once configured, you can ask your AI assistant:
"Find all sitemaps for stripe.com"
"Extract every URL from https://example.com/sitemap.xml"
"Get the full URL list for shopify.com, up to 5000 URLs"
This MCP server is designed for on-demand discovery and extraction. To watch a website continuously, detect newly published pages, and send signed webhook alerts, use SitemapKit Monitoring.
The free plan includes one daily monitor. Paid plans add more websites, higher URL limits, and checks as often as every hour. See the webhook documentation for payloads, signatures, and retry behavior.
Limits depend on your SitemapKit plan. The meta.quota field in each response tells you how many requests you have remaining this month.
Plan | API requests/month | URLs per extraction |
Free | 100 | 1,000 |
Starter | 5,000 | 10,000 |
Pro | 50,000 | 50,000 |
See current API and monitoring allowances on the pricing page.
MIT
Find all sitemap files for a given domain by checking robots.txt, common paths, and sitemap indexes. Returns a list of sitemap URLs. Use this first to understand a site's sitemap structure.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The domain to check, e.g. 'https://example.com'. Only the domain is used — path is ignored. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries the burden. It discloses the method (checking robots.txt, common paths, sitemap indexes) and notes that only the domain is used, not the path. This is adequate transparency for a simple discovery tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no waste. First sentence describes functionality, second gives usage guidance. Efficient and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With one parameter, no output schema, and no annotations, the description sufficiently explains return value (list of sitemap URLs) and discovery method. The recommendation to use it first adds valuable context relative to sibling tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and schema already explains the 'url' parameter well. Description adds little beyond reinforcing domain-only scope; baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool finds sitemap files for a domain using robots.txt, common paths, and sitemap indexes, and returns URLs. It distinguishes from siblings by recommending its use first.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly advises 'Use this first to understand a site's sitemap structure', giving clear context for when to employ this tool relative to alternatives like extract_sitemap or full_crawl.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Extract all URLs from a specific sitemap file (including sitemap indexes that link to child sitemaps). Returns URL entries with optional lastmod, changefreq, and priority metadata.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The full URL of the sitemap file, e.g. 'https://example.com/sitemap.xml'. | |
| max_urls | No | Maximum number of URLs to return. Defaults to 1000. Max allowed by your plan. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses that the tool handles sitemap indexes and returns metadata, but does not mention rate limits, authentication, error behavior, or whether the operation is read-only. This is adequate but not thorough.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with no extraneous information. The first sentence states the core purpose, and the second adds return details. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 2 parameters and no output schema, the description provides sufficient context: it explains the input (sitemap URL), boundaries (handles indexes), and output (URLs with metadata). Missing explicit guidance on when to choose this over siblings, but overall complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already describes both parameters (url, max_urls). The description adds context about sitemap indexes and return metadata but does not significantly enhance parameter semantics beyond the schema. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: extracting all URLs from a specific sitemap file, including sitemap indexes. It distinguishes itself from siblings like discover_sitemaps (which finds sitemaps) and full_crawl (which performs a broader crawl) by specifying it acts on a given sitemap URL.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use the tool (when you have a specific sitemap URL) but does not explicitly state when not to use it or provide alternatives. Siblings are listed but not contrasted, leaving the agent to infer usage boundaries.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Discover all sitemaps for a domain and extract every URL across all of them in one call. This is the most convenient tool when you want the complete URL list for a site. Returns deduplicated URLs with metadata.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The domain to crawl, e.g. 'https://example.com'. | |
| max_urls | No | Maximum number of URLs to return. Defaults to 1000. Max allowed by your plan. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It mentions discovering sitemaps, extracting URLs, and deduplication, but lacks details on rate limits, concurrency, destructive actions, or permissions. Adequate but not deeply transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, about 30 words, front-loaded with the core action. No filler or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given simple input schema (two params, no enums) and no output schema, the description provides sufficient context: it returns deduplicated URLs with metadata. Missing explicit output structure is acceptable for low complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers both parameters with descriptions. The tool description does not add additional meaning beyond what is in the schema, so baseline of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool discovers all sitemaps and extracts every URL, providing a complete URL list for a domain. It distinguishes itself from siblings 'discover_sitemaps' and 'extract_sitemap' by combining both actions in one call.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'This is the most convenient tool when you want the complete URL list for a site,' giving clear when-to-use guidance. It does not explicitly state when not to use, but the sibling tools imply alternatives for partial tasks.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v1.0.0discover_sitemapsextract_sitemapfull_crawlScored across 3 tools
Each tool has a clearly distinct purpose: discover_sitemaps finds sitemap URLs, extract_sitemap extracts URLs from a specific sitemap, and full_crawl combines both. There is no overlap or ambiguity.
All tool names use a consistent verb_noun pattern (e.g., discover_sitemaps, extract_sitemap, full_crawl). The names are clear and follow the same style.
Three tools is well-scoped for a server focused on sitemap discovery and URL extraction. Each tool serves a distinct need without being excessive or insufficient.
The tool set covers the full workflow: discovering sitemaps, extracting from a single sitemap, and a convenient combined crawl. There are no obvious missing operations for this domain.
Live web checks for AI agents: sitemaps, robots.txt, URL status, broken links, feeds, citations.
Web tools for AI agents: scrape pages to Markdown, audit SEO, detect tech stacks, check sitemaps
Generate 18 AI readiness files (llms.txt, ai.txt, RAG indexes, schema) for any website.
Extract every URL from a website sitemap, including index and gzipped sitemaps, with status checks
