Skip to main content
Glama
mysleekdesigns

CrawlForge MCP Server

map_site

Read-onlyIdempotent

Map all URLs on a domain by reading sitemap.xml and discovering links without fetching full page content. Useful for site audits and finding specific sections.

Instructions

Use this when you need to know all URLs on a domain without fetching full page content — e.g. before a crawl_deep, for a site audit, or to find specific section URLs. Reads sitemap.xml when available. Example: map_site({url: "https://example.com", include_sitemap: true, max_urls: 500})

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
urlYesThe website URL to map
searchNoWhen set, rank discovered URLs by relevance to this string and emit ranked_urls:[{url,score}]
max_urlsNoMaximum number of URLs to discover
domain_filterNoPer-domain allow/deny lists and URL include/exclude patterns
group_by_pathNoGroup URLs by path segments
include_sitemapNoInclude sitemap.xml data in results
include_metadataNoInclude page metadata for each URL
import_filter_configNoJSON string of a previously exported domain-filter config
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, openWorldHint=true, idempotentHint=true, and destructiveHint=false, indicating safe, read-only, idempotent behavior. The description adds that it reads sitemap.xml and discovers URLs without fetching content, which is useful but does not go into depth about rate limits, error handling, or behavior of domain filters. The description does not contradict annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise: two sentences and an example. Every sentence adds value, with no fluff or repetition of schema details. The structure front-loads the purpose and usage context, making it easy for an AI agent to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (8 parameters, including a nested domain_filter), the description provides a high-level overview but lacks detail on what the return value looks like (no output schema). It does not explain how search or domain_filter modify the behavior. The example helps, but for a tool with rich parameters, more guidance on parameter combinations would improve completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% coverage, so the description does not need to re-explain each parameter. However, the description provides a concrete example call (url, include_sitemap, max_urls) which adds value. It does not explain nested objects like domain_filter or the search parameter beyond what the schema provides. Thus, it meets the baseline but does not exceed it.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: to discover all URLs on a domain without fetching full page content. It gives specific use cases (before crawl_deep, site audit, finding section URLs) and distinguishes from sibling tools like crawl_deep by emphasizing it doesn't fetch content.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description includes explicit guidance on when to use the tool (e.g., 'before a crawl_deep, for a site audit, or to find specific section URLs'). It provides an example call with parameters. While it does not explicitly list alternative tools, the mention of crawl_deep as a later step gives context. No explicit 'when not to use' but the guidance is clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Install Server

Other Tools

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/mysleekdesigns/crawlforge-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server