map
Discover all URLs on a domain to create a sitemap-like list for web crawling and data gathering.
Instructions
Discover all URLs on a domain. Returns a sitemap-like list. Costs 2 credits.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Domain or URL to map | |
| max_pages | No | Maximum URLs to discover (default: 100) | |
| search | No | Filter URLs matching this keyword |
Implementation Reference
- src/index.ts:174-178 (handler)The handler function for the 'map' tool that constructs the request body and calls the SearchClaw API POST /map endpoint, returning JSON formatted results.
async ({ url, max_pages, search }) => { const body: Record<string, unknown> = { url, max_pages }; if (search) body.search = search; return jsonResult(await apiPost("/map", body)); } - src/index.ts:169-173 (schema)Zod schema defining the input parameters for the 'map' tool: url (required string), max_pages (optional number, default 100), and search (optional string filter).
{ url: z.string().describe("Domain or URL to map"), max_pages: z.number().optional().default(100).describe("Maximum URLs to discover (default: 100)"), search: z.string().optional().describe("Filter URLs matching this keyword"), }, - src/index.ts:166-179 (registration)Registration of the 'map' tool with the MCP server using server.tool(), including tool name, description, input schema, and handler function.
server.tool( "map", "Discover all URLs on a domain. Returns a sitemap-like list. Costs 2 credits.", { url: z.string().describe("Domain or URL to map"), max_pages: z.number().optional().default(100).describe("Maximum URLs to discover (default: 100)"), search: z.string().optional().describe("Filter URLs matching this keyword"), }, async ({ url, max_pages, search }) => { const body: Record<string, unknown> = { url, max_pages }; if (search) body.search = search; return jsonResult(await apiPost("/map", body)); } ); - src/index.ts:41-59 (helper)Helper function apiPost() that makes HTTP POST requests to the SearchClaw API with timeout handling and error management.
async function apiPost(path: string, body: Record<string, unknown>) { const controller = new AbortController(); const timeout = setTimeout(() => controller.abort(), 30000); try { const response = await fetch(`${API_BASE}${path}`, { method: "POST", headers: { ...headers, "Content-Type": "application/json" }, body: JSON.stringify(body), signal: controller.signal, }); if (!response.ok) { const text = await response.text(); throw new Error(`SearchClaw API error ${response.status}: ${text}`); } return response.json(); } finally { clearTimeout(timeout); } }