get_website_urls
Extract and filter website URLs based on search queries to identify relevant pages for content analysis or research purposes.
Instructions
Search and retrieve relevant URLs from a website
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The URL of the website to map. | |
| search_query | Yes | The search query to sort URLs by. |
Implementation Reference
- src/tools/getWebsiteURLs.ts:19-76 (handler)The async handler function that implements the core logic: sends a POST request to the Olostep Map API with the provided URL and search query to retrieve relevant URLs.
handler: async ({ url, search_query }: { url: string; search_query: string }, apiKey: string) => { try { const headers = new Headers({ 'Content-Type': 'application/json', 'Authorization': `Bearer ${apiKey}` }); const payload = { url: url, search_query: search_query, top_n: 100 }; const response = await fetch(OLOSTEP_MAP_API_URL, { method: 'POST', headers: headers, body: JSON.stringify(payload) }); if (!response.ok) { const errorDetails = await response.json(); return { isError: true, content: [{ type: "text", text: `Olostep API Error: ${response.status} ${response.statusText}. Details: ${JSON.stringify(errorDetails)}` }] }; } const data = await response.json() as OlostepMapApiResponse; if (data.urls && data.urls.length > 0) { return { content: [{ type: "text", text: `Found ${data.urls_count} URLs matching your query:\n\n${data.urls.join('\n')}` }] }; } else { return { content: [{ type: "text", text: "No URLs found matching your search query." }] }; } } catch (error: unknown) { return { isError: true, content: [{ type: "text", text: `Error: Failed to fetch website map. ${error instanceof Error ? error.message : String(error)}` }] }; } } - src/tools/getWebsiteURLs.ts:15-18 (schema)Zod schema defining the input parameters for the tool: 'url' (website URL) and 'search_query' (query to match URLs).
schema: { url: z.string().url().describe("The URL of the website to map."), search_query: z.string().describe("The search query to sort URLs by."), }, - src/index.ts:148-161 (registration)Tool registration using McpServer.tool(), including API key check and content type normalization.
// Register the website map tool server.tool( getWebsiteMap.name, getWebsiteMap.description, getWebsiteMap.schema, async (params) => { if (!OLOSTEP_API_KEY) return missingApiKeyError; const result = await getWebsiteMap.handler(params, OLOSTEP_API_KEY); return { ...result, content: result.content.map(item => ({ ...item, type: item.type as "text" })) }; } );