fetch-url-docs
Extract library documentation from any URL across multiple programming languages, retrieving READMEs, API docs, and code examples for analysis without API keys.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | URL of the library documentation to fetch |
Implementation Reference
- src/index.ts:138-171 (handler)The handler function that executes the 'fetch-url-docs' tool logic. It takes a URL, fetches the documentation using scraperService, and returns the content or an error response.async ({ url }) => { console.error(`Fetching documentation from URL: ${url}`); try { const documentationContent = await scraperService.fetchLibraryDocumentation(url); return { content: [ { type: "text", text: documentationContent, }, ], }; } catch (error) { console.error("Error fetching URL content:", error); const errorMessage = `Error fetching URL content: ${ error instanceof Error ? error.message : String(error) }`; return { content: [ { type: "text", text: errorMessage, }, ], isError: true, }; } } );
- src/index.ts:135-137 (schema)Input schema validation using Zod for the tool's 'url' parameter.{ url: z.string().url().describe("URL of the library documentation to fetch"), },
- src/index.ts:133-171 (registration)The server.tool call that registers the 'fetch-url-docs' tool, including schema and handler.server.tool( "fetch-url-docs", { url: z.string().url().describe("URL of the library documentation to fetch"), }, async ({ url }) => { console.error(`Fetching documentation from URL: ${url}`); try { const documentationContent = await scraperService.fetchLibraryDocumentation(url); return { content: [ { type: "text", text: documentationContent, }, ], }; } catch (error) { console.error("Error fetching URL content:", error); const errorMessage = `Error fetching URL content: ${ error instanceof Error ? error.message : String(error) }`; return { content: [ { type: "text", text: errorMessage, }, ], isError: true, }; } } );
- Key helper method implementing the core scraping, crawling, processing, and markdown compilation logic called by the tool handler.public async fetchLibraryDocumentation( url: string, maxPages = 5 ): Promise<string> { try { // If input is not a URL, assume it's a package name if (!url.startsWith("http")) { url = `https://www.npmjs.com/package/${url}`; } // Extract library name from URL const libraryName = extractLibraryName(url); // Crawl documentation const pages = await this.crawlDocumentation(url, libraryName, maxPages); if (pages.length === 0) { throw new Error(`Failed to fetch documentation from ${url}`); } // Compile documentation into a single markdown document const documentation = this.compileDocumentation(pages, libraryName); // Include instructions for using the prompt const promptInstructions = ` --- 🔍 For better summarization, use the "summarize-library-docs" prompt with: - libraryName: "${libraryName}" - documentation: <the content above> Example: @summarize-library-docs with libraryName="${libraryName}" `; return documentation + promptInstructions; } catch (error) { console.error(`Error fetching URL content:`, error); // Extract library name from URL const libraryName = extractLibraryName(url); const errorMessage = `Error fetching URL content: ${ error instanceof Error ? error.message : String(error) }`; // Include error-specific prompt instructions const promptInstructions = ` --- 🔍 For information about this library despite the fetch error, use the "summarize-library-docs" prompt with: - libraryName: "${libraryName}" - errorStatus: "${error instanceof Error ? error.message : String(error)}" Example: @summarize-library-docs with libraryName="${libraryName}" and errorStatus="fetch failed" `; return errorMessage + promptInstructions; } }