fetch-url-docs
Extracts comprehensive package documentation from specified URLs across programming languages like JavaScript, Python, and Java, enabling LLM integrations without API keys.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | URL of the library documentation to fetch |
Implementation Reference
- src/index.ts:134-172 (registration)Registration of the 'fetch-url-docs' tool on the MCP server, including input schema and the handler function."fetch-url-docs", { url: z.string().url().describe("URL of the library documentation to fetch"), }, async ({ url }) => { console.error(`Fetching documentation from URL: ${url}`); try { const documentationContent = await scraperService.fetchLibraryDocumentation(url); return { content: [ { type: "text", text: documentationContent, }, ], }; } catch (error) { console.error("Error fetching URL content:", error); const errorMessage = `Error fetching URL content: ${ error instanceof Error ? error.message : String(error) }`; return { content: [ { type: "text", text: errorMessage, }, ], isError: true, }; } } );
- src/index.ts:139-170 (handler)The handler function that executes the tool's core logic: fetches documentation from the URL using ScraperService and returns structured content or error.console.error(`Fetching documentation from URL: ${url}`); try { const documentationContent = await scraperService.fetchLibraryDocumentation(url); return { content: [ { type: "text", text: documentationContent, }, ], }; } catch (error) { console.error("Error fetching URL content:", error); const errorMessage = `Error fetching URL content: ${ error instanceof Error ? error.message : String(error) }`; return { content: [ { type: "text", text: errorMessage, }, ], isError: true, }; } }
- src/index.ts:136-137 (schema)Zod schema defining the input parameter 'url' as a required URL string.url: z.string().url().describe("URL of the library documentation to fetch"), },
- Core helper method implementing the documentation fetching, crawling multiple pages, compiling into markdown, and adding prompt instructions. Called by the tool handler.public async fetchLibraryDocumentation( url: string, maxPages = 5 ): Promise<string> { try { // If input is not a URL, assume it's a package name if (!url.startsWith("http")) { url = `https://www.npmjs.com/package/${url}`; } // Extract library name from URL const libraryName = extractLibraryName(url); // Crawl documentation const pages = await this.crawlDocumentation(url, libraryName, maxPages); if (pages.length === 0) { throw new Error(`Failed to fetch documentation from ${url}`); } // Compile documentation into a single markdown document const documentation = this.compileDocumentation(pages, libraryName); // Include instructions for using the prompt const promptInstructions = ` --- 🔍 For better summarization, use the "summarize-library-docs" prompt with: - libraryName: "${libraryName}" - documentation: <the content above> Example: @summarize-library-docs with libraryName="${libraryName}" `; return documentation + promptInstructions; } catch (error) { console.error(`Error fetching URL content:`, error); // Extract library name from URL const libraryName = extractLibraryName(url); const errorMessage = `Error fetching URL content: ${ error instanceof Error ? error.message : String(error) }`; // Include error-specific prompt instructions const promptInstructions = ` --- 🔍 For information about this library despite the fetch error, use the "summarize-library-docs" prompt with: - libraryName: "${libraryName}" - errorStatus: "${error instanceof Error ? error.message : String(error)}" Example: @summarize-library-docs with libraryName="${libraryName}" and errorStatus="fetch failed" `; return errorMessage + promptInstructions; } }