fetch-url-docs
Fetch and extract comprehensive package documentation from programming language ecosystems like JavaScript, Python, and Java for LLMs without requiring API keys.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | URL of the library documentation to fetch |
Implementation Reference
- src/index.ts:133-171 (registration)Registration of the 'fetch-url-docs' tool, including inline input schema (URL validation with Zod) and handler function that fetches and returns documentation content from the given URL using ScraperService or handles errors.server.tool( "fetch-url-docs", { url: z.string().url().describe("URL of the library documentation to fetch"), }, async ({ url }) => { console.error(`Fetching documentation from URL: ${url}`); try { const documentationContent = await scraperService.fetchLibraryDocumentation(url); return { content: [ { type: "text", text: documentationContent, }, ], }; } catch (error) { console.error("Error fetching URL content:", error); const errorMessage = `Error fetching URL content: ${ error instanceof Error ? error.message : String(error) }`; return { content: [ { type: "text", text: errorMessage, }, ], isError: true, }; } } );
- src/index.ts:138-170 (handler)The inline handler function for 'fetch-url-docs' tool execution logic, which calls scraperService.fetchLibraryDocumentation(url) and formats the response as MCP content or error.async ({ url }) => { console.error(`Fetching documentation from URL: ${url}`); try { const documentationContent = await scraperService.fetchLibraryDocumentation(url); return { content: [ { type: "text", text: documentationContent, }, ], }; } catch (error) { console.error("Error fetching URL content:", error); const errorMessage = `Error fetching URL content: ${ error instanceof Error ? error.message : String(error) }`; return { content: [ { type: "text", text: errorMessage, }, ], isError: true, }; } }
- src/index.ts:135-137 (schema)Zod input schema for the tool defining the 'url' parameter as a valid URL.{ url: z.string().url().describe("URL of the library documentation to fetch"), },
- Core helper method implementing the documentation fetching logic: resolves URL if package name, extracts library name, crawls up to 5 relevant pages, compiles into structured Markdown with TOC, code examples, API refs, and adds LLM prompt instructions. Handles errors gracefully.public async fetchLibraryDocumentation( url: string, maxPages = 5 ): Promise<string> { try { // If input is not a URL, assume it's a package name if (!url.startsWith("http")) { url = `https://www.npmjs.com/package/${url}`; } // Extract library name from URL const libraryName = extractLibraryName(url); // Crawl documentation const pages = await this.crawlDocumentation(url, libraryName, maxPages); if (pages.length === 0) { throw new Error(`Failed to fetch documentation from ${url}`); } // Compile documentation into a single markdown document const documentation = this.compileDocumentation(pages, libraryName); // Include instructions for using the prompt const promptInstructions = ` --- š For better summarization, use the "summarize-library-docs" prompt with: - libraryName: "${libraryName}" - documentation: <the content above> Example: @summarize-library-docs with libraryName="${libraryName}" `; return documentation + promptInstructions; } catch (error) { console.error(`Error fetching URL content:`, error); // Extract library name from URL const libraryName = extractLibraryName(url); const errorMessage = `Error fetching URL content: ${ error instanceof Error ? error.message : String(error) }`; // Include error-specific prompt instructions const promptInstructions = ` --- š For information about this library despite the fetch error, use the "summarize-library-docs" prompt with: - libraryName: "${libraryName}" - errorStatus: "${error instanceof Error ? error.message : String(error)}" Example: @summarize-library-docs with libraryName="${libraryName}" and errorStatus="fetch failed" `; return errorMessage + promptInstructions; } }