fetch_markdown
Convert website content to Markdown format by fetching URLs, enabling structured extraction of web data for documentation or analysis.
Instructions
Fetch a website and return its contents converted content to Markdown
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | URL of the website to fetch | |
| headers | No | Optional headers to include in the request | |
| max_length | No | Maximum number of characters to return (default: 5000}) | |
| start_index | No | Start content from this character index (default: 0) |
Implementation Reference
- src/Fetcher.ts:127-148 (handler)The core handler function for fetch_markdown tool. Fetches the URL, converts HTML to Markdown using TurndownService, applies length limits, and returns the content.
static async markdown(requestPayload: RequestPayload) { try { const response = await this._fetch(requestPayload); const html = await response.text(); const turndownService = new TurndownService(); let markdown = turndownService.turndown(html); // Apply length limits markdown = this.applyLengthLimits( markdown, requestPayload.max_length ?? 5000, requestPayload.start_index ?? 0 ); return { content: [{ type: "text", text: markdown }], isError: false }; } catch (error) { return { content: [{ type: "text", text: (error as Error).message }], isError: true, }; } } - src/index.ts:56-81 (schema)Tool schema definition including input schema for fetch_markdown, registered in ListTools response.
{ name: "fetch_markdown", description: "Fetch a website and return its contents converted content to Markdown", inputSchema: { type: "object", properties: { url: { type: "string", description: "URL of the website to fetch", }, headers: { type: "object", description: "Optional headers to include in the request", }, max_length: { type: "number", description: `Maximum number of characters to return (default: ${downloadLimit}})`, }, start_index: { type: "number", description: "Start content from this character index (default: 0)", }, }, required: ["url"], }, }, - src/index.ts:156-159 (registration)Registration of the fetch_markdown handler in the CallToolRequestSchema handler, dispatching to Fetcher.markdown.
if (request.params.name === "fetch_markdown") { const fetchResult = await Fetcher.markdown(validatedArgs); return fetchResult; } - src/types.ts:5-10 (schema)Shared Zod schema for request payload validation, used for all fetch tools including fetch_markdown.
export const RequestPayloadSchema = z.object({ url: z.string().url(), headers: z.record(z.string()).optional(), max_length: z.number().int().min(0).optional().default(downloadLimit), start_index: z.number().int().min(0).optional().default(0), }); - src/Fetcher.ts:7-14 (helper)Helper method to apply max_length and start_index limits to the fetched content.
private static applyLengthLimits(text: string, maxLength: number, startIndex: number): string { if (startIndex >= text.length) { return ""; } const end = maxLength > 0 ? Math.min(startIndex + maxLength, text.length) : text.length; return text.substring(startIndex, end); }