fetch-url
Fetch web content from URLs using HTTP methods like GET, POST, PUT, and DELETE. Configure requests with headers, body, timeout, and response formats including text, JSON, and binary.
Instructions
Fetch content from a URL
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | URL to fetch | |
| method | No | HTTP method | GET |
| headers | No | HTTP headers | |
| body | No | Request body for POST/PUT/PATCH requests | |
| timeout | No | Request timeout in milliseconds | |
| responseType | No | How to parse the response | text |
| followRedirects | No | Whether to follow redirects |
Implementation Reference
- src/index.ts:179-286 (handler)Main execution logic for the 'fetch-url' tool: validates input, performs HTTP fetch with undici, processes response based on type (text/json/binary/html-fragment), includes metadata, handles errors.async function handleFetchUrl(args: FetchUrlArgs): Promise<z.infer<typeof CallToolResultSchema>> { try { const { url, method, headers, body, timeout, responseType, followRedirects } = args; // Log the fetch request console.error(`Fetching ${url} with method ${method}`); // Create request options const options: any = { method, headers: headers || {}, body: body || undefined, redirect: followRedirects ? "follow" : "manual", }; if (timeout) { // @ts-ignore - undici specific options options.bodyTimeout = timeout; // @ts-ignore - undici specific options options.headersTimeout = timeout; } // Perform the request const response = await fetch(url, options); // Create a result object with metadata const result: Record<string, any> = { status: response.status, statusText: response.statusText, headers: Object.fromEntries(response.headers.entries()), url: response.url, }; // Process the response based on the requested type switch (responseType) { case "json": try { const jsonContent = await response.json(); result.content = JSON.stringify(jsonContent, null, 2); } catch (e) { throw new Error(`Failed to parse response as JSON: ${(e as Error).message}`); } break; case "binary": const buffer = await response.arrayBuffer(); result.content = Buffer.from(buffer).toString("base64"); result.encoding = "base64"; break; case "html-fragment": try { const htmlContent = await response.text(); // Use JSDOM to parse the HTML const dom = new JSDOM(htmlContent); const document = dom.window.document; // Only look for fragments if a selector is provided if (args.fragmentSelector) { const elements = document.querySelectorAll(args.fragmentSelector); if (elements.length === 0) { throw new Error(`No elements found matching selector "${args.fragmentSelector}"`); } // Extract the HTML from the selected element(s) if (elements.length === 1) { result.content = elements[0].outerHTML; } else { result.content = Array.from(elements).map(el => el.outerHTML).join('\n'); } result.matchCount = elements.length; } else { // No selector provided, return the full HTML result.content = htmlContent; } result.contentType = 'text/html'; } catch (e) { throw new Error(`Failed to parse or extract HTML fragment: ${(e as Error).message}`); } break; case "text": default: result.content = await response.text(); break; } return { content: [ { type: "text", text: JSON.stringify(result, null, 2) } ] }; } catch (error) { console.error(`Error fetching URL:`, error); return { isError: true, content: [ { type: "text", text: `Error fetching URL: ${(error as Error).message}` } ] }; } }
- src/index.ts:16-25 (schema)Zod input validation schema for the fetch-url tool arguments.const FetchUrlArgsSchema = z.object({ url: z.string().url().describe("URL to fetch"), method: z.enum(["GET", "POST", "PUT", "DELETE", "HEAD", "OPTIONS", "PATCH"]).default("GET").describe("HTTP method"), headers: z.record(z.string()).optional().describe("HTTP headers"), body: z.string().optional().describe("Request body for POST/PUT/PATCH requests"), timeout: z.number().positive().optional().describe("Request timeout in milliseconds"), responseType: z.enum(["text", "json", "binary", "html-fragment"]).default("text").describe("How to parse the response"), followRedirects: z.boolean().default(true).describe("Whether to follow redirects"), fragmentSelector: z.string().optional().describe("CSS selector for the HTML fragment to extract (when responseType is html-fragment)") });
- src/index.ts:85-119 (registration)MCP tool registration object defining name, description, and input schema for 'fetch-url', added to TOOLS array used in ListTools handler.{ name: "fetch-url", description: "Fetch content from a URL", inputSchema: { type: "object", properties: { url: { type: "string", description: "URL to fetch" }, method: { type: "string", enum: ["GET", "POST", "PUT", "DELETE", "HEAD", "OPTIONS", "PATCH"], default: "GET", description: "HTTP method" }, headers: { type: "object", additionalProperties: { type: "string" }, description: "HTTP headers" }, body: { type: "string", description: "Request body for POST/PUT/PATCH requests" }, timeout: { type: "number", description: "Request timeout in milliseconds" }, responseType: { type: "string", enum: ["text", "json", "binary", "html-fragment"], default: "text", description: "How to parse the response" }, followRedirects: { type: "boolean", default: true, description: "Whether to follow redirects" } }, required: ["url"] } },
- src/index.ts:159-160 (registration)Dispatch case in the CallToolRequestSchema handler that invokes the fetch-url handler after argument validation.case "fetch-url": return handleFetchUrl(FetchUrlArgsSchema.parse(args));