Skip to main content
Glama
wlmwwx
by wlmwwx

read_url

Extract web page content and convert it to clean markdown format for reading articles, documentation, or bypassing paywalls.

Instructions

Extract and convert web page content to clean, readable markdown format. Perfect for reading articles, documentation, blog posts, or any web content. Use this when you need to analyze text content from websites, bypass paywalls, or get structured data.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
urlYesThe complete URL of the webpage or PDF file to read and convert (e.g., 'https://example.com/article'). Can be a single URL string or an array of URLs for parallel reading.
withAllLinksNoSet to true to extract and return all hyperlinks found on the page as structured data
withAllImagesNoSet to true to extract and return all images found on the page as structured data

Implementation Reference

  • MCP tool handler function that imports and calls the core readUrlFromConfig utility, handles errors, and formats the YAML response.
    async ({ url, withAllLinks, withAllImages }: { url: string; withAllLinks?: boolean; withAllImages?: boolean }) => { try { const props = getProps(); // Import the utility function const { readUrlFromConfig } = await import("../utils/read.js"); // Use the shared utility function const result = await readUrlFromConfig({ url, withAllLinks: withAllLinks || false, withAllImages: withAllImages || false }, props.bearerToken); if ('error' in result) { return createErrorResponse(result.error); } return { content: [{ type: "text" as const, text: yamlStringify(result.structuredData), }], }; } catch (error) { return createErrorResponse(`Error: ${error instanceof Error ? error.message : String(error)}`); } },
  • Zod schema for tool inputs: required URL, optional flags for including all links and images.
    { url: z.string().url().describe("The complete URL of the webpage or PDF file to read and convert (e.g., 'https://example.com/article')"), withAllLinks: z.boolean().optional().describe("Set to true to extract and return all hyperlinks found on the page as structured data"), withAllImages: z.boolean().optional().describe("Set to true to extract and return all images found on the page as structured data")
  • Registration of the 'read_url' tool on the MCP server, including name, description, schema, and handler.
    server.tool( "read_url", "Extract and convert web page content to clean, readable markdown format. Perfect for reading articles, documentation, blog posts, or any web content. Use this when you need to analyze text content from websites, bypass paywalls, or get structured data. 💡 Tip: Use `parallel_read_url` if you need to read multiple web pages simultaneously.", { url: z.string().url().describe("The complete URL of the webpage or PDF file to read and convert (e.g., 'https://example.com/article')"), withAllLinks: z.boolean().optional().describe("Set to true to extract and return all hyperlinks found on the page as structured data"), withAllImages: z.boolean().optional().describe("Set to true to extract and return all images found on the page as structured data") }, async ({ url, withAllLinks, withAllImages }: { url: string; withAllLinks?: boolean; withAllImages?: boolean }) => { try { const props = getProps(); // Import the utility function const { readUrlFromConfig } = await import("../utils/read.js"); // Use the shared utility function const result = await readUrlFromConfig({ url, withAllLinks: withAllLinks || false, withAllImages: withAllImages || false }, props.bearerToken); if ('error' in result) { return createErrorResponse(result.error); } return { content: [{ type: "text" as const, text: yamlStringify(result.structuredData), }], }; } catch (error) { return createErrorResponse(`Error: ${error instanceof Error ? error.message : String(error)}`); } }, );
  • Core utility function implementing the tool logic: normalizes URL, calls r.jina.ai API with custom headers, parses response into structured data including optional links/images, handles errors.
    export async function readUrlFromConfig( urlConfig: ReadUrlConfig, bearerToken?: string ): Promise<ReadUrlResponse> { try { // Normalize the URL first const normalizedUrl = normalizeUrl(urlConfig.url); if (!normalizedUrl) { return { error: "Invalid or unsupported URL", url: urlConfig.url }; } const headers: Record<string, string> = { 'Accept': 'application/json', 'Content-Type': 'application/json', 'X-Md-Link-Style': 'discarded', }; // Add Authorization header if bearer token is available if (bearerToken) { headers['Authorization'] = `Bearer ${bearerToken}`; } if (urlConfig.withAllLinks) { headers['X-With-Links-Summary'] = 'all'; } if (urlConfig.withAllImages) { headers['X-With-Images-Summary'] = 'true'; } else { headers['X-Retain-Images'] = 'none'; } const response = await fetch('https://r.jina.ai/', { method: 'POST', headers, body: JSON.stringify({ url: normalizedUrl }), }); if (!response.ok) { return { error: `HTTP ${response.status}: ${response.statusText}`, url: urlConfig.url }; } const data = await response.json() as any; if (!data.data) { return { error: "Invalid response data from r.jina.ai", url: urlConfig.url }; } // Prepare structured data const structuredData: any = { url: data.data.url, title: data.data.title, }; if (urlConfig.withAllLinks && data.data.links) { structuredData.links = data.data.links.map((link: [string, string]) => ({ anchorText: link[0], url: link[1] })); } if (urlConfig.withAllImages && data.data.images) { structuredData.images = data.data.images; } structuredData.content = data.data.content || ""; return { success: true, url: urlConfig.url, structuredData, withAllLinks: urlConfig.withAllLinks || false, withAllImages: urlConfig.withAllImages || false }; } catch (error) { return { error: error instanceof Error ? error.message : String(error), url: urlConfig.url }; } }
  • TypeScript interfaces and type for input configuration, success result, error, and union response type used by the helper function.
    export interface ReadUrlConfig { url: string; withAllLinks?: boolean; withAllImages?: boolean; } export interface ReadUrlResult { success: boolean; url: string; structuredData: any; withAllLinks: boolean; withAllImages: boolean; } export interface ReadUrlError { error: string; url: string; } export type ReadUrlResponse = ReadUrlResult | ReadUrlError;

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/wlmwwx/jina-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server