Skip to main content
Glama
jina-ai

Jina AI Remote MCP Server

Official
by jina-ai

read_url

Extract web page content and convert it to clean, readable markdown format for analysis, bypassing paywalls and obtaining structured text data from websites.

Instructions

Extract and convert web page content to clean, readable markdown format. Perfect for reading articles, documentation, blog posts, or any web content. Use this when you need to analyze text content from websites, bypass paywalls, or get structured data.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
urlYesThe complete URL of the webpage or PDF file to read and convert (e.g., 'https://example.com/article'). Can be a single URL string or an array of URLs for parallel reading.
withAllLinksNoSet to true to extract and return all hyperlinks found on the page as structured data
withAllImagesNoSet to true to extract and return all images found on the page as structured data

Implementation Reference

  • Primary MCP tool handler for 'read_url': registers the tool, defines input schema, and implements execution logic by delegating to utils/read.ts helpers for single/parallel URL content extraction via r.jina.ai.
    if (isToolEnabled("read_url")) {
    	server.tool(
    		"read_url",
    		"Extract and convert web page content to clean, readable markdown format. Perfect for reading articles, documentation, blog posts, or any web content. Use this when you need to analyze text content from websites, bypass paywalls, or get structured data.",
    		{
    			url: z.union([z.string().url(), z.array(z.string().url())]).describe("The complete URL of the webpage or PDF file to read and convert (e.g., 'https://example.com/article'). Can be a single URL string or an array of URLs for parallel reading."),
    			withAllLinks: z.boolean().optional().describe("Set to true to extract and return all hyperlinks found on the page as structured data"),
    			withAllImages: z.boolean().optional().describe("Set to true to extract and return all images found on the page as structured data")
    		},
    		async ({ url, withAllLinks, withAllImages }: { url: string | string[]; withAllLinks?: boolean; withAllImages?: boolean }) => {
    			try {
    				const props = getProps();
    
    				// Handle single URL or single-element array
    				if (typeof url === 'string' || (Array.isArray(url) && url.length === 1)) {
    					const singleUrl = typeof url === 'string' ? url : url[0];
    
    					// Import the utility function
    					const { readUrlFromConfig } = await import("../utils/read.js");
    
    					// Use the shared utility function
    					const result = await readUrlFromConfig({ url: singleUrl, withAllLinks: withAllLinks || false, withAllImages: withAllImages || false }, props.bearerToken);
    
    					if ('error' in result) {
    						return createErrorResponse(result.error);
    					}
    
    					return applyTokenGuardrail({
    						content: [{
    							type: "text" as const,
    							text: yamlStringify(result.structuredData),
    						}],
    					}, props.bearerToken, getClientName());
    				}
    
    				// Handle multiple URLs with parallel reading
    				if (Array.isArray(url) && url.length > 1) {
    					const urls = url.map(u => ({ url: u, withAllLinks: withAllLinks || false, withAllImages: withAllImages || false }));
    
    					const uniqueUrls = urls.filter((urlConfig, index, self) =>
    						index === self.findIndex(u => u.url === urlConfig.url)
    					);
    
    					// Import the utility functions
    					const { executeParallelUrlReads } = await import("../utils/read.js");
    
    					// Execute parallel URL reads using the utility
    					const results = await executeParallelUrlReads(uniqueUrls, props.bearerToken, 30000);
    
    					// Format results for consistent output
    					const contentItems: Array<{ type: 'text'; text: string }> = [];
    
    					for (const result of results) {
    						if ('success' in result && result.success) {
    							contentItems.push({
    								type: "text" as const,
    								text: yamlStringify(result.structuredData),
    							});
    						} else if ('error' in result) {
    							contentItems.push({
    								type: "text" as const,
    								text: `Error reading ${result.url}: ${result.error}`,
    							});
    						}
    					}
    
    					return applyTokenGuardrail({
    						content: contentItems,
    					}, props.bearerToken, getClientName());
    				}
    
    				return createErrorResponse("Invalid URL format");
    			} catch (error) {
    				return createErrorResponse(`Error: ${error instanceof Error ? error.message : String(error)}`);
    			}
    		},
    	);
  • Type definitions for ReadUrlConfig, ReadUrlResult, ReadUrlError, and ReadUrlResponse used by the read_url handler and helpers.
    export interface ReadUrlConfig {
        url: string;
        withAllLinks?: boolean;
        withAllImages?: boolean;
    }
    
    export interface ReadUrlResult {
        success: boolean;
        url: string;
        structuredData: any;
        withAllLinks: boolean;
        withAllImages: boolean;
    }
    
    export interface ReadUrlError {
        error: string;
        url: string;
    }
    
    export type ReadUrlResponse = ReadUrlResult | ReadUrlError;
  • Core utility function that performs HTTP request to r.jina.ai for URL content extraction, normalizes URL, handles auth, extracts title/content/links/images.
    export async function readUrlFromConfig(
        urlConfig: ReadUrlConfig,
        bearerToken?: string
    ): Promise<ReadUrlResponse> {
        try {
            // Normalize the URL first
            const normalizedUrl = normalizeUrl(urlConfig.url);
            if (!normalizedUrl) {
                return { error: "Invalid or unsupported URL", url: urlConfig.url };
            }
    
            const headers: Record<string, string> = {
                'Accept': 'application/json',
                'Content-Type': 'application/json',
                'X-Md-Link-Style': 'discarded',
            };
    
            // Add Authorization header if bearer token is available
            if (bearerToken) {
                headers['Authorization'] = `Bearer ${bearerToken}`;
            }
    
            if (urlConfig.withAllLinks) {
                headers['X-With-Links-Summary'] = 'all';
            }
    
            if (urlConfig.withAllImages) {
                headers['X-With-Images-Summary'] = 'true';
            } else {
                headers['X-Retain-Images'] = 'none';
            }
    
            const response = await fetch('https://r.jina.ai/', {
                method: 'POST',
                headers,
                body: JSON.stringify({ url: normalizedUrl }),
            });
    
            if (!response.ok) {
                return { error: `HTTP ${response.status}: ${response.statusText}`, url: urlConfig.url };
            }
    
            const data = await response.json() as any;
    
            if (!data.data) {
                return { error: "Invalid response data from r.jina.ai", url: urlConfig.url };
            }
    
            // Prepare structured data
            const structuredData: any = {
                url: data.data.url,
                title: data.data.title,
            };
    
            if (urlConfig.withAllLinks && data.data.links) {
                structuredData.links = data.data.links.map((link: [string, string]) => ({
                    anchorText: link[0],
                    url: link[1]
                }));
            }
    
            if (urlConfig.withAllImages && data.data.images) {
                structuredData.images = data.data.images;
            }
            structuredData.content = data.data.content || "";
    
            return {
                success: true,
                url: urlConfig.url,
                structuredData,
                withAllLinks: urlConfig.withAllLinks || false,
                withAllImages: urlConfig.withAllImages || false
            };
        } catch (error) {
            return {
                error: error instanceof Error ? error.message : String(error),
                url: urlConfig.url
            };
        }
    }
  • Utility for executing multiple URL reads in parallel, used by read_url handler for array inputs.
    export async function executeParallelUrlReads(
        urlConfigs: ReadUrlConfig[],
        bearerToken?: string,
        timeout: number = 30000
    ): Promise<ReadUrlResponse[]> {
        const timeoutPromise = new Promise<never>((_, reject) =>
            setTimeout(() => reject(new Error('Parallel URL read timeout')), timeout)
        );
    
        const readPromises = urlConfigs.map(urlConfig => readUrlFromConfig(urlConfig, bearerToken));
    
        return Promise.race([
            Promise.all(readPromises),
            timeoutPromise
        ]);
    }
  • src/index.ts:100-102 (registration)
    Invokes registerJinaTools which conditionally registers 'read_url' among other tools based on enabledTools filter.
    registerJinaTools(server, () => currentProps, enabledTools);
    
    return server;
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions capabilities like bypassing paywalls and extracting structured data, which adds useful context beyond basic functionality. However, it lacks details on error handling, rate limits, authentication needs, or performance characteristics that would be important for an agent to use it effectively.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is appropriately sized and front-loaded, starting with the core purpose. Every sentence adds value, such as use cases and capabilities. It could be slightly more concise by combining some phrases, but overall it avoids redundancy and maintains clarity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (web content extraction with multiple parameters) and no output schema, the description is moderately complete. It covers the purpose and use cases but lacks details on return values, error conditions, or limitations. With no annotations and no output schema, more behavioral context would improve completeness for agent usage.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description does not add any parameter-specific information beyond what the schema provides (e.g., it doesn't explain URL formats or the implications of withAllLinks/withAllImages). Baseline 3 is appropriate when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose with specific verbs ('extract and convert') and resources ('web page content'), distinguishing it from siblings like capture_screenshot_url (visual capture) or extract_pdf (PDF-specific). It explicitly mentions the output format ('clean, readable markdown format'), which helps differentiate its function.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context for when to use the tool ('when you need to analyze text content from websites, bypass paywalls, or get structured data'), including specific use cases like reading articles or documentation. However, it does not explicitly state when NOT to use it or name alternatives among sibling tools, such as extract_pdf for PDF files or parallel_read_url for batch processing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Install Server

Other Tools

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/jina-ai/MCP'

If you have feedback or need assistance with the MCP directory API, please join our Discord server