get_url_content
Extract and retrieve web page content from any URL to analyze information for further processing.
Instructions
Get the content of a URL. Use this for further information retrieving to understand the content of each URL.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | URL |
Implementation Reference
- index.ts:284-302 (handler)Handler logic for the 'get_url_content' tool. Extracts URL from arguments, checks Puppeteer scraper readiness, fetches content using helper function, and returns markdown text.
if (name === "get_url_content") { if (!scraperReady) { return { content: [ { type: 'text', text: 'Tool not ready: Puppeteer is still initializing. Please try again in a few moments.' } ], isError: true } } const { url } = args const result = await fetchAndConvertToMarkdown(url as string) return { content: [{ type: 'text', text: result }], isError: false } } - index.ts:61-76 (schema)Schema definition for the 'get_url_content' tool, including name, description, and input schema requiring a 'url' parameter.
const READ_URL_TOOL: Tool = { name: "get_url_content", description: "Get the content of a URL. " + "Use this for further information retrieving to understand the content of each URL.", inputSchema: { type: "object", properties: { url: { type: "string", description: "URL", }, }, required: ["url"], }, }; - index.ts:88-96 (registration)Registration of the 'get_url_content' tool in the MCP server capabilities alongside web_search.
web_search: { description: WEB_SEARCH_TOOL.description, schema: WEB_SEARCH_TOOL.inputSchema, }, get_url_content: { description: READ_URL_TOOL.description, schema: READ_URL_TOOL.inputSchema, }, }, - index.ts:226-244 (helper)Helper function implementing the core logic: scrapes the URL using PuppeteerScraper instance and returns the text content (assumed markdown). Invoked by the handler.
async function fetchAndConvertToMarkdown(url: string, timeoutMs: number = 10000) { if (!scraperReady || !scraper) { throw new Error('Puppeteer is not ready. Please try again in a few moments.') } try { const response = await scraper.scrapePage(url) if (response == null) { throw new Error(`Failed to fetch the URL: ${url}`) } const { content } = response return content.text } catch (error: any) { console.error('Error during scrape:', error.message) throw error } } - index.ts:246-248 (registration)Registration of the ListTools handler which exposes 'get_url_content' via READ_URL_TOOL in the tools list.
server.setRequestHandler(ListToolsRequestSchema, async () => ({ tools: [WEB_SEARCH_TOOL, READ_URL_TOOL], }));