@missionsquad/mcp-searxng-puppeteer

Instructions

Implementation Reference

• index.ts:284-302 (handler)

  Handler logic for the 'get_url_content' tool. Extracts URL from arguments, checks Puppeteer scraper readiness, fetches content using helper function, and returns markdown text.

  if (name === "get_url_content") {
    if (!scraperReady) {
      return {
        content: [
          {
            type: 'text',
            text: 'Tool not ready: Puppeteer is still initializing. Please try again in a few moments.'
          }
        ],
        isError: true
      }
    }
    const { url } = args
    const result = await fetchAndConvertToMarkdown(url as string)
    return {
      content: [{ type: 'text', text: result }],
      isError: false
    }
  }

• index.ts:61-76 (schema)

  Schema definition for the 'get_url_content' tool, including name, description, and input schema requiring a 'url' parameter.

  const READ_URL_TOOL: Tool = {
    name: "get_url_content",
    description:
      "Get the content of a URL. " +
      "Use this for further information retrieving to understand the content of each URL.",
    inputSchema: {
      type: "object",
      properties: {
        url: {
          type: "string",
          description: "URL",
        },
      },
      required: ["url"],
    },
  };

• index.ts:88-96 (registration)

  Registration of the 'get_url_content' tool in the MCP server capabilities alongside web_search.

    web_search: {
      description: WEB_SEARCH_TOOL.description,
      schema: WEB_SEARCH_TOOL.inputSchema,
    },
    get_url_content: {
      description: READ_URL_TOOL.description,
      schema: READ_URL_TOOL.inputSchema,
    },
  },

• index.ts:226-244 (helper)

  Helper function implementing the core logic: scrapes the URL using PuppeteerScraper instance and returns the text content (assumed markdown). Invoked by the handler.

  async function fetchAndConvertToMarkdown(url: string, timeoutMs: number = 10000) {
    if (!scraperReady || !scraper) {
      throw new Error('Puppeteer is not ready. Please try again in a few moments.')
    }
  
    try {
      const response = await scraper.scrapePage(url)
  
      if (response == null) {
        throw new Error(`Failed to fetch the URL: ${url}`)
      }
  
      const { content } = response
      return content.text
    } catch (error: any) {
      console.error('Error during scrape:', error.message)
      throw error
    }
  }

• index.ts:246-248 (registration)

  Registration of the ListTools handler which exposes 'get_url_content' via READ_URL_TOOL in the tools list.

  server.setRequestHandler(ListToolsRequestSchema, async () => ({
    tools: [WEB_SEARCH_TOOL, READ_URL_TOOL],
  }));