fetch_link_documentation

Extract documentation components, APIs, and usage examples from website links using configurable crawling depth and content selectors.

Instructions

Fetch and analyze documentation from a website link, extracting all available components, APIs, and usage examples

Input Schema

TableJSON Schema

Name	Required	Description	Default
`depth`	No	How deep to crawl links (0-3, default: 1)
`selector`	No	Optional CSS selector to focus on specific content	body
`url`	Yes	The URL to fetch documentation from (e.g., WindUI documentation)

Implementation Reference

src/tools/linkDocumentationTool.ts:24-41 (handler)

The entry-point handler function for the 'fetch_link_documentation' tool. It clears visited URLs, crawls the provided URL (with optional selector and depth), formats the result, and returns it as MCP content.

async fetchDocumentation(args: FetchDocumentationArgs) {
  this.visitedUrls.clear();

  try {
    const result = await this.crawlPage(args.url, args.selector || 'body', args.depth || 1);
    
    return {
      content: [
        {
          type: "text",
          text: this.formatDocumentationResult(result),
        },
      ],
    };
  } catch (error) {
    throw new Error(`Failed to fetch documentation: ${error instanceof Error ? error.message : String(error)}`);
  }
}

src/tools/linkDocumentationTool.ts:43-91 (helper)

Core helper method that performs the actual web crawling, content extraction (title, text, components, APIs, examples, links), and recursive crawling based on depth.

private async crawlPage(url: string, selector: string, depth: number): Promise<DocumentationResult> {
  if (this.visitedUrls.has(url) || depth < 0) {
    return this.createEmptyResult(url);
  }

  this.visitedUrls.add(url);

  try {
    const response = await axios.get(url, {
      timeout: 10000,
      headers: {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36'
      }
    });

    const $ = cheerio.load(response.data);
    const selectedContent = $(selector);

    const result: DocumentationResult = {
      url,
      title: $('title').text().trim() || $('h1').first().text().trim() || 'Untitled',
      content: this.extractTextContent(selectedContent),
      components: this.extractComponents($),
      apis: this.extractAPIs($),
      examples: this.extractExamples($),
      links: this.extractLinks($, url),
    };

    // 如果depth > 0，继续爬取相关链接
    if (depth > 0) {
      const relatedLinks = result.links
        .filter(link => this.isRelevantLink(link))
        .slice(0, 5); // 限制每页最多5个链接

      for (const link of relatedLinks) {
        const subResult = await this.crawlPage(link, selector, depth - 1);
        result.content += `\n\n--- Linked Page: ${subResult.title} ---\n${subResult.content}`;
        result.components.push(...subResult.components);
        result.apis.push(...subResult.apis);
        result.examples.push(...subResult.examples);
      }
    }

    return result;
  } catch (error) {
    console.error(`Error crawling ${url}:`, error);
    return this.createEmptyResult(url);
  }
}

src/index.ts:39-58 (schema)

JSON schema defining the input parameters for the tool: required 'url', optional 'selector' and 'depth'.

inputSchema: {
  type: "object",
  properties: {
    url: {
      type: "string",
      description: "The URL to fetch documentation from (e.g., WindUI documentation)",
    },
    selector: {
      type: "string",
      description: "Optional CSS selector to focus on specific content",
      default: "body"
    },
    depth: {
      type: "number",
      description: "How deep to crawl links (0-3, default: 1)",
      default: 1
    }
  },
  required: ["url"],
},

src/index.ts:33-62 (registration)

Registers the tool in MCP ListToolsRequestHandler, providing name, description, and input schema.

this.server.setRequestHandler(ListToolsRequestSchema, async () => {
  return {
    tools: [
      {
        name: "fetch_link_documentation",
        description: "Fetch and analyze documentation from a website link, extracting all available components, APIs, and usage examples",
        inputSchema: {
          type: "object",
          properties: {
            url: {
              type: "string",
              description: "The URL to fetch documentation from (e.g., WindUI documentation)",
            },
            selector: {
              type: "string",
              description: "Optional CSS selector to focus on specific content",
              default: "body"
            },
            depth: {
              type: "number",
              description: "How deep to crawl links (0-3, default: 1)",
              default: 1
            }
          },
          required: ["url"],
        },
      },
    ],
  };
});

src/index.ts:68-71 (registration)
Dispatches calls to 'fetch_link_documentation' to the LinkDocumentationTool instance in the CallToolRequestHandler.
```
switch (name) {
  case "fetch_link_documentation":
    return await this.linkTool.fetchDocumentation(args as any);
```

Link MCP