extract_links
Extract all links from a web page to analyze content structure, discover resources, or gather URLs for development workflows. Supports both static and dynamic content extraction.
Instructions
Extract all links from a web page
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | URL to scrape | |
| useBrowser | No | Use browser for dynamic content |
Implementation Reference
- src/tools/web-scraping.ts:302-309 (handler)Main handler dispatch for 'extract_links' tool within handleWebScrapingTool function. Chooses between dynamic and static scraper based on useBrowser flag and returns the links.
case 'extract_links': { if (config.useBrowser) { const data = await dynamicScraper.scrapeDynamicContent(config); return data.links; } else { return await staticScraper.extractLinks(config); } } - src/tools/web-scraping.ts:56-72 (schema)Input schema definition for the 'extract_links' tool.
name: 'extract_links', description: 'Extract all links from a web page', inputSchema: { type: 'object', properties: { url: { type: 'string', description: 'URL to scrape', }, useBrowser: { type: 'boolean', description: 'Use browser for dynamic content', default: false, }, }, required: ['url'], }, - src/tools/web-scraping.ts:55-73 (registration)Registration of the 'extract_links' tool in the webScrapingTools array, which is later included in the MCP server's allTools.
{ name: 'extract_links', description: 'Extract all links from a web page', inputSchema: { type: 'object', properties: { url: { type: 'string', description: 'URL to scrape', }, useBrowser: { type: 'boolean', description: 'Use browser for dynamic content', default: false, }, }, required: ['url'], }, }, - src/scrapers/static-scraper.ts:126-129 (handler)Static scraper implementation of extractLinks: invokes scrapeHTML to parse page and returns the links array.
async extractLinks(config: ScrapingConfig): Promise<string[]> { const data = await this.scrapeHTML(config); return data.links || []; } - src/scrapers/static-scraper.ts:32-43 (helper)Core logic for extracting links using Cheerio: iterates over <a> tags, resolves relative URLs, and collects unique hrefs.
const links: string[] = []; $('a[href]').each((_, element) => { const href = $(element).attr('href'); if (href) { try { const url = new URL(href, config.url); links.push(url.href); } catch { // Invalid URL, skip } } }); - src/server.ts:18-25 (registration)Top-level registration: includes webScrapingTools (containing extract_links) into the allTools list provided to the MCP server for tool listing.
const allTools = [ ...codeAnalysisTools, ...codeQualityTools, ...dependencyAnalysisTools, ...lintingTools, ...webScrapingTools, ...apiDiscoveryTools, ];