how-to-crawl-a-website-and-extract-all-links – MCP tools

run_seo_audit
SEO Crawler MCP
Crawl websites to extract SEO data, identify technical issues, and analyze site structure for comprehensive SEO auditing.
Apache 2.0
deep_crawl_site
Crawl-MCP
Crawl multiple pages from a website with configurable depth and parameters to extract structured content for analysis.
MIT
check_site
Broken Link Checker MCP Server
Recursively crawl websites to identify broken links by scanning internal and external URLs across multiple pages, with options to respect robots.txt and limit concurrent requests.
crawl-site
MCP Webscan Server
Scan and extract all unique URLs from a website by recursively crawling from a given URL up to a specified depth. Designed for web content analysis.
MIT
extract_links
MCP Server for Crawl4AI
Extract and categorize all links from a webpage to build sitemaps, analyze site structure, find broken links, or discover resources. Groups links by type for efficient web analysis.
MIT
crawl_recursive
MCP Server for Crawl4AI
Deep crawl websites by following internal links to map entire sites, find all pages, and build comprehensive indexes with configurable depth and page limits.
MIT

Interested in MCP?

Join the MCP community for support and updates.