Web crawling techniques and tools MCP servers

"Web crawling techniques and tools" matching MCP servers:

• MCP Web Tools Server

  JoaoPedroLanca

  A

  security

  F

  license

  A

  quality

  Provides web connectivity tools for searching the web via DuckDuckGo or SerpAPI, fetching URL content, and extracting readable text from web pages.

  Last updated 2 months ago

  3

• mcp-web

  TsFreddie

  -

  security

  A

  license

  -

  quality

  Enables AI agents to search DuckDuckGo and fetch web content with human-assisted CAPTCHA handling, designed for local use where humans can intervene when bot detection occurs.

  Last updated 2 months ago

  11

  MIT License

• Web Research Assistant

  elad12390

  A

  security

  A

  license

  A

  quality

  Comprehensive web research toolkit with 13 tools for searching (via SearXNG), crawling, package discovery, GitHub metrics, error translation, API documentation lookup, data extraction, technology comparison, and service status checking.

  Last updated 20 days ago

  13

  1

  MIT License

• MCP Web Tools Server

  surya-madhav

  -

  security

  A

  license

  -

  quality

  A Model Context Protocol server that allows LLMs to interact with web content through standardized tools, currently supporting web scraping functionality.

  Last updated 9 months ago

  1

  MIT License

• MCP Web Research Agent

  SnotacusNexus

  -

  security

  A

  license

  -

  quality

  Enables automated web research and intelligence gathering through recursive web crawling, multi-engine search integration, and persistent SQLite storage with support for keyword filtering and multiple export formats.

  Last updated a month ago

  MIT License

• Web Search MCP Server

  vishalkg

  -

  security

  F

  license

  -

  quality

  Enables web search across multiple search engines (DuckDuckGo, Bing, Startpage) with parallel execution and result deduplication. Also provides web page content extraction capabilities.

  Last updated 2 months ago

  1