F
licenseNot graded
qualityD
maintenanceThis MCP server provides tools for AI agents to search the web, fetch page content, and query specific elements from pages using DuckDuckGo.
9 npm
-
webx-mcp-server
by ceeyang-ai
The server integrates DuckDuckGo search to enable web searching capabilities, allowing users to search the web and retrieve results via the DuckDuckGo HTML search engine.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@webx-mcp-server fetch clean text from https://en.wikipedia.org/wiki/Web_scraping"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
WebX MCP Server 🌐
A Model Context Protocol (MCP) server for web content extraction — fetch clean text, extract links, query by CSS selector, and search the web.
Built for AI agents. Works with Hermes Agent, Claude Code, Cursor, and any MCP-compatible client.
✨ Features
Tool | Description |
| Fetch a URL and extract clean readable text (up to 50K chars) |
| Extract all links with anchor text, optionally filtered by domain |
| Extract content by CSS selector (text, HTML, or attribute) |
| Search the web via DuckDuckGo HTML search |
Related MCP server: web-mcp-server
🚀 Quick Start
# Install from GitHub
pip install git+https://github.com/ceeyang-ai/webx-mcp-server.git
# Run as MCP server
webx-mcp-server🔌 Usage with AI Agents
Hermes Agent
Add to ~/.hermes/config.yaml:
mcp_servers:
webx:
command: "webx-mcp-server"Restart → use mcp_webx_fetch_clean_text, mcp_webx_extract_links, etc.
Claude Code / Cursor / Any MCP Client
Add to your MCP config:
{
"mcpServers": {
"webx": {
"command": "webx-mcp-server"
}
}
}📖 Examples
Fetch clean text
result = fetch_clean_text(
url="https://en.wikipedia.org/wiki/Web_scraping",
max_chars=5000
)Extract links
result = extract_links(
url="https://example.com",
filter_domain=True,
max_links=20
)Search the web
result = search_web(
query="latest AI research papers",
max_results=10
)🛠 Requirements
Python 3.10+
requests ≥ 2.28
beautifulsoup4 ≥ 4.11
lxml ≥ 4.9
mcp ≥ 1.0
👨💻 Development
git clone https://github.com/ceeyang-ai/webx-mcp-server.git
cd webx-mcp-server
pip install -e .
webx-mcp-server # Start MCP server📄 License
MIT — free for personal and commercial use.
Available Tools
4 toolsextract_by_selectorA
Extract content from a web page using CSS selectors.
Args: url: The URL to fetch css_selector: CSS selector (e.g. 'h1', '.article', '#main p') attribute: What to extract: 'text' (inner text), 'html' (inner HTML), or an attribute name like 'href', 'src', 'alt' timeout: Request timeout in seconds max_results: Maximum results to return (default 50, max 200)
Inputs: url (required), css_selector (required), attribute, timeout, +1 more. Returns: result
extract_linksA
Extract all links from a web page.
Args: url: The URL to fetch filter_domain: If True, only return links to the same domain timeout: Request timeout in seconds max_links: Maximum links to return (default 50, max 200)
Inputs: url (required), filter_domain, timeout, max_links. Returns: result
fetch_clean_textA
Fetch a web page and extract clean readable text.
Args: url: The URL to fetch max_chars: Maximum characters to return (default 10000, max 50000) timeout: Request timeout in seconds (default 15)
Inputs: url (required), max_chars, timeout. Returns: result
search_webA
Search the web using DuckDuckGo HTML search.
Args: query: Search query max_results: Number of results to return (default 5, max 20) timeout: Request timeout in seconds
Inputs: query (required), max_results, timeout. Returns: result
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
A4.1/5.0
Scored across 4 tools
Disambiguation5/5
Each tool has a clearly distinct purpose: extract content via CSS selectors, extract links, fetch clean text, and search the web. No overlap in functionality.
Naming Consistency5/5
All tool names follow a consistent verb_noun pattern (extract_by_selector, extract_links, fetch_clean_text, search_web), making them predictable and easy to understand.
Tool Count5/5
With 4 tools, the server is well-scoped for web scraping and search. Each tool earns its place without being excessive or insufficient.
Completeness4/5
The tool set covers the core web scraping operations: search, fetching clean text, extracting links, and extracting via CSS selectors. Minor gaps like handling cookies or downloading files exist, but the essential features are present.
Maintenance
ActivityInactive
ResponsivenessNo recent community issues
Related MCP Connectors
Web scraping MCP server: any URL to Markdown or JSON, with JavaScript rendering and batch jobs.
Web scraping MCP server — scrape, extract structured data, screenshot any site with anti-bot bypass.
One MCP server for all web data. Scrape, search, crawl, and AI research in one install.
Web search API and MCP server for developers building AI agents.
Related MCP Servers
- FlicenseNot gradedqualityCmaintenanceMCP server that exposes web_search and web_fetch tools, allowing LLM applications to search the web via DuckDuckGo and fetch page content as cleaned markdown.-
- AlicenseNot gradedqualityCmaintenanceMCP server for web search and content extraction using DuckDuckGo or SearXNG, with Playwright-based fetching and LLM-powered data extraction.139MIT
- AlicenseAqualityDmaintenanceAn MCP server that fetches web pages and extracts clean, AI-usable context from them, enabling tools for link discovery, content search, and integrated fetch-and-search operations.517 npm1MIT
