webx-mcp-server
by ceeyang-ai
README.md
# WebX MCP Server 🌐
[](https://www.python.org/downloads/)
[](LICENSE)
[](https://github.com/ceeyang-ai/webx-mcp-server)
[](https://github.com/ceeyang-ai/webx-mcp-server)
[](https://github.com/ceeyang-ai/webx-mcp-server)
A **Model Context Protocol (MCP)** server for web content extraction — fetch clean text, extract links, query by CSS selector, and search the web.
> Built for AI agents. Works with **Hermes Agent**, **Claude Code**, **Cursor**, and any MCP-compatible client.
## ✨ Features
| Tool | Description |
|------|-------------|
| `fetch_clean_text` | Fetch a URL and extract clean readable text (up to 50K chars) |
| `extract_links` | Extract all links with anchor text, optionally filtered by domain |
| `extract_by_selector` | Extract content by CSS selector (text, HTML, or attribute) |
| `search_web` | Search the web via DuckDuckGo HTML search |
## 🚀 Quick Start
```bash
# Install from GitHub
pip install git+https://github.com/ceeyang-ai/webx-mcp-server.git
# Run as MCP server
webx-mcp-server
```
## 🔌 Usage with AI Agents
### Hermes Agent
Add to `~/.hermes/config.yaml`:
```yaml
mcp_servers:
webx:
command: "webx-mcp-server"
```
Restart → use `mcp_webx_fetch_clean_text`, `mcp_webx_extract_links`, etc.
### Claude Code / Cursor / Any MCP Client
Add to your MCP config:
```json
{
"mcpServers": {
"webx": {
"command": "webx-mcp-server"
}
}
}
```
## 📖 Examples
### Fetch clean text
```python
result = fetch_clean_text(
url="https://en.wikipedia.org/wiki/Web_scraping",
max_chars=5000
)
```
### Extract links
```python
result = extract_links(
url="https://example.com",
filter_domain=True,
max_links=20
)
```
### Search the web
```python
result = search_web(
query="latest AI research papers",
max_results=10
)
```
## 🛠 Requirements
- Python 3.10+
- requests ≥ 2.28
- beautifulsoup4 ≥ 4.11
- lxml ≥ 4.9
- mcp ≥ 1.0
## 👨💻 Development
```bash
git clone https://github.com/ceeyang-ai/webx-mcp-server.git
cd webx-mcp-server
pip install -e .
webx-mcp-server # Start MCP server
```
## 📄 License
MIT — free for personal and commercial use.
TDQS
A4.1/5.0
Scored across 4 tools
Disambiguation5/5
Each tool has a clearly distinct purpose: extract content via CSS selectors, extract links, fetch clean text, and search the web. No overlap in functionality.
Naming Consistency5/5
All tool names follow a consistent verb_noun pattern (extract_by_selector, extract_links, fetch_clean_text, search_web), making them predictable and easy to understand.
Tool Count5/5
With 4 tools, the server is well-scoped for web scraping and search. Each tool earns its place without being excessive or insufficient.
Completeness4/5
The tool set covers the core web scraping operations: search, fetching clean text, extracting links, and extracting via CSS selectors. Minor gaps like handling cookies or downloading files exist, but the essential features are present.
Maintenance
ActivityInactive
ResponsivenessNo recent community issues