Skip to main content
Glama
webx-mcp-server
by ceeyang-ai
README.md
# WebX MCP Server 🌐

[![Python](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
[![License](https://img.shields.io/badge/license-MIT-green.svg?style=flat-square)](LICENSE)
[![Stars](https://img.shields.io/github/stars/ceeyang-ai/webx-mcp-server?style=flat-square)](https://github.com/ceeyang-ai/webx-mcp-server)
[![MCP](https://img.shields.io/badge/MCP%20Server-5e81ac?style=flat-square)](https://github.com/ceeyang-ai/webx-mcp-server)
[![GitHub](https://img.shields.io/badge/GitHub-ceeyang--ai/webx--mcp--server-black?logo=github)](https://github.com/ceeyang-ai/webx-mcp-server)

A **Model Context Protocol (MCP)** server for web content extraction — fetch clean text, extract links, query by CSS selector, and search the web.

> Built for AI agents. Works with **Hermes Agent**, **Claude Code**, **Cursor**, and any MCP-compatible client.

## ✨ Features

| Tool | Description |
|------|-------------|
| `fetch_clean_text` | Fetch a URL and extract clean readable text (up to 50K chars) |
| `extract_links` | Extract all links with anchor text, optionally filtered by domain |
| `extract_by_selector` | Extract content by CSS selector (text, HTML, or attribute) |
| `search_web` | Search the web via DuckDuckGo HTML search |

## 🚀 Quick Start

```bash
# Install from GitHub
pip install git+https://github.com/ceeyang-ai/webx-mcp-server.git

# Run as MCP server
webx-mcp-server
```

## 🔌 Usage with AI Agents

### Hermes Agent

Add to `~/.hermes/config.yaml`:

```yaml
mcp_servers:
  webx:
    command: "webx-mcp-server"
```

Restart → use `mcp_webx_fetch_clean_text`, `mcp_webx_extract_links`, etc.

### Claude Code / Cursor / Any MCP Client

Add to your MCP config:

```json
{
  "mcpServers": {
    "webx": {
      "command": "webx-mcp-server"
    }
  }
}
```

## 📖 Examples

### Fetch clean text

```python
result = fetch_clean_text(
    url="https://en.wikipedia.org/wiki/Web_scraping",
    max_chars=5000
)
```

### Extract links

```python
result = extract_links(
    url="https://example.com",
    filter_domain=True,
    max_links=20
)
```

### Search the web

```python
result = search_web(
    query="latest AI research papers",
    max_results=10
)
```

## 🛠 Requirements

- Python 3.10+
- requests ≥ 2.28
- beautifulsoup4 ≥ 4.11
- lxml ≥ 4.9
- mcp ≥ 1.0

## 👨‍💻 Development

```bash
git clone https://github.com/ceeyang-ai/webx-mcp-server.git
cd webx-mcp-server
pip install -e .
webx-mcp-server  # Start MCP server
```

## 📄 License

MIT — free for personal and commercial use.

TDQS

A4.1/5.0

Scored across 4 tools

Disambiguation5/5

Each tool has a clearly distinct purpose: extract content via CSS selectors, extract links, fetch clean text, and search the web. No overlap in functionality.

Naming Consistency5/5

All tool names follow a consistent verb_noun pattern (extract_by_selector, extract_links, fetch_clean_text, search_web), making them predictable and easy to understand.

Tool Count5/5

With 4 tools, the server is well-scoped for web scraping and search. Each tool earns its place without being excessive or insufficient.

Completeness4/5

The tool set covers the core web scraping operations: search, fetching clean text, extracting links, and extracting via CSS selectors. Minor gaps like handling cookies or downloading files exist, but the essential features are present.

Maintenance

ActivityInactive
ResponsivenessNo recent community issues