# WebX MCP Server 🌐

[![Python](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
[![License](https://img.shields.io/badge/license-MIT-green.svg?style=flat-square)](LICENSE)
[![Stars](https://img.shields.io/github/stars/ceeyang-ai/webx-mcp-server?style=flat-square)](https://github.com/ceeyang-ai/webx-mcp-server)
[![MCP](https://img.shields.io/badge/MCP%20Server-5e81ac?style=flat-square)](https://github.com/ceeyang-ai/webx-mcp-server)
[![GitHub](https://img.shields.io/badge/GitHub-ceeyang--ai/webx--mcp--server-black?logo=github)](https://github.com/ceeyang-ai/webx-mcp-server)

A **Model Context Protocol (MCP)** server for web content extraction — fetch clean text, extract links, query by CSS selector, and search the web.

> Built for AI agents. Works with **Hermes Agent**, **Claude Code**, **Cursor**, and any MCP-compatible client.

## ✨ Features

| Tool | Description |
|------|-------------|
| `fetch_clean_text` | Fetch a URL and extract clean readable text (up to 50K chars) |
| `extract_links` | Extract all links with anchor text, optionally filtered by domain |
| `extract_by_selector` | Extract content by CSS selector (text, HTML, or attribute) |
| `search_web` | Search the web via DuckDuckGo HTML search |

## 🚀 Quick Start

```bash
# Install from GitHub
pip install git+https://github.com/ceeyang-ai/webx-mcp-server.git

# Run as MCP server
webx-mcp-server
```

## 🔌 Usage with AI Agents

### Hermes Agent

Add to `~/.hermes/config.yaml`:

```yaml
mcp_servers:
  webx:
    command: "webx-mcp-server"
```

Restart → use `mcp_webx_fetch_clean_text`, `mcp_webx_extract_links`, etc.

### Claude Code / Cursor / Any MCP Client

Add to your MCP config:

```json
{
  "mcpServers": {
    "webx": {
      "command": "webx-mcp-server"
    }
  }
}
```

## 📖 Examples

### Fetch clean text

```python
result = fetch_clean_text(
    url="https://en.wikipedia.org/wiki/Web_scraping",
    max_chars=5000
)
```

### Extract links

```python
result = extract_links(
    url="https://example.com",
    filter_domain=True,
    max_links=20
)
```

### Search the web

```python
result = search_web(
    query="latest AI research papers",
    max_results=10
)
```

## 🛠 Requirements

- Python 3.10+
- requests ≥ 2.28
- beautifulsoup4 ≥ 4.11
- lxml ≥ 4.9
- mcp ≥ 1.0

## 👨‍💻 Development

```bash
git clone https://github.com/ceeyang-ai/webx-mcp-server.git
cd webx-mcp-server
pip install -e .
webx-mcp-server  # Start MCP server
```

## 📄 License

MIT — free for personal and commercial use.