# WikiCleanHTML

An MCP server that fetches Wikipedia articles and returns cleaned, readable HTML — stripped of citations, infoboxes, navboxes, edit sections, and other clutter.

Also includes a standalone browser UI (`index.html`) that works without any server.

## Features

- Fetches any Wikipedia article by title, or a random one
- Strips Wikipedia-specific clutter: citations `[1]`, edit sections, infoboxes, navboxes, TOC, metadata
- Sanitizes output HTML with a strict tag/attribute allowlist
- Normalizes relative links to absolute Wikipedia URLs
- Saves cleaned HTML + plain text to local files
- Options to keep/remove images, tables, and links

## Quick Start — MCP Server

### 1. Clone and install

```bash
git clone https://github.com/alavna/wikicleanhtml.git
cd wikicleanhtml
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
```

### 2. Register with Claude Code

```bash
claude mcp add --scope user wikicleanhtml -- /path/to/wikicleanhtml/.venv/bin/python /path/to/wikicleanhtml/mcp_server.py
```

Or manually add to your MCP config:

```json
{
  "mcpServers": {
    "wikicleanhtml": {
      "command": "/path/to/wikicleanhtml/.venv/bin/python",
      "args": ["/path/to/wikicleanhtml/mcp_server.py"]
    }
  }
}
```

### 3. Use it

In Claude Code, just say:

> "Fetch the Wikipedia article on Insulin"

Claude calls `wiki_fetch_article(title="Insulin")` and gets back:

```json
{
  "title": "Insulin",
  "source_url": "https://en.wikipedia.org/wiki/Insulin",
  "html_file": "~/.wikicleanhtml/articles/Insulin.html",
  "text_file": "~/.wikicleanhtml/articles/Insulin.txt",
  "html_length": 45230,
  "text_length": 28100,
  "removed": { "references": 142, "infoboxes": 1, "navboxes": 4, "tables": 3 }
}
```

The cleaned article is saved locally — Claude can read the file if needed, without loading raw Wikipedia HTML into context.

## MCP Tools

| Tool | Description |
|------|-------------|
| `wiki_fetch_article` | Fetch a specific article by title |
| `wiki_fetch_random` | Fetch a random Wikipedia article |

**Options** (all tools):

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `title` | string | required | Wikipedia article title (fetch_article only) |
| `keep_images` | bool | `false` | Preserve `<img>` tags |
| `keep_tables` | bool | `false` | Preserve `<table>` elements |
| `keep_links` | bool | `true` | Preserve `<a>` hyperlinks (text always kept) |

Output files are saved to `~/.wikicleanhtml/articles/`.

## Browser UI

Open `index.html` directly in any browser. No server needed — it fetches from Wikipedia's CORS-enabled API and cleans client-side with DOMPurify.

## What gets removed

| Clutter | Selector examples |
|---------|-------------------|
| Citation markers | `sup.reference` (`[1]`, `[2]`, ...) |
| Edit sections | `.mw-editsection` |
| Hatnotes | `.hatnote` |
| Infoboxes | `.infobox` |
| Navigation boxes | `.navbox` |
| Table of contents | `#toc`, `.toc` |
| Reference lists | `.reflist`, `.references` |
| Message boxes | `.ambox`, `.tmbox`, etc. |
| Category links | `#catlinks` |
| Scripts, styles, metadata | `script`, `style`, `meta`, `link` |

## What gets preserved

Headings (`h1`-`h4`), paragraphs, lists (`ul`/`ol`/`li`), links, emphasis (`strong`/`em`), `code`/`pre`, `blockquote`, `br`/`hr`.

All output is sanitized through a strict allowlist — no inline styles, event handlers, scripts, iframes, or unknown tags survive.

## Tests

```bash
cd wikicleanhtml
source .venv/bin/activate
pytest tests/ -v
```

23 unit tests covering: reference removal, edit section removal, hatnote removal, infobox/navbox removal, TOC removal, image/table toggling, XSS sanitization, link normalization, structure preservation, and plain text extraction.

## Wikimedia API Etiquette

This tool follows [Wikimedia API guidelines](https://www.mediawiki.org/wiki/API:Etiquette):

- **User-Agent**: All requests include a descriptive `User-Agent` header
- **Rate**: One Wikipedia request per tool call — no bulk fetching
- **Caching**: Articles are saved to disk; re-read the file instead of re-fetching
- Not intended for high-traffic or production use without Wikimedia approval

## Project Structure

```
wikicleanhtml/
├── mcp_server.py      # MCP server entry point (2 tools)
├── cleaner.py         # HTML cleaning/sanitization logic
├── index.html         # Standalone browser UI
├── requirements.txt   # Python dependencies
├── tests/
│   └── test_cleaner.py
└── README.md
```

## License

MIT