clean-search-mcp 🧹

A lightweight MCP (Model Context Protocol) service that provides clean, spam-free search results for AI agents. Filters out content farms, SEO garbage, and low-quality sites before they reach your LLM.

Features

• Three search engines — Yandex + Bing + DuckDuckGo auto-fallback

• 176K+ domain blocklist — auto-updated from 25+ community sources, covers malware/scam/ads/content farms

• Three-layer filtering — domain blacklist → content rules → quality scoring

• Content extraction — full page text via trafilatura + selectolax

• Result scoring — 0-1 quality score (official docs 0.8 > tutorials 0.6 > garbage 0)

• LRU cache — search cache 6h, fetch cache 24h, auto-cleanup

• User blacklist — add domains on the fly, report bad results

• Deep mode — optional Playwright fallback for JS-heavy pages

• No heavy dependencies — pure HTTP, no browser required

Related MCP server: internet-context-mcp

Quick Start

pip install -r requirements.txt
python main.py

MCP Client Config

{
  "mcpServers": {
    "clean-search": {
      "command": "python",
      "args": ["/path/to/clean_search_mcp/main.py"]
    }
  }
}

Test Locally

python test_local.py "your search query" -n 5
python test_local.py "your query" -n 5 --no-content   # skip page content
python test_local.py "your query" --deep               # use Playwright fallback

API

clean_search(query, max_results=5, with_content=True, deep_mode=False)

Returns [{title, url, snippet, content, score}] sorted by quality.

add_user_blacklist(domain)

Add a domain to personal blocklist.

report_bad_result(url)

Report a low-quality URL (domain auto-blocked).

Configuration

Edit config.py to tune:

• Search providers: enable/disable Yandex, Bing, DuckDuckGo

• Blacklist sources: add or remove community blocklist URLs

• Scoring weights: adjust domain authority, content quality bonuses

• Caching: TTL, max files, cleanup interval

• Proxy: set PROXY for HTTP/Playwright

Dependencies

mcp, httpx, selectolax, trafilatura, duckduckgo-search

All lightweight pip packages. Playwright is optional (deep mode only).

License

MIT