🔎 Deep Search MCP

Research beyond Google: academic databases, open data, web archives, independent search engines and the Tor dark web, behind a safety filter that never switches off.

Why · Information flow · Example · Tools · Sources · Safety · Install · Configuration · Development

💡 Why

Mainstream search engines rank for popularity and commerce. A lot of valuable material never makes it to page one:

• theses, preprints and open-access versions of paywalled papers sitting in university repositories;

• datasets, reports and other grey literature that journals never index;

• small, independent and personal websites that big engines bury;

• pages that were deleted or quietly rewritten;

• what is being said on Tor onion services.

Deep Search is a Model Context Protocol server that gives Claude (or any MCP client) one tool to search all of that at once, tools to read what it finds (HTML, PDF, .onion) and to recover old versions of pages. Every query, result and page goes through a safety layer, so research on sensitive subjects stays possible without surfacing abusive content.

Related MCP server: mcp-web-search

✨ Highlights

🔀 Information flow

Overview

flowchart TD
    U(["🙋 You"]) -->|"plain-language request"| C["🤖 Claude (MCP client)"]
    C -->|"tool call over stdio"| S["🔎 deep-search MCP server"]
    S --> Q{"🛡️ Safety guard<br/>checks the query"}
    Q -->|"refused channel"| R["Refusal, explained"]
    Q -->|"allowed"| F(("parallel<br/>fan-out"))
    F --> A["📚 Academic<br/>8 sources"]
    F --> D["🗄️ Open data<br/>2 sources"]
    F --> H["🏛️ Archive<br/>Internet Archive"]
    F --> W["🌐 Web, no Google<br/>3 sources"]
    F --> K["🧅 Dark web<br/>Ahmia"]
    A & D & H & W & K --> P["Normalize → filter → de-duplicate → paginate"]
    P --> O["Markdown or JSON, ≤ 25k characters"]
    O -->|"results + per-source status"| C

Anatomy of a search

sequenceDiagram
    autonumber
    participant C as Claude
    participant S as deep-search
    participant G as Safety guard
    participant P as Sources
    participant B as Ahmia blacklist
    C->>S: deepsearch_search(query, scopes)
    S->>G: checkQuery(query)
    G-->>S: channels to refuse
    par every allowed source
        S->>P: search(query, offset, limit)
        P-->>S: results, or an isolated error
    end
    S->>B: onion host banned?
    S->>G: classify(result, channel)
    S->>S: filters, dedupe, paginate
    S->>S: render ≤ 25k chars
    S-->>C: results by source, status, next_offset

Reading a page

flowchart TD
    I["URL, domain, PDF link,<br/>.onion address or Wayback snapshot"] --> N["Normalize<br/>(add scheme, raw Wayback form)"]
    N --> V{"Allowed?"}
    V -->|"local or private address"| X1["✖ Blocked by the SSRF guard"]
    V -->|".onion without Tor"| X2["✖ Explains how to enable Tor"]
    V -->|"onion on Ahmia's blacklist"| X3["✖ Never opened"]
    V -->|"ok"| G["Fetch: clear web directly,<br/>.onion through Tor (SOCKS5),<br/>every redirect re-checked"]
    G --> T{"Content type"}
    T -->|"image, video, binary"| X4["✖ Not downloaded"]
    T -->|"HTML"| H["Readable text and links"]
    T -->|"PDF"| PDF["Text layer of the PDF"]
    H & PDF --> SC{"Safety screening"}
    SC -->|"filtered content"| X5["✖ Withheld, no text returned"]
    SC -->|"clean"| OUT["Text chunk, filtered links<br/>and next_start_char"]

🧪 Example

You ask Claude: "Find research on post-quantum cryptography, and check what onion sites are saying about it."

Claude calls the tool with this input:

{
  "tool": "deepsearch_search",
  "arguments": {
    "query": "post-quantum cryptography",
    "scopes": ["academic", "web", "darkweb"],
    "limit_per_source": 2
  }
}

Output (real run, shortened: abstracts replaced by placeholders, onion addresses abbreviated, … marks omitted sources):

# Deep Search: "post-quantum cryptography"
Scopes: academic, web, darkweb · up to 2 per source
**Safety filter:** 3 results hidden (Hacking & malware services: 1, Fraud, carding & stolen data: 1, Violence for hire: 1).

## Academic literature

### OpenAlex — 2 of 53,349
1. **Post-quantum cryptography** (2017)
   Daniel J. Bernstein, Tanja Lange · Nature · article · cited 1,397× · open access · en
   https://doi.org/10.1038/nature23461
   PDF: https://research.tue.nl/nl/publications/5bbd4ea6-f4a5-452c-b9aa-f535cd8f4a50
2. **Post-Quantum Cryptography** (2009)
   Daniel J. Bernstein, Johannes A Buchmann, Erik Dahmen · Springer eBooks · book · cited 839× · en
   https://doi.org/10.1007/978-3-540-88702-7

### arXiv — 2 of 1,070
…
2. **Towards post-quantum blockchain: A review on blockchain cryptography resistant to quantum computing attacks** (2024)
   Tiago M. Fernandez-Carames, Paula Fraga-Lamas · arXiv cs.CR · preprint · open access
   https://arxiv.org/abs/2402.00922v1
   PDF: https://arxiv.org/pdf/2402.00922v1
   > (abstract snippet)

### BDTD — 2 of 37
1. **Attacking and defending post-quantum cryptography candidates** (2022)
   Paiva, Thales Areco Bandiera · BDTD (Brazilian theses & dissertations) · doctoral thesis · open access · eng
   https://bdtd.ibict.br/vufind/Record/USP_ff089afa0775bd5e3004182a245cdfd8
   > (abstract snippet)
…

## Independent web search (no Google)

### Marginalia Search — error: Marginalia's shared 'public' API key is over its rate limit. A free personal key (e-mail contact@marginalia-search.com) set as MARGINALIA_API_KEY avoids this.

### DuckDuckGo — 2 of 10
1. **Post-Quantum Cryptography | CSRC**
   https://csrc.nist.gov/projects/post-quantum-cryptography
   > NIST's Post-Quantum Cryptography (PQC) project leads the national and global effort…
…

## Dark web (Tor onion services)

### Ahmia — 2 of 172 · 3 hidden by safety filter
1. **Blog Stéphane Bortzmeyer: RFC 9958: Post-Quantum Cryptography for Engineers**
   sjnrk23r…s3eyid.onion · last seen Nov. 1, 2026, 7:55 a.m.
   http://sjnrk23r…s3eyid.onion/9958.html
2. **Quantum Computing Spurs Urgency on Cryptography Upgrades**
   wgruul6c…zcx3yd.onion · last seen Nov. 2, 2026, 12:49 a.m.
   http://wgruul6c…zcx3yd.onion/quantum-computing-spurs-urgency-on-cryptography-upgrades

> .onion links only open through Tor. deepsearch_fetch_page can read them when DEEP_SEARCH_TOR_PROXY is configured. Treat dark web content as unverified.

---
1 source(s) failed; the rest are shown.
More results: call again with offset=2.

Things to notice: the filter hid three dark web listings (hacker-for-hire, carding and violence-for-hire spam that piggybacks on popular keywords), one rate-limited source failed without breaking the search, and the footer tells Claude how to get the next page.

{
  "query": "post-quantum cryptography",
  "scopes": ["academic", "web", "darkweb"],
  "offset": 0,
  "limit_per_source": 2,
  "sources": [
    { "id": "openalex", "name": "OpenAlex", "scope": "academic", "status": "ok", "count": 2, "total": 53349, "has_more": true, "hidden": 0, "duplicates": 0 },
    { "id": "marginalia", "name": "Marginalia Search", "scope": "web", "status": "error", "count": 0, "has_more": false, "hidden": 0, "duplicates": 0, "error": "Marginalia's shared 'public' API key is over its rate limit. …" },
    { "id": "ahmia", "name": "Ahmia", "scope": "darkweb", "status": "ok", "count": 2, "total": 172, "has_more": true, "hidden": 3, "duplicates": 0 }
  ],
  "results": [
    {
      "source": "openalex",
      "title": "Post-quantum cryptography",
      "url": "https://doi.org/10.1038/nature23461",
      "authors": ["Daniel J. Bernstein", "Tanja Lange"],
      "year": 2017,
      "venue": "Nature",
      "doi": "10.1038/nature23461",
      "pdf_url": "https://research.tue.nl/nl/publications/5bbd4ea6-f4a5-452c-b9aa-f535cd8f4a50",
      "open_access": true,
      "type": "article",
      "citations": 1397,
      "language": "en"
    }
  ],
  "safety": {
    "hidden": 3,
    "by_reason": { "hacking_services": 1, "fraud": 1, "violence_for_hire": 1 },
    "refused": [],
    "banned_list_loaded": true
  },
  "has_more": true,
  "next_offset": 2
}

The JSON text is sent compact (no indentation) and the same object is returned as MCP structuredContent.

More things to ask:

• "Survey studies on glyphosate and cancer since 2015, including Brazilian theses."

• "This government page was taken down. Show me its archived versions and what changed."

• "Find independent blogs about the history of computing in Brazil, excluding Wikipedia."

• Run the deepsearch_investigate prompt with a topic for a guided, cited investigation.

🧰 Tools

All tools are read-only and support "response_format": "markdown" (default) or "json".

📚 Sources

Bold sources run by default. The others run when their key is set, or when requested through the sources parameter.

🛡️ Safety model

The filter is a core part of the project and cannot be switched off entirely.

Locked categories (no configuration can disable them):

Configurable categories (on by default): sexual, gore, violence_for_hire, weapons, drugs, fraud, hacking_services, extremism. They target listings and transactions ("buy cocaine", "guns for sale", "hacker for hire"), not topics, so news and research about those subjects still come through.

Channels: every source and page is screened according to where it comes from.

Other protections:

• Ahmia's blacklist: ~60,000 onion domains banned for abuse are dropped from results and never opened. Onion pages are not opened at all if the list cannot be loaded (fail-closed). The list is cached per user (never in a shared temp folder), rejected if implausibly small, and downloaded through Ahmia's onion mirror when Tor is configured.

• No side doors: onion addresses hidden inside other URLs (tor2web gateways such as <address>.onion.ly, translation proxies, Wayback copies) are recognized and face the same blacklist and dark web rules.

• Evasion-resistant matching: invisible characters, leetspeak and Cyrillic/Greek look-alike letters are normalized before any rule runs.

• Text only: pages are read as text; images, video, downloads and binary data are never returned. A page dominated by filtered content is withheld.

• SSRF guard: localhost, private networks and cloud metadata endpoints are refused, including after redirects.

• Tor all the way: once a request enters Tor, every redirect stays on Tor, so an onion page cannot bounce you to the clear web and reveal your IP. .onion names are resolved by Tor, never by your DNS.

• Keys stay home: API keys and cookies are never forwarded when a redirect leaves the original domain, and error messages never include URLs that could carry keys.

• Untrusted content is labeled: page text is marked as third-party data, a basic defense against prompt injection.

• Hostile files can't take the server down: PDFs are parsed one at a time in an isolated worker thread with page, text, time and memory limits (including a watchdog on decompression buffers); HTML, titles and metadata are clipped and processed with linear-time code, so crafted content cannot freeze the server.

• No evasion of others' defenses: the client identifies itself as deep-search-mcp on the clear web and never solves or bypasses CAPTCHAs or bot checks. Over Tor it uses the Tor Browser User-Agent, to stay in its anonymity set.

• A refusal is final: tool descriptions tell the model not to rephrase a query to get around a refusal.

Found a security problem? See SECURITY.md.

📦 Installation

Requires Node.js 20.18+.

git clone https://github.com/Edx0n/deep-search-mcp.git
cd deep-search-mcp
npm install
npm run build

Claude Code

claude mcp add --scope user deep-search -- node /path/to/deep-search-mcp/dist/index.js

With environment variables (Tor in this example):

claude mcp add --scope user deep-search -e DEEP_SEARCH_TOR_PROXY=socks5h://127.0.0.1:9150 -- node /path/to/deep-search-mcp/dist/index.js

Claude Desktop: add to claude_desktop_config.json:

{
  "mcpServers": {
    "deep-search": {
      "command": "node",
      "args": ["/path/to/deep-search-mcp/dist/index.js"],
      "env": {
        "DEEP_SEARCH_TOR_PROXY": "socks5h://127.0.0.1:9150"
      }
    }
  }
}

On Windows, write the path as C:\\path\\to\\deep-search-mcp\\dist\\index.js in JSON (or use forward slashes).

⚙️ Configuration

Every variable is optional; see .env.example.

🧅 Tor (optional)

Dark web search works without Tor (through ahmia.fi). Tor is needed to read .onion pages and to search through Ahmia's own onion mirror.

• Tor Browser: open it and keep it running; the proxy is socks5h://127.0.0.1:9150.

• tor service (Tor Expert Bundle from torproject.org): run tor; the proxy is socks5h://127.0.0.1:9050.

Use socks5h so that .onion names are resolved by Tor, never by your DNS.

What goes through Tor (DEEP_SEARCH_TOR_ROUTE):

With sensitive or all, a request that must use Tor fails when Tor is not running instead of going out directly, so your IP never leaks by accident. Tor hides who searches, not what: each service still sees the query it receives.

Tested on the live Tor network (through a Snowflake bridge): every source works over Tor except DuckDuckGo, which answers Tor users with a bot check (reported, never bypassed). Expect a search over Tor to take 20–45 s instead of a few seconds, mostly the dark web index; onion services sometimes need a second attempt, which the client makes automatically.

🛠️ Development

npm test

npm run build && npm run handshake

npm run build && npm run smoke

npm run live -- "your query"

• npm test: unit tests without network. Covers the safety rules (including a false-positive suite), every source parser, the HTTP client against a simulated SOCKS5/Tor proxy, HTML extraction, the search orchestrator and output sizing.

• npm run handshake: offline check of the built server over stdio (tools, prompt, sources, locked policy, input validation, refusals, SSRF guard, Tor requirement). No network needed.

• npm run smoke: end-to-end check. Starts the built server over stdio like an MCP host and calls every tool against the real services.

• npm run live: runs each source once and prints status, counts and timing.

Continuous integration: every push and pull request runs GitHub Actions on Ubuntu and Windows with Node 22 and 24 (typecheck, unit tests, build, handshake), plus a build-and-handshake job on Node 20.18, the minimum supported version. The network-dependent smoke and live scripts stay manual, so CI never depends on third-party rate limits.

src/
├── index.ts / server.ts   stdio entry point, tools and prompt (MCP SDK v2, Zod 4)
├── search.ts              orchestrator: selection, refusal, fan-out, filter, dedupe, pagination
├── format.ts              Markdown/JSON rendering under 25k characters
├── http/                  undici client (SSRF guard, Tor via SOCKS5, size limits)
├── safety/                rules, normalization, guard, Ahmia blacklist, scholarly hosts
├── sources/               one parser + search() per source, grouped by scope
├── extract/               HTML → text + links, PDF → text
└── tools/                 page reader and Wayback lookup

Design notes and trade-offs: docs/DESIGN.md.

⚠️ Known limitations

• Rate limits: Semantic Scholar and Marginalia's public key are usually exhausted; anonymous CORE and Crossref (1 request/s, already queued) have quotas too. Free keys solve it.

• DuckDuckGo: first page only. It may answer with a bot check, in which case the source reports an error and the others carry on.

• Ahmia: broad queries return thousands of results and take around 10 seconds. Onion services are often slow or offline.

• Strict on the open web: clear-web pages about child abuse (news, encyclopedias) are withheld as well. Use the academic scope to research the subject.

• Scanned PDFs without a text layer (no OCR) yield no text.

• Tor fingerprint: requests over Tor use Tor Browser's User-Agent and language, but not its full set and order of headers, so an onion operator could tell this tool apart from a browser. That never reveals your IP; for anonymity-critical browsing, use Tor Browser itself.

• Filters are keyword-based: they handle accents, leetspeak, invisible characters, look-alike letters and URL encoding, but deliberate misspellings and new slang can still slip through. Ahmia's own filtering and its blacklist remain the main defense on the dark web.

• Torch, Haystak and similar onion engines: Torch's widely shared addresses are v2 onions (retired in 2021), Haystak's circulate in a malformed form and, above all, these indexes do not filter abuse content. Ahmia is the only one with a maintained blacklist.

• BASE requires registering your IP address to use its API.

• OATD, SciELO's own search and Internet Archive Scholar block automated access (SciELO is covered through Crossref).

• Mojeek answers automated requests with a CAPTCHA, which this project does not bypass.

🤝 Responsible use

This is a tool for research, journalism and OSINT. Accessing Tor may be legal where you live, but what you open and what you do with it is your responsibility. Do not use this project to obtain, distribute or facilitate illegal content or activities.

📄 License

MIT © Edx0n