# 深搜 · shensou

**Divergent iterative web search.** Anchor on one question; an *explorer* LLM keeps
proposing keyword queries, reads what the web returns, distills it into findings,
and then branches the next round toward the facets it *hasn't* understood yet —
with an adversarial slice that deliberately hunts **counter-evidence**. Low-yield
queries are triaged into genuine **knowledge gaps** vs. bad phrasing. It ends with
a **question-map** (distinct branches + next steps) and a big-picture **synthesis**.

Not "one query → ten links." It's a loop that *opens a question* — novelty-deduped
so it keeps reaching for new angles instead of re-searching what it already knows.

Provider-agnostic: the explorer is any OpenAI-compatible model; search is the Brave
Search API. Ships as an **MCP tool**, a **web demo**, and a **CLI**.

![深搜 run](docs/screenshot.png)

<sub>*Illustrative example — rounds of queries (extend + counter), findings, blind spots, and the final question-map.*</sub>

## How the loop works

```
question
   │  explorer LLM seeds N keyword queries (extend + counter)
   ▼
 ┌── search each (Brave) ──► evidence
 │        │
 │        ▼  explorer digests → findings (facets on the question)
 │        ▼  proposes next round's queries — cover what's NOT yet understood
 │   novelty filter (Jaccard) drops anything too close to past queries
 └───◄ loop until budget / saturation / max rounds
   │
   ├─ triage low-yield queries → knowledge gaps vs typos
   ├─ curate → question-map { direct, overview, branches[next/premise/cost], more }
   └─ synthesize → big-picture narrative
```

Key behaviours: **anchored** (every query serves the main question, no drift),
**adversarial** (each round seeks reasons it's wrong), **coverage-driven** (branches
toward gaps), **novelty-deduped** (won't spin on near-duplicate searches).

## Quick start

```bash
git clone https://github.com/taoyongac/shensou
cd shensou
pip install -r requirements.txt
cp .env.example .env        # set EXPLORER_MODEL/API_KEY + BRAVE_API_KEY
```

**CLI**
```bash
set -a; source .env; set +a
python -m shensou "CRISPR base editor off-target 2025 controversy"
python -m shensou -b 32 "your question"      # bigger search budget
python -m shensou --json "your question" > record.json
```

**Web demo**
```bash
set -a; source .env; set +a
python web/server.py        # → http://127.0.0.1:8040
```

**MCP server** — add to your host config (e.g. Claude Code `~/.claude.json`), with
the `EXPLORER_*` + `BRAVE_API_KEY` vars in its environment:
```json
{
  "mcpServers": {
    "shensou": {
      "command": "python",
      "args": ["/abs/path/to/shensou/mcp_server/server.py"],
      "env": { "EXPLORER_MODEL": "gpt-4o", "EXPLORER_API_KEY": "sk-...",
               "BRAVE_API_KEY": "..." }
    }
  }
}
```
Then call `deep_search(question, budget)` — it runs the full loop and returns a
structured record (findings, blind spots, map, synthesis). It makes many sequential
LLM + search calls, so a run takes minutes; size it with `budget` (4–96).

## Configuration

All via environment (see `.env.example`).

| var | meaning | default |
|---|---|---|
| `EXPLORER_MODEL` | explorer model id (**required**) | `OPENAI_MODEL` |
| `EXPLORER_BASE_URL` / `EXPLORER_API_KEY` | OpenAI-compatible endpoint | `OPENAI_*` |
| `BRAVE_API_KEY` / `BRAVE_API_KEYS` | Brave Search key(s) | — |
| `SHENSOU_BUDGET` | default searches per run | 24 |
| `SHENSOU_ROUND_WIDTH` / `SHENSOU_NARROW_WIDTH` | queries per round / late-game | 5 / 2 |
| `SHENSOU_MAX_ROUNDS` | round cap | 12 |
| `SHENSOU_TAU` | novelty threshold (0–1) | 0.30 |
| `SHENSOU_ADVERSARIAL` | include counter-evidence queries | 1 |
| `SHENSOU_LANG` / `SHENSOU_COUNTRY` | Brave search locale | en / US |

The web demo also reads `SHENSOU_HOST/PORT` and optional `SHENSOU_USER/PASS`
(HTTP Basic Auth). A global lock serializes web runs (concurrency = 1) so a shared
demo never piles up parallel searches.

## Notes

- Get a Brave Search API key at <https://brave.com/search/api/> (free tier exists).
- Multiple `BRAVE_API_KEYS` are rotated on rate-limit; calls are serialized to
  respect the ~1 req/s free-tier cap. No usage state is written to disk.
- The search provider is the one swappable piece — `shensou/search.py` is a small
  typed module; drop in another engine behind the same `search(q, count)` contract.

---

## From the Tao Lab

Built and used at **[Tao Lab](https://taolab.tail0ea5ac.ts.net/), School of Life
Sciences, Yunnan University** (云南大学 · 陶勇课题组) — epigenetics, aging, cancer,
and **AI-for-Science**. 深搜 is the lab's tool for turning a fuzzy question into a
mapped, evidence-anchored understanding — and its sibling
[**三人行 / sanrenxing**](https://github.com/taoyongac/sanrenxing) opens the same
question divergently across three AI seats.

🔗 **Lab site:** https://taolab.tail0ea5ac.ts.net/

## License

MIT © 2026 Yong Tao (Tao Lab, Yunnan University). See [LICENSE](LICENSE).