Scout
README.md
# Scout
A private search engine your coding agent calls over MCP.
Localhost only · five tools · no API key · the read path never touches the network.
**Status:** v1.3, works end to end. Python 3.12, SQLite + faiss. One loopback service, one thin stdio proxy per agent session.

*A 52-document RFC corpus. That is the tool call, the JSON it returns, and the answer — nothing left the machine.*
## What it does
- **`search`** — hybrid BM25 + vector search, 8 results inside 1,200 tokens by default
- **`expand`** — the whole stored chunk behind one result
- **`fetch`** — one page's text by URL
- **`corpus_stats`**, **`job_status`** — what you have, what is running
- **Honest coverage** — every answer carries `thin`, a `gap` (`entity:duckdb`) and a `recommend`, so the agent is told when Scout does not know
- **Grows on its own** — thin queries become crawl intents; a Brave key is optional and only ever finds new domains
## Install
Needs macOS or Linux, Python 3.12 and [uv](https://docs.astral.sh/uv/). Ollama is optional — without it, search is lexical only.
```bash
git clone https://github.com/paradise-runner/scout.git
cd scout
./scripts/bootstrap.sh # .venv from requirements.lock, ~25 s
```
## First run
```bash
# 1. A corpus. 50 RFCs, one to two minutes of polite fetching.
PYTHONPATH=.:src .venv/bin/python -m bootstrap rfc --db scout.db --from 9110 --limit 50
# 2. The service.
./src/api/scout serve --db scout.db &
# 3. The same JSON your agent gets.
./src/api/scout search "HTTP semantics"
# 4. Register: paste the printed block into ~/.pi/agent/mcp.json, then /reload in pi.
./src/api/scout register
```
Ask pi: **Use scout to find what RFC 9110 says about identifiers.**
## Use
| you want | run |
|---|---|
| service up / down | `./src/api/scout serve --db scout.db` · `Ctrl-C` |
| is it alive | `./src/api/scout serve --health` |
| add one page | `./src/api/scout add https://example.com/docs` |
| crawl a site | `PYTHONPATH=.:src .venv/bin/python -m crawl.cli --db scout.db --domain example.com --page-cap 200` |
| a real corpus | `src/bootstrap/README.md` (Wikipedia dump, RFC set) |
## Limits (v1)
- **Corpus** is only what you bootstrap or crawl. It is not an index of the public web.
- **`coverage.thin: true`** means "few results, small corpus". That is the contract working, not a bug.
- **HTML only.** No PDFs, video, or code repositories.
- **No auth.** Loopback only — do not bind `0.0.0.0`.
- **Vector search** needs a local Ollama and a built index; without them you get lexical answers.
- **Speed.** ~2–130 ms per query on the 50-RFC corpus above (`meta.took_ms`, warm to cold). A 3,400-document corpus can take seconds on a cold disk; numbers are in `bench/REPORT.md`.
- **No UI.** The v1.1 browser page is only in git history: `git checkout 183abb5 -- src/ui`.
## How it runs
```
scout serve one process: corpus, index, HTTP on 127.0.0.1:8771
python -m src.api.mcp_proxy one per agent session: five MCP tools, no corpus open
```
Agent sessions hold nothing but a socket, so any number of them share one warm corpus. Raw HTML is kept forever; SQLite, markdown and vectors rebuild from it offline.
## Development
```bash
./scripts/test-all.sh # 12 packages, one pytest run each
./scripts/test-all.sh api # just one
```
Every `src/<package>/README.md` documents that package: what it does, its tests, its repro command.
## Docs
[`PROJECT_VISION.md`](PROJECT_VISION.md) · [`SPEC.md`](SPEC.md) · [`ROADMAP.md`](ROADMAP.md) · [`deploy/`](deploy/README.md) · [`roadmap/status/`](roadmap/status/)
## License
Apache-2.0. See [`LICENSE`](LICENSE).
This server cannot be deployed
Maintenance
ActivityMaintained
ResponsivenessNo recent community issues