ag-mcp-search
by AG-Bureau
README.md
<div align="center">
# search
**A self-hosted MCP server for web search that reports how much of each answer to believe**
[](https://registry.modelcontextprotocol.io)
[](LICENSE)
[](adapter/Dockerfile)
[](#-install)
[](https://glama.ai/mcp/servers/AG-Bureau/mcp-search)
</div>
---
A search tool fails in ways that look exactly like success. An engine answers with
somebody else's subject. A page returns text that is an anti-bot shield. Sixteen
sources turn out to be two engines counted eight times. None of that raises an
error, and the model on the other end builds on it.
**This server's job is to make those cases distinguishable, in fields you can
branch on.** It runs on your machine, over your own metasearch instance, with your
own model key — or none at all.
## 🔧 Tools
| Tool | What it does | Required | Notable options |
|---|---|---|---|
| `web_search` | Finds pages **and reads the top ones** — one call, links with their text | `query` | `read: false` for links only · `read_top` how many to read · `min_engines` to force breadth · `corroborate` |
| `web_read` | Reads pages by address: text, PDF, or a scan recognised by a vision model | `urls` | `mode: browser` for JS-rendered pages · `expect` to assert what must be there · `offset` to continue |
| `web_image_search` | Finds images: the address of the FILE and, separately, of the page it sits on | `query` | `max_results`, `page` |
| `web_screenshot` | A PNG of a page **plus its text from the same visit**, so the two can be cross-checked | `url` | `max_chars` for how much text · `full_page` · `expect` |
| `web_deep_search` | Composes its own queries, reads in waves, and answers from several sources — saying what it could not confirm | `question` | `waves` |
Full argument reference, response shapes and failure modes: **[HOWTO-CALL.md](HOWTO-CALL.md)**.
## 📦 Install
From an open repository page to a working answer. Nothing is assumed to be on
your disk already:
```bash
git clone https://github.com/AG-Bureau/mcp-search
cd mcp-search
cp .env.example .env
echo "SEARXNG_SECRET=$(openssl rand -hex 32)" >> .env
docker compose -f docker-compose.yml -f wiring/expose-localhost.yml up -d --build
curl -s http://127.0.0.1:8081/healthz
```
The fourth line is not decoration. Without a value in `SEARXNG_SECRET` the very
next command refuses — and that refusal is deliberate: with no key of its own the
metasearch does not fail, it comes up with a publicly known one from its image
template, silently.
The overlay publishes the port **on loopback only**. A published container port
does not go through the host firewall's usual chain, so exposing it more widely
is a separate, deliberate step — see [Deployment](#-deployment-and-exposure).
### Two transports
MCP has two, and they answer different questions. **HTTP** — the commands above —
is for a server that is already running somewhere. **stdio** is the protocol's
default: the client starts the server as a process and talks to it through the
pipes, which is how most desktop clients and wrappers work.
```bash
python adapter/server.py --stdio # or MCP_TRANSPORT=stdio
```
One JSON-RPC object per line in, one answer per line out. The mode is chosen
explicitly and never guessed from whether a terminal is attached — that sign
merely sits next to the subject, and one day it answers for a case nobody meant.
In stdio mode **stdout is the protocol**: answers and nothing else, with the log
on stderr. One stray line of anything else breaks the client reading it.
The sidecars do not depend on the choice. Started by a client with no compose
project around it, the module still works and names what is missing instead of
pretending: the browser path reports `not_wired_up`, and `trouble` carries
`pool_unmeasured` — the engine pool was never computed from observation.
## ⬆️ Upgrading from 0.2.x — the answer changed shape
**If you already run 0.2.1 or earlier, read this before updating.** Nothing here
is a new feature you may ignore; it is what your existing calls will return
differently.
**The search and image answers carry fewer fields by default**, and they say so
in their name: `ag.search/3` and `ag.images/3` instead of `/2`. A caller that
branched on `contract` will break loudly, which is the intent — a field that
simply vanished would read as "nothing was wrong" in most languages.
| in 0.2.x | in 0.3 |
|---|---|
| `search_aborted`, `engines_unasked` | `trouble.search_aborted`, `trouble.engines_unasked` |
| `unresponsive_engines` | `trouble.unresponsive_engines` |
| `engines_irrelevant` | `trouble.engines_irrelevant` |
| `pool_source: "seed"` | `trouble.pool_unmeasured`, with the reason |
| `arguments_adjusted` | `trouble.arguments_adjusted` |
| `corroborated_by_url`, `corroborated_by_domain` — always present, `1` on the cheap path | absent when only one engine found results: there `1` meant "nobody else was asked", not a measurement |
| `count`, `query`, `page`, `read`, `read_top`, `pages_*`, `timing_ms`, `engines_skipped`, `engines_used`, `tiers_used`, `pool_source`, `pool_reason` | returned when you ask: `verbose: true` |
`trouble` is always present and empty when nothing went wrong, so `if not
trouble` replaces the four separate checks. Nothing was deleted from the module —
the accounting moved behind a request.
**Booleans are now parsed rather than cast.** In 0.2.x `"read": "false"` over MCP
read the pages anyway — eight times the wall clock — while the plain door
understood the same word correctly. Both doors now accept `true/false`, `1/0`,
`yes/no`, `on/off`, `y/n`, `t/f` and Python's `True/False`, case-blind. **A value that cannot be read
turns the flag off and is named in `arguments_adjusted`**, and an EMPTY value
counts as unreadable: `read=""` no longer buys the expensive default in silence.
**`read_top: 0` means "no preference"**, not "read nothing" — for nothing, use
`read: false`. The schema used to declare `minimum: 1` while accepting zero.
The full value-by-value history, including the Russian field values of `/1`, is
in the contracts: [`contracts/ag.search.v3.md`](contracts/ag.search.v3.md) and
[`contracts/ag.images.v3.md`](contracts/ag.images.v3.md), section "What changed".
Reading (`ag.read/2`), deep search (`ag.deep/2`) and screenshots (`ag.shot/1`)
kept their numbers: they gained fields, and an addition breaks nobody.
## ⚙️ Configuration
| Variable | Required | What it is |
|---|---|---|
| `SEARXNG_SECRET` | **yes** | Session key for the metasearch. Any long random string that is not from somebody's history. |
| `LLM_API_KEY` | no | Key for any OpenAI-compatible endpoint. **Secret.** |
| `LLM_API_BASE` | no | Base URL of that endpoint. Take it from your provider's documentation, not by analogy — the obvious guess can answer `429: Insufficient balance` because the subscription lives on a different path of the same domain. |
| `LLM_MODEL_TEXT` | no | Model that plans queries and composes answers. No default is shipped: a default would silently ask your provider for a model it may not have. |
| `LLM_MODEL_VISION` | no | Model that reads scanned PDFs. Unset, such documents return an explicit refusal naming the reason. |
| `LLM_DISABLE_THINKING` | no | Set for providers whose reasoning budget swallows the answer, leaving it empty with `finish_reason: length`. |
| `READ_CONTACT` | no | Contact placed in the `User-Agent` when fetching pages. Defaults to this repository; set your own if you run this at scale. |
| `READ_LANGUAGES` | no | `Accept-Language` when reading. Unset by default — the language of the pages you read is not ours to choose. |
Pacing, pool size and read limits have their own variables with measured
defaults; see [`.env.example`](.env.example), which explains each one where you
set it.
**A model key is optional.** Search, reading, image search and screenshots are
HTTP requests and spend no model tokens. A model is called in exactly two places,
and both are named in the answer: `web_deep_search`, and recognising a PDF with no
text layer — which happens only when you ask to read such a document, never behind
your back in a search.
## 🎯 What a bundled search tool does not do
**Cost you control.** One argument changes the answer by an order of magnitude:
| call | payload | time | model tokens |
|---|---|---|---|
| `read: false`, 6 links | 3.8 KB | 0.6 s | **0** |
| `read_top: 1`, 3 links | 8.8 KB | 1.6 s | **0** |
| `read_top: 3`, 6 links | 9.8 KB | 6.3 s | **0** |
| `web_deep_search` | full account | 36 s | 6 calls |
*Measured on one machine, one query. Take the shape, not the digits.* A consumer
measured the same fork from outside and got 4.8-10.5 s against 0.7 s — the shape
holds, the digits depend on the pages the query happens to find.
**The choice is made before the call, not after the bill.** `read: false` when
you are mapping what exists or working under a narrow context ceiling; the
default when you want the text of the top results and would otherwise fetch it
yourself. `read_top: 0` means "no preference", not "read nothing".
**The engine list maintains itself.** A hand-written list goes stale in silence:
an engine that was the best returns nothing weeks later and says nothing about it.
Ours was revised three times in a single day — each revision against the previous
one, each correct on its own data. The problem was never the engines: a decision
freezes while observation goes on.
So the list is not written here. A prober asks every known engine, continuously,
with questions whose correct answer is known in advance, and the pool is the best
few by reference hit share — recomputed on its own. Verified by falsification: a
planted bad run took an engine out of the pool **with no code change**, and
restoring the run brought it back by itself.
Until enough observation accumulates, the pool is a seed list and every answer
says so: `trouble.pool_unmeasured` by default, `pool_source` itself under
`verbose`.
**Failure is distinguishable from success.** Four ways an engine can fail, and
what shows each:
| how it fails | what shows it |
|---|---|
| answers with a refusal: captcha, rate limit, ban | `trouble.unresponsive_engines` |
| silently returns nothing | the difference between `engines_asked` and `engines_answered` |
| answers a different question | `trouble.engines_irrelevant` — its results are already discarded |
| substitutes the subject with a better-indexed namesake | `engines_trust`, earned against references |
The same applies to reading: seven distinct outcomes, and a page that returned a
shield is `stub`, not empty text.
## ⚖️ What it does with `robots.txt`, and why you must decide
**The module REPORTS a site's rules and does not enforce them.** Every read
carries `robots`: `allowed`, `disallowed_by_site`, or `not_checked` when the file
could not be read. A page a site forbids is still fetched, and the answer says so.
That is a decision, not an omission, and it belongs to whoever runs this rather
than to the module. Two reasons. Whether a tool called by a person obeys
robots.txt is the operator's call — a rule written for crawlers indexing the web
is not obviously a rule for fetching one page a user asked for. And the reference
behaviour — treat `401`/`403` on robots.txt as a ban — produces false bans on
ordinary sources, because the same sites answer `401` to everyone from behind an
anti-bot service.
**So the gate is yours to add.** If your use requires obeying robots, branch on
the field: `robots == "disallowed_by_site"` means the site says no. If you obey
it, treat `not_checked` as a stop too — it means we could not read the rules, not
that there are none.
## 📖 How it works
- **[ALGORITHM.md](ALGORITHM.md)** — what happens, step by step, on each call.
- **[contracts/](contracts/)** — the call contracts, versioned separately from
the code that implements them.
- **[measures/](measures/)** — dated measurements: which engines were alive, what
the load ladder gives, what the transport change bought. Numbers, with what was
measured and when.
- **[contracts/ag.search.v3.md](contracts/ag.search.v3.md)** — and its
neighbours: what each capability promises, plus the table of what changed
against `/1` and `/2` for anyone with stored answers to read.
## 🔒 Deployment and exposure
`wiring/expose-localhost.yml` publishes the adapter on `127.0.0.1` only. Anything
wider is a separate overlay, and its header says what to check first: Docker
passes traffic to published ports through `FORWARD` after DNAT, while the
firewall's own chain sits before its hooks — so a firewall that says "closed" can
be open to the internet on a published port.
A search server open to the outside is an open proxy that goes to the network in
the machine owner's name.
## ✅ Tests
```bash
IMAGE=ag-mod-search/adapter:0.3.0 bash tests/in-image.sh
```
Three suites — the protocol and search against a fake metasearch, reading against
a fake site, the computed pool against a database built in memory. **Not one of
them makes a single outbound request**, and the runner holds that with
`--network none` rather than on trust: for reading it matters more than for
search, because a test that went to the internet would spend the very resource
the tool protects — the reputation of the one address it calls from.
They run inside the built image rather than on the machine where the code is
edited: the PDF parser lives in the image, and a suite run outside would skip
everything that touches it. The skip is not silent — the check goes red with a
note saying where to run it.
What these suites cannot check is written down in
[tests/README.md](tests/README.md).
## 🤝 Contributing
A capability, engine or heuristic is not accepted until its **reference
attribute** is declared — a property of the correct answer that the thing being
tested could not have told us itself — and a pool of checked questions is
attached. See [CONTRIBUTING](https://github.com/AG-Bureau/.github/blob/main/.github/CONTRIBUTING.md).
## 📄 License
[GNU Affero General Public License v3.0](LICENSE). Run it, change it, build on
it. If you make it available to others OVER A NETWORK, the changes you made go
back out under the same licence — that is the one obligation, and running a
service counts as making it available.
For whoever cannot live with that clause, a commercial licence is a question to
ask rather than a fork to make.
TDQS
A4.9/5.0
Scored across 5 tools
Disambiguation5/5
Each tool has a clearly distinct purpose: web_search finds pages, web_read extracts content from a known address, web_deep_search synthesizes answers from multiple sources, web_image_search locates images, and web_screenshot captures visual page previews. No two tools overlap in a way that would confuse an agent.
Naming Consistency5/5
All tool names share the predictable 'web_' prefix and follow a consistent pattern: web_search, web_read, web_image_search, web_screenshot, web_deep_search. The naming convention is uniform and signals the tool's function clearly.
Tool Count5/5
Five tools is an ideal size for a web-focused module. It covers search, reading, image lookup, deep synthesis, and screenshotting without redundancy or bloat. Each tool earns its place.
Completeness5/5
The tool surface covers the full web research lifecycle: discovering pages, reading their content, synthesizing multi-source answers, finding images, and visual verification. There are no obvious missing capabilities or dead ends within the stated purpose.
Maintenance
ActivityMaintained
ResponsivenessNo issues