# agent-systems — MCP server from the wire protocol up

An [MCP](https://modelcontextprotocol.io) server implemented **by hand over raw
JSON-RPC 2.0 stdio** — the `initialize` handshake, `tools/list`, `tools/call`,
error codes — with message-level conformance tests asserting on actual wire
bytes.

It exposes a **real system**: the hybrid retrieval service and evaluation
harness from [retrieval-lab](https://github.com/harsha135135/retrieval-lab).
An MCP server is only interesting if the thing on the other side of the protocol
is worth reaching.

**Why it matters:** MCP is the de-facto standard for connecting tools to models,
and "can you write an MCP server" is now a real interview question. Writing it
from the spec rather than the SDK is what makes the follow-up — *"walk me
through the handshake"* — answerable.

---

## Results

| Check | Result |
|---|---|
| Conformance tests (wire-level) | **35 passed** |
| End-to-end stdio subprocess tests | **5 passed** |
| Total | **40 passed in 0.83s** |
| Protocol versions supported | `2025-06-18`, `2025-03-26`, `2024-11-05` |
| Tools exposed | 4 |

A live session, driven by piping JSON into the process:

```
initialize -> {'name': 'retrieval-lab-mcp', 'version': '0.1.0'} 2025-06-18
tools/list -> ['search_documents', 'corpus_info', 'run_evaluation', 'search_my_repos']
```

### The three spec rules most implementations get wrong

Each has a test that fails if the rule is broken:

| Rule | Why it bites | Test |
|---|---|---|
| **Never reply to a notification** | `notifications/initialized` has no `id`. Replying makes a strict client error on an unsolicited response — and it surfaces one message *later*, so it looks like a bug in whatever came next | `test_notifications_are_never_answered` |
| **The field is `inputSchema`, not `input_schema`** | Snake_case makes every client see a tool with no parameters. Fails **silently** | `test_tools_list_uses_camelCase_inputSchema` |
| **A failing tool is a *successful* RPC** | `isError: true` inside the result means the tool ran and reported a problem the model can react to. A JSON-RPC `error` means the protocol failed. Conflating them denies the model its self-correction | `test_a_failing_tool_is_a_successful_rpc_with_isError` |

### The stdio trap

**stdout *is* the transport.** A stray `print()` — yours, or a library's — lands
mid-stream and corrupts the session with a parse error pointing at the protocol
rather than at the print.

Two defences: every diagnostic goes to stderr, and `sys.stdout` is redirected
while tool handlers run so a third-party library that prints cannot break the
channel. `test_a_tool_printing_to_stdout_does_not_corrupt_the_stream` registers
a tool that deliberately prints and asserts the wire bytes stay clean.

---

## Quick start

```bash
git clone https://github.com/harsha135135/retrieval-lab.git
git clone https://github.com/harsha135135/agent-systems.git
cd agent-systems
uv sync --extra dev

uv run pytest -q                 # 40 passed
uv run python -m agent_systems --list-tools
```

Drive a session by hand — no client needed:

```bash
printf '%s\n%s\n%s\n' \
 '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","clientInfo":{"name":"demo"}}}' \
 '{"jsonrpc":"2.0","method":"notifications/initialized"}' \
 '{"jsonrpc":"2.0","id":2,"method":"tools/list"}' \
 | uv run python -m agent_systems
```

### Register it in Claude Code

`.mcp.json` ships in the repo:

```bash
claude mcp list        # retrieval-lab shows as connected
```

Then ask Claude Code to *"search the arxiv corpus for papers on distributed
training"* or *"run the retrieval evaluation with filters off"*.

---

## The tools

| Tool | What it does |
|---|---|
| `search_documents` | Hybrid retrieval — hard metadata filters plus semantic vectors — over 340 real arXiv papers or a synthetic listings corpus |
| `corpus_info` | Size, filterable fields, and **whether the data is real or synthetic** (surfaced deliberately — a model reasoning over it should know) |
| `run_evaluation` | Scores the retriever live: recall@k with its ceiling, precision, MRR, latency. Toggle `apply_filters` to run the ablation |
| `search_my_repos` | Read-only grep across the local repos |

`run_evaluation` is the one worth pointing at: the model can **measure the
retrieval system it is querying**, in the same session.

### Every file, and why it exists

| Path | Responsibility |
|---|---|
| `protocol.py` | Framing, JSON-RPC 2.0 envelope, error codes, version negotiation |
| `server.py` | Method dispatch, lifecycle rules, the stdio loop, stdout guarding |
| `tools.py` | The four tools; imports retrieval-lab lazily so its absence is a readable tool error, not a startup crash |
| `docs/handshake.md` | The handshake, message by message, with the traps |
| `tests/unit/test_conformance.py` | 35 wire-level tests — assert on bytes, not internal state |
| `tests/integration/test_stdio_session.py` | 5 tests against a real subprocess: pipes, framing, flushing, recovery |

---

## What this does not do

- **No `resources` or `prompts`.** Only `tools` is implemented, and capabilities
  advertise only that — declaring more would make clients call methods that 404.
- **No SDK rebuild yet.** The spec calls for rebuilding on the official SDK and
  diffing the two; the raw implementation and its conformance suite came first
  because that is where the understanding is.
- **stdio transport only.** No HTTP/SSE.
- **No auth.** Local stdio server; the trust boundary is the process.

---

## Troubleshooting

| Problem | Fix |
|---|---|
| `retrieval-lab is not importable` | Clone it beside this repo, then `uv sync`. The protocol layer works without it |
| Client hangs after connecting | Usually a reply sent to a notification — check `id` handling |
| Tools show with no parameters | `input_schema` instead of `inputSchema` |
| Garbled JSON / parse errors | Something printed to stdout. Diagnostics belong on stderr |
| `claude mcp list` shows disconnected | Run `uv run python -m agent_systems` directly; startup errors go to stderr |