agent-systems — MCP server from the wire protocol up

An MCP server implemented by hand over raw JSON-RPC 2.0 stdio — the initialize handshake, tools/list, tools/call, error codes — with message-level conformance tests asserting on actual wire bytes.

It exposes a real system: the hybrid retrieval service and evaluation harness from retrieval-lab. An MCP server is only interesting if the thing on the other side of the protocol is worth reaching.

Why it matters: MCP is the de-facto standard for connecting tools to models, and "can you write an MCP server" is now a real interview question. Writing it from the spec rather than the SDK is what makes the follow-up — "walk me through the handshake" — answerable.

Results

Check	Result
Conformance tests (wire-level)	35 passed
End-to-end stdio subprocess tests	5 passed
Total	40 passed in 0.83s
Protocol versions supported	`2025-06-18`, `2025-03-26`, `2024-11-05`
Tools exposed	4

A live session, driven by piping JSON into the process:

initialize -> {'name': 'retrieval-lab-mcp', 'version': '0.1.0'} 2025-06-18
tools/list -> ['search_documents', 'corpus_info', 'run_evaluation', 'search_my_repos']

The three spec rules most implementations get wrong

Each has a test that fails if the rule is broken:

Rule	Why it bites	Test
Never reply to a notification	`notifications/initialized` has no `id`. Replying makes a strict client error on an unsolicited response — and it surfaces one message later, so it looks like a bug in whatever came next	`test_notifications_are_never_answered`
The field is `inputSchema`, not `input_schema`	Snake_case makes every client see a tool with no parameters. Fails silently	`test_tools_list_uses_camelCase_inputSchema`
A failing tool is a successful RPC	`isError: true` inside the result means the tool ran and reported a problem the model can react to. A JSON-RPC `error` means the protocol failed. Conflating them denies the model its self-correction	`test_a_failing_tool_is_a_successful_rpc_with_isError`

The stdio trap

stdout is the transport. A stray print() — yours, or a library's — lands mid-stream and corrupts the session with a parse error pointing at the protocol rather than at the print.

Two defences: every diagnostic goes to stderr, and sys.stdout is redirected while tool handlers run so a third-party library that prints cannot break the channel. test_a_tool_printing_to_stdout_does_not_corrupt_the_stream registers a tool that deliberately prints and asserts the wire bytes stay clean.

Quick start

git clone https://github.com/harsha135135/retrieval-lab.git
git clone https://github.com/harsha135135/agent-systems.git
cd agent-systems
uv sync --extra dev

uv run pytest -q                 # 40 passed
uv run python -m agent_systems --list-tools

Drive a session by hand — no client needed:

printf '%s\n%s\n%s\n' \
 '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","clientInfo":{"name":"demo"}}}' \
 '{"jsonrpc":"2.0","method":"notifications/initialized"}' \
 '{"jsonrpc":"2.0","id":2,"method":"tools/list"}' \
 | uv run python -m agent_systems

Register it in Claude Code

.mcp.json ships in the repo:

claude mcp list        # retrieval-lab shows as connected

Then ask Claude Code to "search the arxiv corpus for papers on distributed training" or "run the retrieval evaluation with filters off".

The tools

Tool	What it does
`search_documents`	Hybrid retrieval — hard metadata filters plus semantic vectors — over 340 real arXiv papers or a synthetic listings corpus
`corpus_info`	Size, filterable fields, and whether the data is real or synthetic (surfaced deliberately — a model reasoning over it should know)
`run_evaluation`	Scores the retriever live: recall@k with its ceiling, precision, MRR, latency. Toggle `apply_filters` to run the ablation
`search_my_repos`	Read-only grep across the local repos

run_evaluation is the one worth pointing at: the model can measure the retrieval system it is querying, in the same session.

Every file, and why it exists

Path	Responsibility
`protocol.py`	Framing, JSON-RPC 2.0 envelope, error codes, version negotiation
`server.py`	Method dispatch, lifecycle rules, the stdio loop, stdout guarding
`tools.py`	The four tools; imports retrieval-lab lazily so its absence is a readable tool error, not a startup crash
`docs/handshake.md`	The handshake, message by message, with the traps
`tests/unit/test_conformance.py`	35 wire-level tests — assert on bytes, not internal state
`tests/integration/test_stdio_session.py`	5 tests against a real subprocess: pipes, framing, flushing, recovery

What this does not do

No resources or prompts. Only tools is implemented, and capabilities advertise only that — declaring more would make clients call methods that 404.
No SDK rebuild yet. The spec calls for rebuilding on the official SDK and diffing the two; the raw implementation and its conformance suite came first because that is where the understanding is.
stdio transport only. No HTTP/SSE.
No auth. Local stdio server; the trust boundary is the process.

Troubleshooting

Problem	Fix
`retrieval-lab is not importable`	Clone it beside this repo, then `uv sync`. The protocol layer works without it
Client hangs after connecting	Usually a reply sent to a notification — check `id` handling
Tools show with no parameters	`input_schema` instead of `inputSchema`
Garbled JSON / parse errors	Something printed to stdout. Diagnostics belong on stderr
`claude mcp list` shows disconnected	Run `uv run python -m agent_systems` directly; startup errors go to stderr

retrieval-lab-mcp