mcp-tools

A Model Context Protocol server, implemented from the spec — no MCP SDK, no dependencies.

python MCP tests

MCP is how a language-model client (Claude Desktop, an agent) discovers and calls tools a server exposes. It's JSON-RPC 2.0; a local server speaks it over stdio. This repo implements that protocol directly — the whole surface a tool server needs is initialize → notifications/initialized → tools/list → tools/call — so the protocol is legible instead of hidden behind a library.

It exposes five tools, all safe by construction — three fully local and deterministic, two read-only lookups against public endpoints (no keys, no writes):

Tool	What it does	Why it's safe
`calc`	Evaluate an arithmetic expression	Parses to an AST and allow-lists arithmetic nodes only — no `eval`, so `__import__('os')` is rejected, not executed. The OWASP LLM06 (Excessive Agency) mitigation: a tool that can do arithmetic and nothing else.
`search`	BM25 keyword search over a bundled corpus	Read-only, no network. The corpus is read once at startup; no tool argument can reach the filesystem. The ranking is Okapi BM25 — the same length-normalised, saturation-aware scoring that matches the published SciFact baseline in rag-eval-lab, reimplemented here so this server has zero dependencies.
`model_drift`	Is a live model still scoring what it used to?	Read-only GET of the public model-drift board — accuracy, latency, answer length, reliability and refusal rate for 16 models, plus what moved since last week's run. No key, no write.
`compare_runs`	Did a project's latest eval run regress against the one before it?	Read-only GET of eval-history's per-case comparison — so a better average can't hide the case that broke.
`grade_answer`	Check a draft answer against its sources and name the sentences they don't support	No LLM judge. A model grading hallucination is itself a model output — you can't tell a real unsupported claim from the judge having an off day, and you can't reproduce last week's verdict. This is lexical: a figure that appears nowhere in the sources fails the sentence outright (invented statistics are the strongest tell), and low content-word coverage flags claims the sources never make.

A real MCP session over stdio — no client, no key, no network. calc is handed __import__("os").system("rm -rf ~") and answers isError with the AST element it refused; the server stays up, and the next call flags the one sentence the source does not support. Run it yourself: ./demo/session.sh. Play it as a terminal session — the text is selectable.

Use it with Claude Desktop

Add this to claude_desktop_config.json (Settings → Developer → Edit Config):

{
  "mcpServers": {
    "mcp-tools": { "command": "python", "args": ["-m", "mcptools"] }
  }
}

Restart Claude Desktop and ask it to "search your notes for how rate limiting allows bursts", "use calc to work out 17 * 23 + 4", or — the useful one — paste some source material and ask it to draft an answer and then grade its own answer against those sources. It discovers the tools and calls them.

faithfulness 50% — 1 of 2 claim(s) not supported by the sources

Claims your sources do not support:
  • It was adopted by 80% of search engines in 2011.
    ↳ figure(s) not in sources: 2011, 80

Cut these, or cite a source that backs them.

That last tool is the point of the whole thing: it gives an agent a way to check its own work before it answers, without trusting another model's opinion about it. Point search at your own notes with "env": {"MCPTOOLS_CORPUS": "/path/to/notes.json"} (a { "id": "text", ... } file).

Run it directly

pip install -e .
python -m mcptools        # serves on stdio; type/paste JSON-RPC, one message per line

# the handshake, by hand:
{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{}}}
{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"calc","arguments":{"expression":"2 + 3 * 4"}}}
# → {"jsonrpc":"2.0","id":2,"result":{"content":[{"type":"text","text":"14"}],"isError":false}}

The part worth stealing: it's testable without a client

An MCP server you can only exercise with Claude Desktop open isn't really testable. Because the protocol is plain JSON-RPC, the dispatch is a pure function of a message — so the suite drives the real handshake directly and launches the server in a subprocess and speaks MCP to it over stdio, asserting that three requests get three replies and the notification gets none. The guardrail is tested through the protocol too: code thrown at calc comes back as an MCP tool-error (isError: true), so the model sees the failure and the server stays up.

pip install -e ".[dev]" && pytest -q     # 39 tests, stdlib only

The two live tools are tested against fixtures, never the network: the fetcher is resolved at call time so a test can substitute it, and the suite passes with sockets blocked. What is tested for real is failure — a network problem comes back as an MCP tool error the model can read and route around, not an exception that takes the server down for every other tool.

Operating it: logs, metrics, and an optional trail

A server you deploy needs more than correct replies — and all three of these are stdlib, none of them touch stdout (that's the JSON-RPC channel; a stray write corrupts the protocol):

Structured logs on stderr. Every tools/call emits one JSON line — tool, duration_ms, isError, and the size of each argument — with a serving and a stopping line bracketing the session, the latter carrying a per-tool usage summary. See obs.py.
In-process metrics. Calls and errors per tool, as a snapshot and on shutdown.
An optional result trail. Point MCPTOOLS_DB at a file and the results of grade_answer and model_drift are persisted to SQLite — one row per call, behind a hand-rolled migration keyed on PRAGMA user_version (the zero-dependency form of Alembic). Unset, nothing is written and the server behaves exactly as before. See store.py.

Run it in Docker

docker build -t mcp-tools .
docker run -i --rm mcp-tools           # MCP over stdio; -i keeps stdin open
# persist the trail — mount a dir and point MCPTOOLS_DB at it:
docker run -i --rm -v "$PWD/data:/data" -e MCPTOOLS_DB=/data/history.db mcp-tools

CI builds this image and completes a real MCP handshake through it, so "it runs in a container" is checked, not claimed.

Design notes

Notifications get no reply. A JSON-RPC message with no id is a notification; notifications/initialized is handled by producing nothing, per the spec.
Two error channels, on purpose. An unknown method or a missing argument is a JSON-RPC protocol error (-32601 / -32602); a tool that fails returns a result with isError: true. The model should adapt to a failed tool call, not have the connection torn down under it.
Why from scratch. The official SDK is excellent and the right choice for production. Implementing the protocol directly here is the point of the repo: ~150 lines makes the whole lifecycle visible, and it keeps the dependency count at zero.

MIT · by Erik Hill