mcp-tools
Use this MCP server over stdio to safely compute arithmetic, search a bundled corpus, check answer grounding, inspect live model drift, and compare eval runs.
calc: evaluate arithmetic expressions safely via an AST allow-list; rejects code/imports/calls and refuses overly expensive computations.search: run read-only BM25 keyword search over a bundled or configured corpus, returning top matches with scores and optionalk.grade_answer: lexically check a draft answer against source texts, flagging unsupported sentences and invented figures, with an optional coverage threshold.model_drift: look up current public eval metrics for tracked LLMs and changes since the previous run; omitmodelto list all.compare_runs: check whether a project’s latest stored eval run regressed per case against the previous run.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@mcp-tools Calculate 2 + 3 * 4"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
mcp-tools
A Model Context Protocol server, implemented from the spec — no MCP SDK, no dependencies.
MCP is how a language-model client (Claude Desktop, an agent) discovers and calls tools a server exposes. It's JSON-RPC 2.0; a local server speaks it over stdio. This repo implements that protocol directly — the whole surface a tool server needs is initialize → notifications/initialized → tools/list → tools/call — so the protocol is legible instead of hidden behind a library.
It exposes five tools: three fully local and deterministic, two read-only lookups against public endpoints (no keys, no writes). Each one refuses what it cannot do safely instead of attempting it, and refusal covers two separate things: an input that isn't arithmetic at all, and an input that is arithmetic but whose answer is too expensive to compute.
Tool | What it does | Why it's safe |
| Evaluate an arithmetic expression | Parses to an AST and allow-lists arithmetic nodes only — no |
| BM25 keyword search over a bundled corpus | Read-only, no network. The corpus is read once at startup; no tool argument can reach the filesystem. The ranking is Okapi BM25 — the same length-normalised, saturation-aware scoring that matches the published SciFact baseline in rag-eval-lab, reimplemented here so this server has zero dependencies. |
| Is a live model still scoring what it used to? | Read-only GET of the public model-drift board ( |
| Did a project's latest eval run regress against the one before it? | Read-only GET of eval-history's per-case comparison — so a better average can't hide the case that broke. |
| Check a draft answer against its sources and name the sentences they don't support | No LLM judge. A model grading hallucination is itself a model output — you can't tell a real unsupported claim from the judge having an off day, and you can't reproduce last week's verdict. This is lexical: a figure that appears nowhere in the sources fails the sentence outright (invented statistics are the strongest tell), and low content-word coverage flags claims the sources never make. |
A real MCP session over stdio — no client, no key, no network. calc is handed
__import__("os").system("rm -rf ~") and answers isError with the AST element it refused; the
server stays up, and the next call flags the one sentence the source does not support.
Run it yourself: ./demo/session.sh.
Play it as a terminal session — the text is selectable.
Use it with Claude Desktop
Add this to claude_desktop_config.json (Settings → Developer → Edit Config):
{
"mcpServers": {
"mcp-tools": { "command": "python", "args": ["-m", "mcptools"] }
}
}Restart Claude Desktop and ask it to "search your notes for how rate limiting allows bursts", "use calc to work out 17 * 23 + 4", or — the useful one — paste some source material and ask it to draft an answer and then grade its own answer against those sources. It discovers the tools and calls them.
faithfulness 50% — 1 of 2 claim(s) not supported by the sources
Claims your sources do not support:
• It was adopted by 80% of search engines in 2011.
↳ figure(s) not in sources: 2011, 80
Cut these, or cite a source that backs them.That last tool is the point of the whole thing: it gives an agent a way to check its own work before it answers, without trusting another model's opinion about it. Point search at your own notes with "env": {"MCPTOOLS_CORPUS": "/path/to/notes.json"} (a { "id": "text", ... } file).
Related MCP server: math-anchor
Run it directly
pip install -e .
python -m mcptools # serves on stdio; type/paste JSON-RPC, one message per line# the handshake, by hand:
{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{}}}
{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"calc","arguments":{"expression":"2 + 3 * 4"}}}
# → {"jsonrpc":"2.0","id":2,"result":{"content":[{"type":"text","text":"14"}],"isError":false}}The part worth stealing: it's testable without a client
An MCP server you can only exercise with Claude Desktop open isn't really testable. Because the protocol is plain JSON-RPC, the dispatch is a pure function of a message — so the suite drives the real handshake directly and launches the server in a subprocess and speaks MCP to it over stdio, asserting that three requests get three replies and the notification gets none. The guardrail is tested through the protocol too: code thrown at calc comes back as an MCP tool-error (isError: true), so the model sees the failure and the server stays up.
pip install -e ".[dev]" && pytest -q # 60 tests, stdlib onlyThe two live tools are tested against fixtures, never the network: the fetcher is resolved at call time so a test can substitute it, and the suite passes with sockets blocked. What is tested for real is failure — a network problem comes back as an MCP tool error the model can read and route around, not an exception that takes the server down for every other tool.
Hand-written fixtures could not see the URL, which is how model_drift returned
HTTP 404 for seven weeks with this suite green: model-drift renamed
dashboard/metrics.json to dashboard/drift_board.json on 2026-08-30. The drift
tests now also parse a trimmed recording of the real board
(tests/fixtures/, re-recorded by
scripts/record_drift_board.py, which refuses
anything but HTTP 200), and hold the URL constant to the one it was recorded from.
One test GETs the real board; it is skipped unless MCPTOOLS_LIVE=1, and CI runs
it on every push and on a weekly schedule, so the next upstream rename fails a run
even when nobody pushes here.
Operating it: logs, metrics, and an optional trail
A server you deploy needs more than correct replies — and all three of these are stdlib, none of them touch stdout (that's the JSON-RPC channel; a stray write corrupts the protocol):
Structured logs on stderr. Every
tools/callemits one JSON line — tool,duration_ms,isError, and the size of each argument — with aservingand astoppingline bracketing the session, the latter carrying a per-tool usage summary. Seeobs.py.In-process metrics. Calls and errors per tool, as a snapshot and on shutdown.
An optional result trail. Point
MCPTOOLS_DBat a file and the results ofgrade_answerandmodel_driftare persisted to SQLite — one row per call, behind a hand-rolled migration keyed onPRAGMA user_version(the zero-dependency form of Alembic). Unset, nothing is written and the server behaves exactly as before. Seestore.py.
Run it in Docker
docker build -t mcp-tools .
docker run -i --rm mcp-tools # MCP over stdio; -i keeps stdin open
# persist the trail — mount a dir and point MCPTOOLS_DB at it:
docker run -i --rm -v "$PWD/data:/data" -e MCPTOOLS_DB=/data/history.db mcp-toolsCI builds this image and completes a real MCP handshake through it, so "it runs in a container" is checked, not claimed.
Design notes
Notifications get no reply. A JSON-RPC message with no
idis a notification;notifications/initializedis handled by producing nothing, per the spec.Two error channels, on purpose. An unknown method or a missing argument is a JSON-RPC protocol error (
-32601/-32602); a tool that fails returns a result withisError: true. The model should adapt to a failed tool call, not have the connection torn down under it.Why from scratch. The official SDK is excellent and the right choice for production. Implementing the protocol directly here is the point of the repo: ~150 lines makes the whole lifecycle visible, and it keeps the dependency count at zero.
MIT · by Erik Hill
Available Tools
5 toolscalcA
Evaluate an arithmetic expression safely (no code execution; names, calls and imports are rejected).
Inputs: expression (required).
compare_runsA
Ask whether a project's most recent stored eval run regressed against the one before it — per-case, so a better average can't hide a case that broke.
Inputs: suite (required).
grade_answerA
Check a draft answer against its sources and report which sentences the sources do NOT support — fabricated figures and claims the sources never make. Deterministic and lexical, not a model judgement. Call this on your own answer before giving it to the user when the answer is supposed to be grounded in provided material.
Inputs: answer (required), sources (required), threshold.
model_driftA
Look up how a live LLM is currently scoring on a public, frozen eval suite (accuracy, latency, answer length, reliability, refusal rate) and whether those moved since the previous weekly run. Use it to check whether a model you're about to rely on has quietly changed. Omit model to list every tracked model.
Inputs: model.
searchA
BM25 keyword search over a small bundled document corpus; returns the top matches with their scores.
Inputs: query (required), k.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 5 tools
Each tool has a completely distinct purpose: arithmetic evaluation, keyword search, answer grading, model monitoring, and eval comparison. No overlap in functionality.
Names are mostly descriptive and use lowercase with underscores, but there is slight inconsistency: calc and search are single words, while others follow a verb_noun or noun_noun pattern. Still very readable.
5 tools is a well-scoped set for a utility server. Each tool serves a clear, non-redundant purpose, neither too few nor too many.
The tool set covers the stated utilities (math, search, answer checking, model drift, run comparison) comprehensively for its scope. Minor gaps like source management could exist but aren't critical.
Maintenance
Related MCP Connectors
Find, vet, and run MCP tools through a secure audited gateway with prompt-injection risk scoring
Safety tools for AI agents: safe page reader, prompt-injection, secrets, PII and MCP server scans
Public web tools for agents: product extraction, claim checks, webpage QA and ranked audits.
Your org's AI agents, tasks, runs, search, and brain files as MCP tools and resources.
Related MCP Servers
- FlicenseNot gradedqualityDmaintenanceProvides 18 tools including calculator operations (add, subtract, multiply, divide, power, sqrt, log, trigonometric functions) and secure sandboxed file operations (read, write, append, delete, list) for LangGraph agents and MCP clients. Features async support, YAML configuration, comprehensive logging, and path traversal protection.-
- AlicenseAqualityBmaintenanceProvides a safe scientific runtime for agents with typed math operations including calculus, algebra, statistics, unit conversion, and more via MCP tools.4Apache 2.0
- AlicenseNot gradedqualityDmaintenanceProvides MCP servers (stdio and HTTP) with tools for calculation, text statistics, and unit conversion, plus an agent host that uses Gemini to orchestrate tool calls across them.ISC
- AlicenseAqualityCmaintenanceProvides a document store, web search, and safe calculator as MCP tools, enabling both offline and LLM-driven clients to discover and call them over the Model Context Protocol.6MIT