tutu-mcp-proxy

A compacting/grounding MCP proxy in front of mcp.tutu.ru, built for the Tutu hackathon (track 2 — "tool optimization"). Same 16 tools, same behavior, plus:

• Trimmed always-loaded catalog. tools/list on the real server is ~108 KB even before the first search (uv run python tutu.py measure, reproducible from fixtures/). The three heaviest tools (search_rail, get_rail_seatmap, search_hotels) get a short top-level description; the trimmed prose is not lost but moves into the call result of the paired get_<domain>_instructions (tutu_mcp/proxy/compact_tools.py) — it is paid for only by the session that actually reads it. inputSchema is untouched everywhere.

• check_groundedness. Deterministically cross-checks a draft answer against the tool_results it relies on — extracts prices/times/train-route numbers/links from the text and verifies their actual presence in the JSON, with no LLM judge (tutu_mcp/groundedness.py).

• Explanation for empty results. The most common failure: the agent reads an empty filtered search as "this train does not run," even though the tool only returned what is on sale. Tutu's own counters meta.post_filter_dropped_* say which filter emptied the list — the proxy expands them into a sentence and attaches it to the result (_empty_result_note), so the agent states the fact instead of guessing about a schedule it was never given (tutu_mcp/proxy/empty_results.py).

• Premise gate + assess_request. check_groundedness checks the OUTPUT of a turn; this one checks the INPUT: a value that narrows the search must come from the user or from a past tool_result. A filter invented by the agent (the classic: silently assume an event's end time and filter return routes by it) gets clarification_required instead of data (tutu_mcp/premises.py).

• Mock mode. Responds with recorded fixtures instead of the live server — you can run it as often as you like without touching the hackathon's shared rate limit.

Trace viewer

Each eval run turns into a single self-contained HTML file: double-click, no server and no network. The published showcase is https://trum-ok.github.io/tutu-mcp-hackathon/trace-viewer.html.

make viewer        # из последнего настоящего прогона эвалов
make viewer-demo   # из рукописных демо-трейсов — без модели и без ключа

In the UI: overview mode — the entire scenario × variant matrix in one table; failures only narrows the list to what failed; side by side places one scenario from both variants next to each other, highlighting the checks that diverged. Clicking any highlighted value in the answer opens a drawer with the exact server response fragment it came from — or with a direct statement that it is absent from all of them. Synthetic runs (demo:/scripted:) are marked with an amber "NOT A MEASUREMENT" badge — a hand-written demo cannot be confused with a measurement.

How it is built — docs/deploy.md.

Related MCP server: Yourttoo MCP Server

What came out of it

Four runs of 22 scenarios on gpt-5.6-luna with --effort low, August 19, 2026. The spread between runs comes from the model itself: the backend is the same, the scenario set is the same too.

The fabricated-claims row should be read before the percentage: 4 fabrications out of 189 checkable claims and 1 out of 184 is 97.9% vs. 99.5% — the gap looks like noise. In absolute terms that is four times fewer wrong facts reaching the user, and it is exactly those facts the user gets, not a percentage. Percentages are computed over checkable claims: a threshold the user stated themselves ("cheaper than 3000 ₽") does not obligate the payload to confirm it and does not enter the denominator.

Surface savings — 39%, and it does not depend on the run: it is a static cost the agent pays on every session before the first search. Everything else is model behavior, hence the range.

The last row matters as much as the first: the gate fired 8–12 times per run and yet never asked a question on a scenario where there was nothing to ask about (the no_overask negative control plus the did_not_over_ask check). A mechanism that clarifies everything would score perfect premise metrics and ruin the product.

The variants diverge on five scenarios, and the proxy wins all five: an empty filtered result is not read as "the train does not run," a typo in the day of the week is caught before the search, a silently substituted guest count is stopped by the gate, and adjacent seats are searched with the right tool. The only persistent proxy failure is multitransport_basic: the agent prints a price difference (2,275.07 − 1,700 = 575), both halves of which are confirmed, while the number itself is absent from the payload. A deterministic check cannot tell such arithmetic from incorrect one (multiplying a hotel price by the number of nights — a separate scenario, and there it is an error); that is the boundary of the method, not a proxy defect.

Two things for report honesty: fixture misses are counted separately from tool errors — a recording gap should not read as a Tutu failure; and token numbers are marked with ~ when they are estimates — OpenAI has no token-counting endpoint, the exact figure comes from one real probe request (usage.prompt_tokens), and --estimate-tokens substitutes an offline tiktoken estimate instead.

Model and reasoning effort — per run (--model/OPENAI_MODEL, --effort/OPENAI_EFFORT); without both, the reasoning field is not sent at all, and the model applies its own default — that is not the same as an explicit --effort none. The runner hits /v1/responses by default — Chat Completions does not take function tools together with reasoning on current reasoning models; --api chat is for OpenAI-compatible gateways without /v1/responses. Fixture matching ignores default values from inputSchema (the model writes out page: 1, sort: "price_asc", and so on where a person recording a fixture writes nothing) — otherwise nearly every call in a model run would miss the recording.

How a run works, what each metric counts, and why the harness self-check lives in CI — docs/evals.md.

Quick start

You need uv and Python ≥ 3.13 (uv sync will install it itself).

git clone https://github.com/Trum-ok/tutu-mcp-hackathon
cd tutu-mcp-hackathon
uv sync
uv run python tutu.py serve            # mock-режим (по умолчанию) — http://127.0.0.1:8800/mcp

TUTU_PROXY_MODE=live uv run python tutu.py serve   # проксирует настоящий mcp.tutu.ru

Any MCP client — at http://127.0.0.1:8800/mcp (Streamable HTTP, no auth, same as upstream). Below, <URL> is this address or the address of a deployed proxy (see docs/deploy.md).

claude mcp add --transport http tutu <URL>          # Claude Code

// Cursor · ~/.cursor/mcp.json
{ "mcpServers": { "tutu": { "url": "<URL>" } } }

// Claude Desktop · claude_desktop_config.json — через mcp-remote, он не умеет HTTP напрямую
{ "mcpServers": { "tutu": { "command": "npx", "args": ["-y", "mcp-remote", "<URL>"] } } }

What should happen. In the log — two lines: mode and listening address. The client after connecting shows 18 tools: 16 native Tutu ones plus assess_request and check_groundedness. If there are 16 — the client connected to Tutu itself, not to the proxy.

How much was cut

tools/list: 110,164 → 79,411 bytes (−27.9%), and including each side's initialize instructions — −33.1% (the proxy serves its instruction block at 1.9 KB instead of Tutu's 11.2 KB). Both figures are already after adding the two own tools (assess_request 1,313 bytes, check_groundedness 1,100).

A breakdown by catalog layer, the named price of compression, and the boundary we deliberately did not cross — docs/compaction.md.

Documentation

A user-facing walkthrough is a separate page: make docs builds site/index.html, or open the already published one: https://trum-ok.github.io/tutu-mcp-hackathon/.

rezo team

• Artamonov Arkady (@OpSonata)

License

MIT