ABC Vision MCP

An MCP server that exposes a production badminton video-analysis engine as agent tools — and records, at every branch point, how a measured visual result changed what the system did next.

Built on OpenCV 5.0.0. Speaks MCP over Streamable HTTP.

It is running. Go and look.

The conversation on the left is staged. Every call on the right is real — this browser talking to that server over Streamable HTTP, each one openable to its raw JSON-RPC. The page is served from the same origin as /mcp, which is why there is no proxy in between and nothing that could be a recording.

The page runs the loop twice. First the ordinary case, where the player is on court for all three rallies: identity locks on the first attempt, nothing is excluded, and analysis proceeds. Then the harder one, where the player sits out the third and no assignment of detections to people can cover a rally they were not in — the loop tries every anchor, reaches 2 of 3, and asks instead of publishing.

Only the second run branches, and the branch is the point. The first run is there because the page used to show only the second, which reads as a system that never answers.

Verify the OpenCV version yourself rather than taking this README's word for it — call protocol_info on that endpoint and it reports the version of the process that answered you.

Opening /mcp in a browser returns a JSON-RPC error about a missing session ID. That is correct: it is a protocol endpoint, not a web page. The page is at /.

Related MCP server: jev-code

Why this exists

ABC (AI Badminton Coach) is a shipping product: an App Store listing, seven locales, five jurisdictions' privacy compliance, 527 commits, and 1,117 production jobs over the twelve days to 2026-09-17. Its defining behaviour is that it refuses to answer when it is not sure — of the 705 jobs that produced a report, it declined to assign a skill level in 42.7% of them, and recorded a human-readable reason each time. The largest single reason, too little of the player tracked, accounts for 30.2% of those same 705.

That refusal is not a UI message bolted on at the end. It is the outcome of a chain of gates, each fed by a measurement from the vision pipeline: was a court found, did the camera drift, was the player held long enough, were the serves segmented cleanly. When a gate fails, the system does something different — retries with a different anchor, asks the player to identify themselves, or publishes an analysis with the level withheld.

This server makes that chain callable by an agent, and writes down the reasoning as it goes.

What is in here

Decisions, not logs

A log of "scan ran, then lock ran" proves nothing — a fixed script produces the same log. So the trace separates two kinds of record:

• ToolCall — what ran, with inputs and outputs. Supporting detail.

• Decision — a branch point where a measured value selected the next action, recorded together with the branches that were not taken and the rule that maps evidence to branch.

summary() reports branching_decisions separately from decisions, because a decision with no alternatives did not branch on anything and should not be presented as if it had.

Example of one real branch, taken from the engine's identity lock (core/analysis.py, MAX_ANCHOR_ATTEMPTS = 8): when geometric validation of a candidate anchor fails, the rejected anchor key is added to an avoid set and the same tool is called again with different parameters. The visual result changed the next tool call. That loop already exists in production; this server exposes and records it rather than inventing one for a submission.

Privacy

This repository is built to be handed to competition judges, so it is structured to make leaking hard rather than to rely on care:

• .gitignore is an allowlist — everything is ignored, and tracked paths are named explicitly. A sibling repository used a blocklist, missed several directories, and ended up tracking customer data it should never have held. This one starts from the opposite default.

• The trace redacts by key name, so a field added later called subject_name is redacted the day it appears. Paths and identifiers become truncated SHA-256 handles: stable enough to show two calls hit the same video, useless for recovering which video.

• Only de-identified aggregates ship in data/. No video, no frames, no subject records.

• Redaction is covered by tests, not by convention.

Requirements

• Python 3.12

• mcp 2.2.0 — negotiated protocol version 2025-11-25

• OpenCV 5.0.0 (opencv-python-headless==5.0.0.93)

The SDK also exposes LATEST_PROTOCOL_VERSION = 2026-07-28. That is the transport revision, not the protocol version, and it is always the higher of the two — which makes it the tempting one to quote. An earlier version of this README quoted it. See docs/CORRECTIONS.md.

Running

python -m venv .venv
.venv/Scripts/python -m pip install -r requirements.txt

# stdio, for a local client
.venv/Scripts/python -m abc_vision_mcp.server

# Streamable HTTP on http://127.0.0.1:8931/mcp, with the demo page at /
.venv/Scripts/python -m abc_vision_mcp.server --http

Serving on a public interface? Name the hosts you answer to. The SDK enables DNS rebinding protection only for a loopback bind, so a container that binds 0.0.0.0 has none unless it asks:

.venv/Scripts/python -m abc_vision_mcp.server --http --host 0.0.0.0     --allowed-host your.public.name --allowed-origin https://your.public.name

Without it the server starts anyway and prints a warning saying what is not protecting it. protocol_info reports the resulting state, so a deployment can be checked rather than believed.

Tests:

.venv/Scripts/python -m pytest tests/ -q
# 115 passed, 12 skipped   without an engine checkout
# 127 passed              with ABC_ENGINE_PATH set

The twelve skips are the tests that need the analysis engine, which is not in this repository and never will be — see Responsible operation above. Point ABC_ENGINE_PATH at a checkout containing core/identity/lock.py to run them. Nothing else in the suite depends on it, so a reviewer without the engine still exercises the MCP surface, the transport security, the redaction and the demo fixtures.

Reproduce the decision trace shown on the demo page — no video, no customer data, production tools:

.venv/Scripts/python scripts/demo_agentic_loop.py --out traces
# 6 tool calls, 4 decisions, 4 branching, 1 human approval request

Check a running server against the spec, with the checker we published for this (MIT, separate repository):

pip install git+https://github.com/ekyo0224/mcp-conformance
mcp-conformance https://d2xks2we90iwvv.cloudfront.net/mcp   # 12 checks, 0 failures

Status

Deployed and public, running on AWS for the judging period of two 2026 competitions. 16 tools, 127 tests, OpenCV 5.0.0, negotiated protocol 2025-11-25.

The analysis engine itself is not in this repository and never will be. A named allowlist of eleven files is staged at build time inside the owner's own AWS account, and the build refuses to run if anything resembling customer data reaches it.

Nemotron coach notes (NVIDIA × Nebius)

explain_report (and the "Coach notes" block on the demo page, whose model version is behind a "Write with Nemotron" button) turns a report's summary into one to three plain coaching suggestions and, when a level was withheld, an honest sentence on why. The wording comes from an NVIDIA Nemotron model served by Nebius Token Factory; the facts do not.

export NEBIUS_API_KEY=...          # never logged, never returned
export NEBIUS_MODEL=...            # optional; default nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B
export NEBIUS_BASE_URL=...         # optional; default https://api.tokenfactory.nebius.com/v1

• What is sent: only status, level (if the report had one), rallies_total, rallies_covered and decline_reason (only if a tool supplied one). No video, frames, names or paths; other fields are dropped before the call. What the summary fields themselves reveal is the rally counts and the published/declined status.

• Fallback: with no key, a timeout (8 s), an HTTP error, or an answer that states a level the report lacks, uses a number the report lacks, or breaks the length limits, the result is fixed rule-based text from the same fields. Every result carries source: "nemotron" or source: "rules".

• Not claimed: the model never decides or produces a level. The model path has been tested here against mocked responses; nothing in the test suite calls Nebius.

Ask the coach: a Nemotron agent that calls our tools

ask_coach (MCP tool), POST /demo/ask ({"question": "..."}, 300 characters max) and the "Ask the coach (Nemotron agent)" box on the demo page. A player or coach asks in plain language; an NVIDIA Nemotron model on Nebius Token Factory decides which tools to call (OpenAI-style tool calling), the tools run in-process, and the model answers from their results. The result is {answer, tools_called, source, model, usage}; source is nemotron or rules, and rules carries a reason (no_key, cap, ungrounded, max_rounds, error, budget).

export NEBIUS_API_KEY=...            # never logged or returned
export NEBIUS_AGENT_MODEL=...        # optional; default nvidia/nemotron-3-super-120b-a12b
export NEBIUS_DAILY_CALL_CAP=200     # optional; model calls per UTC day
# other OpenAI-compatible host: AGENT_PROVIDER=openai-compatible with
# AGENT_API_KEY, AGENT_BASE_URL, AGENT_MODEL
.venv/Scripts/python -m abc_vision_mcp.server --http

Tool allowlist (coach_agent.py, read-only, in-process on bundled data, no network, no video): production_outcomes, decline_reasons, human_approval_dropoff, demo_case_report (the demo cases' summaries, the same function behind the page's coach notes). A request for any other tool name is refused and the model is told so. Nothing that uploads, writes or needs a real video is reachable.

Guards, all in code and covered by tests:

• Grounding. Every number in the final answer must appear in the tool results or the question; a skill grade is only allowed if a tool returned one. A failing answer gets one correction and one more try; then the answer is replaced by fixed text built from the tool results (source: "rules"). Spelled-out quantities (one to twenty, "twice", "half") are checked against the same sources.

  What this does not check: wording-level claims. An answer can use only numbers the tools returned and still say something the tools did not (a cause, a rule, a comparison), be vague, or answer a neighbouring question, and the guard will pass it. Tool descriptions are kept to what each result actually returns, and the prompt forbids explaining beyond it, but that is an instruction, not a check.

• Limits. At most 4 rounds of tool calls, 600 output tokens and 20 s per model call, 75 s per question. HTTP errors and timeouts fall back at once and are never retried. No threads, timers or loops beyond the round cap.

• Cost cap. A process-wide daily cap on model calls (default 200), counting ask_coach, /demo/ask, and the model path of explain_report and POST /demo/explain; past it the rules answer is returned with reason cap, with no call made.

• Per client. Web visitors: 5 requests a minute and 30 a day, one in flight at a time. MCP callers share one bucket: 100 a day, 2 in flight. Over the limit is HTTP 429 (a tool error for the MCP tools) and Nebius is not called. Identical questions are served from a 10-minute cache. The client key is the CloudFront-Viewer-Address header (port dropped) when present, else the socket address; X-Forwarded-For is ignored because a caller can set it. The table of client keys drops day-old entries and is capped at 10,000.

• Nothing spends on page load. The demo page's coach notes load as fixed rules text (GET /demo/explain never calls a model). The model versions (POST /demo/explain, /demo/ask) are sent only from a click or submit.

• What the model sees. The question, the allowlisted tools' names and descriptions, and their results: published outcome statistics, decline-reason categories in plain words (for example "the camera moved too much"), the demo cases' rally counts, and the long-upload figures. No video, frames, names, paths or source code. The decline categories do name stages of the analysis in plain language, and the results carry the project's own outcome codes; they carry no thresholds, margins or scoring values.

Model. Default is nvidia/nemotron-3-super-120b-a12b; the Nano model (nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B) also calls tools correctly and is faster and cheaper, so it is a reasonable NEBIUS_AGENT_MODEL. Thinking is always switched off (chat_template_kwargs.enable_thinking=false).

Cost. usage sums the API's token counts over a question and prices them from a table in agent_providers.py (Super $0.30 / $0.90 per 1M prompt / completion tokens, read from the account's /models?verbose=true); a model missing from the table reports usd: null rather than a guess. Measured live (6 questions, Super): about 3,100 prompt and 220 completion tokens, roughly $0.0011, per question.

Reuse. agent_core.py imports nothing from this server: bring a Provider, register your own tools in a ToolRegistry, and pass an AgentConfig (system prompt, fallback, extra checks).

Not claimed. The agent cannot analyse a video and never produces a skill level. The tests mock the network; the live runs were small manual checks.