mcp-inference-bridge
by ThuanE3691
README.md
# mcp-inference-bridge
A small [Model Context Protocol](https://modelcontextprotocol.io) server that lets an AI
assistant (Claude Desktop, Claude Code, Cursor, MCP Inspector) call an existing HTTP
image-classification API: classify an image, show a heatmap explaining the prediction,
and read the model's metadata.
The server is a thin wrapper. It does no inference itself: it validates the input, calls
your API with a timeout, and turns the answer (or the failure) into something the
assistant can use.
> Research use only. Not a diagnosis and not for clinical decisions. Every tool result
> carries this disclaimer.
## Tools
| Tool | Input | Returns |
|---|---|---|
| `classify_image` | `image_path` **or** `image_base64` | `label`, `confidence`, `model_version` (structured) |
| `explain_prediction` | `image_path` **or** `image_base64` | the heatmap as MCP image content (PNG, shown inline by clients) plus a one-line summary |
| `get_model_info` | none | model name, version, input size, class names, decision threshold |
All three are read-only and annotated as such.
Failures come back as tool errors written for the assistant, not stack traces:
| Situation | Message the assistant sees |
|---|---|
| Both or neither image input given | `Provide exactly one of image_path or image_base64.` |
| File missing, too large, or not an image | says which, and what is accepted |
| API slow | `The inference API did not answer in time. ...` |
| API down | `Could not reach the inference API (ConnectError). Is it running?` |
| Wrong key | `The inference API rejected the API key (check INFERENCE_API_KEY).` |
| API 4xx | the API's own `detail` text |
| API 5xx | a generic message; the details go to the server log (stderr) only |
## Quick start
Requires Python 3.12+ and [uv](https://docs.astral.sh/uv/).
```bash
git clone https://github.com/ThuanE3691/mcp-inference-bridge.git
cd mcp-inference-bridge
uv sync --extra demo
```
The repository includes a tiny stand-in inference API (scikit-learn digits classifier with
an occlusion heatmap) so everything can be tried without a real model:
```bash
uv run uvicorn demo_api.digits_api:app --port 8765 # terminal 1: the demo API
uv run python -m demo_api.make_samples # writes samples/digit-*.png
```
Then point an MCP client at the bridge (next section) and ask, for example:
*"What does the model predict for samples/digit-3.png, and show me why."*
## Configuration
Everything is read from environment variables. Nothing is hard-coded and no secret is
ever written to tool output.
| Variable | Required | Default | Meaning |
|---|---|---|---|
| `INFERENCE_API_URL` | yes | | Base URL of the inference API |
| `INFERENCE_API_KEY` | no | | Sent as the `X-API-Key` header |
| `INFERENCE_TIMEOUT_SECONDS` | no | `15` | Per-request timeout |
| `MAX_IMAGE_BYTES` | no | `5000000` | Largest accepted image |
The server refuses to start, with a one-line reason, if `INFERENCE_API_URL` is missing.
## Connect a client
### Claude Desktop
Edit `claude_desktop_config.json` (Settings → Developer → Edit Config) and restart
Claude Desktop:
```json
{
"mcpServers": {
"inference": {
"command": "uv",
"args": ["--directory", "/absolute/path/to/mcp-inference-bridge", "run", "mcp-inference-bridge"],
"env": {
"INFERENCE_API_URL": "http://127.0.0.1:8765",
"INFERENCE_API_KEY": "dev-key"
}
}
}
}
```
### Cursor
Put the same `mcpServers` block in `.cursor/mcp.json` (project) or `~/.cursor/mcp.json`
(global).
### Claude Code
```bash
claude mcp add inference \
-e INFERENCE_API_URL=http://127.0.0.1:8765 -e INFERENCE_API_KEY=dev-key \
-- uv --directory /absolute/path/to/mcp-inference-bridge run mcp-inference-bridge
```
### MCP Inspector
```bash
npx @modelcontextprotocol/inspector \
-e INFERENCE_API_URL=http://127.0.0.1:8765 -e INFERENCE_API_KEY=dev-key \
uv --directory /absolute/path/to/mcp-inference-bridge run mcp-inference-bridge
```
Add `--cli --method tools/list` (or `--method tools/call --tool-name classify_image
--tool-arg image_path=/abs/path/digit-3.png`) to drive it from a script.
### Checked with
Against the demo API on Windows 11:
- **MCP Inspector (CLI)**: `tools/list` shows the three tools; `classify_image` returns
structured content; `explain_prediction` returns an `image` content block (PNG).
- **Claude Code**: asked in plain language to classify and explain `samples/digit-3.png`,
it called `get_model_info`, `classify_image` and `explain_prediction` in that order and
described the heatmap it received.
Claude Desktop and Cursor launch the server the same way (stdio), using the config above.
## What the API must look like
The bridge expects three endpoints. Paths are constants at the top of
[`api_client.py`](src/mcp_inference_bridge/api_client.py); change them there if yours differ.
| Endpoint | Request | Response |
|---|---|---|
| `POST /predict` | `{"image_base64": "..."}` | `{"label", "confidence", "model_version"}` |
| `POST /explain` | `{"image_base64": "..."}` | `{"label", "confidence", "heatmap_png_base64"}` |
| `GET /model-info` | | `{"name", "version", "input_size", "classes", "decision_threshold"}` |
## Adding a tool
1. Add a method to `InferenceClient` in `api_client.py` that calls the new endpoint
through `_request` (that is where the timeout and error mapping live).
2. Add a function inside `build_server` in `server.py`, decorated with
`@mcp.tool(title=..., annotations=...)`. Return a pydantic model for structured data, or
`Image(...)` for a picture.
3. Write the docstring for the assistant: what the tool returns, when to use it, and when
to use a sibling tool instead. The docstring is the tool description the model reads.
4. Add a test in `tests/test_mcp_tools.py`. The tests call tools through a real MCP client
session, so they check what an assistant would actually receive.
## Tests
```bash
uv run --extra demo pytest
```
- `test_image_input_validation.py`: file, base64 and data-URL input; rejected inputs.
- `test_mcp_tools.py`: every tool through an in-memory MCP client against a mocked API,
including timeouts, connection errors, a rejected key and 4xx/5xx responses.
- `test_stdio_end_to_end.py`: starts the demo API, launches the bridge as a stdio
subprocess (the way desktop clients do) and runs all three tools for real.
## Notes
- `image_path` reads a file from the machine the server runs on, with the server's
permissions. It accepts image files only (checked with Pillow) up to `MAX_IMAGE_BYTES`.
- Transport is stdio, for local single-user use. Nothing listens on a network port.
- Built on the official Python SDK (`mcp` 2.x, `MCPServer`).
## License
MIT