Skip to main content
Glama
README.md
# mcp-inference-bridge

A small [Model Context Protocol](https://modelcontextprotocol.io) server that lets an AI
assistant (Claude Desktop, Claude Code, Cursor, MCP Inspector) call an existing HTTP
image-classification API: classify an image, show a heatmap explaining the prediction,
and read the model's metadata.

The server is a thin wrapper. It does no inference itself: it validates the input, calls
your API with a timeout, and turns the answer (or the failure) into something the
assistant can use.

> Research use only. Not a diagnosis and not for clinical decisions. Every tool result
> carries this disclaimer.

## Tools

| Tool | Input | Returns |
|---|---|---|
| `classify_image` | `image_path` **or** `image_base64` | `label`, `confidence`, `model_version` (structured) |
| `explain_prediction` | `image_path` **or** `image_base64` | the heatmap as MCP image content (PNG, shown inline by clients) plus a one-line summary |
| `get_model_info` | none | model name, version, input size, class names, decision threshold |

All three are read-only and annotated as such.

Failures come back as tool errors written for the assistant, not stack traces:

| Situation | Message the assistant sees |
|---|---|
| Both or neither image input given | `Provide exactly one of image_path or image_base64.` |
| File missing, too large, or not an image | says which, and what is accepted |
| API slow | `The inference API did not answer in time. ...` |
| API down | `Could not reach the inference API (ConnectError). Is it running?` |
| Wrong key | `The inference API rejected the API key (check INFERENCE_API_KEY).` |
| API 4xx | the API's own `detail` text |
| API 5xx | a generic message; the details go to the server log (stderr) only |

## Quick start

Requires Python 3.12+ and [uv](https://docs.astral.sh/uv/).

```bash
git clone https://github.com/ThuanE3691/mcp-inference-bridge.git
cd mcp-inference-bridge
uv sync --extra demo
```

The repository includes a tiny stand-in inference API (scikit-learn digits classifier with
an occlusion heatmap) so everything can be tried without a real model:

```bash
uv run uvicorn demo_api.digits_api:app --port 8765     # terminal 1: the demo API
uv run python -m demo_api.make_samples                 # writes samples/digit-*.png
```

Then point an MCP client at the bridge (next section) and ask, for example:
*"What does the model predict for samples/digit-3.png, and show me why."*

## Configuration

Everything is read from environment variables. Nothing is hard-coded and no secret is
ever written to tool output.

| Variable | Required | Default | Meaning |
|---|---|---|---|
| `INFERENCE_API_URL` | yes | | Base URL of the inference API |
| `INFERENCE_API_KEY` | no | | Sent as the `X-API-Key` header |
| `INFERENCE_TIMEOUT_SECONDS` | no | `15` | Per-request timeout |
| `MAX_IMAGE_BYTES` | no | `5000000` | Largest accepted image |

The server refuses to start, with a one-line reason, if `INFERENCE_API_URL` is missing.

## Connect a client

### Claude Desktop

Edit `claude_desktop_config.json` (Settings → Developer → Edit Config) and restart
Claude Desktop:

```json
{
  "mcpServers": {
    "inference": {
      "command": "uv",
      "args": ["--directory", "/absolute/path/to/mcp-inference-bridge", "run", "mcp-inference-bridge"],
      "env": {
        "INFERENCE_API_URL": "http://127.0.0.1:8765",
        "INFERENCE_API_KEY": "dev-key"
      }
    }
  }
}
```

### Cursor

Put the same `mcpServers` block in `.cursor/mcp.json` (project) or `~/.cursor/mcp.json`
(global).

### Claude Code

```bash
claude mcp add inference \
  -e INFERENCE_API_URL=http://127.0.0.1:8765 -e INFERENCE_API_KEY=dev-key \
  -- uv --directory /absolute/path/to/mcp-inference-bridge run mcp-inference-bridge
```

### MCP Inspector

```bash
npx @modelcontextprotocol/inspector \
  -e INFERENCE_API_URL=http://127.0.0.1:8765 -e INFERENCE_API_KEY=dev-key \
  uv --directory /absolute/path/to/mcp-inference-bridge run mcp-inference-bridge
```

Add `--cli --method tools/list` (or `--method tools/call --tool-name classify_image
--tool-arg image_path=/abs/path/digit-3.png`) to drive it from a script.

### Checked with

Against the demo API on Windows 11:

- **MCP Inspector (CLI)**: `tools/list` shows the three tools; `classify_image` returns
  structured content; `explain_prediction` returns an `image` content block (PNG).
- **Claude Code**: asked in plain language to classify and explain `samples/digit-3.png`,
  it called `get_model_info`, `classify_image` and `explain_prediction` in that order and
  described the heatmap it received.

Claude Desktop and Cursor launch the server the same way (stdio), using the config above.

## What the API must look like

The bridge expects three endpoints. Paths are constants at the top of
[`api_client.py`](src/mcp_inference_bridge/api_client.py); change them there if yours differ.

| Endpoint | Request | Response |
|---|---|---|
| `POST /predict` | `{"image_base64": "..."}` | `{"label", "confidence", "model_version"}` |
| `POST /explain` | `{"image_base64": "..."}` | `{"label", "confidence", "heatmap_png_base64"}` |
| `GET /model-info` | | `{"name", "version", "input_size", "classes", "decision_threshold"}` |

## Adding a tool

1. Add a method to `InferenceClient` in `api_client.py` that calls the new endpoint
   through `_request` (that is where the timeout and error mapping live).
2. Add a function inside `build_server` in `server.py`, decorated with
   `@mcp.tool(title=..., annotations=...)`. Return a pydantic model for structured data, or
   `Image(...)` for a picture.
3. Write the docstring for the assistant: what the tool returns, when to use it, and when
   to use a sibling tool instead. The docstring is the tool description the model reads.
4. Add a test in `tests/test_mcp_tools.py`. The tests call tools through a real MCP client
   session, so they check what an assistant would actually receive.

## Tests

```bash
uv run --extra demo pytest
```

- `test_image_input_validation.py`: file, base64 and data-URL input; rejected inputs.
- `test_mcp_tools.py`: every tool through an in-memory MCP client against a mocked API,
  including timeouts, connection errors, a rejected key and 4xx/5xx responses.
- `test_stdio_end_to_end.py`: starts the demo API, launches the bridge as a stdio
  subprocess (the way desktop clients do) and runs all three tools for real.

## Notes

- `image_path` reads a file from the machine the server runs on, with the server's
  permissions. It accepts image files only (checked with Pillow) up to `MAX_IMAGE_BYTES`.
- Transport is stdio, for local single-user use. Nothing listens on a network port.
- Built on the official Python SDK (`mcp` 2.x, `MCPServer`).

## License

MIT