# mcp-server-framework-cost

The same eight Python functions, registered as MCP tools in three frameworks,
measured for what they cost the model on every request.

Measured 2026-09-18 on an Apple M2 (4 performance + 4 efficiency cores), 8 GB,
macOS 26.6.2, Python 3.14.7, pydantic 2.13.5.

* `fastmcp` **4.0.5** (released 2026-09-17), on `mcp` 2.2.0
* `mcp` **2.2.0**, the official Model Context Protocol Python SDK (released 2026-09-07)
* `fastapi-mcp` **0.4.0** (released 2025-07-28), pinned to `mcp` 1.30.0 — see below

## Results

```text
TOKENS ADDED TO EVERY REQUEST BY THE SAME EIGHT TOOLS
--------------------------------------------------------------------
framework                tokens   vs cheapest    bytes on wire
--------------------------------------------------------------------
fastmcp 4.0.5               937         1.00x           10,068
mcp 2.2.0 (official)      1,129         1.20x           11,537
fastapi-mcp 0.4.0         1,724         1.84x            6,667
--------------------------------------------------------------------

WHERE THE DIFFERENCE GOES
--------------------------------------------------------------------
framework               tokens  removable  what the removable part is
--------------------------------------------------------------------
fastmcp 4.0.5              937
mcp 2.2.0 (official)     1,129       -210  generated "title" keys
fastapi-mcp 0.4.0        1,724       -554  injected response example
--------------------------------------------------------------------

DOES THE PUBLISHED SCHEMA MATCH THE PYTHON FUNCTION?
--------------------------------------------------------------------
framework               schema wrong  server wrong    of probes
--------------------------------------------------------------------
fastmcp 4.0.5                      0             0           25
mcp 2.2.0 (official)               0             0           25
fastapi-mcp 0.4.0                  4             4           25
--------------------------------------------------------------------

MEDIAN tools/call ROUND TRIP, MS (300 CALLS, 50 WARM-UP DROPPED)
--------------------------------------------------------------------
framework                    get_task   search_tasks    update_task
--------------------------------------------------------------------
fastmcp 4.0.5                    1.08           1.11           1.11
mcp 2.2.0 (official)             0.91           0.94           0.91
fastapi-mcp 0.4.0                1.28           2.02           2.63
--------------------------------------------------------------------
```

## What is being measured

`toolspec.py` holds the models and the function bodies. `tools_shared.py`
registers eight tools and is imported by **both** the FastMCP server and the
official-SDK server, so those two receive the identical function objects,
annotations and docstrings. `server_fastapi.py` expresses the same eight
operations as FastAPI routes, which is the only shape fastapi-mcp accepts.

`capture.py` performs the streamable-HTTP handshake by hand and records the
literal `tools/list` JSON each server puts on the wire. No client library sits
in between, so nothing is normalised before it is counted.

Token counts are the canonical projection a client hands the model —
`{name, description, input_schema}` — serialised minified and encoded with
tiktoken `o200k_base`. `sensitivity.py` repeats the count under `cl100k_base`
and `p50k_base`, and per tool: the ranking is identical in all three encodings
and on all eight tools individually.

`probes.py` holds 26 calls with the verdict each one should get, written
against the Python signatures rather than against any framework's output.
`analyze.py` validates them against each published schema; `runtime_probe.py`
sends the same calls to each live server. One probe is excluded from the
headline count because its verdict depends on how a framework maps nested
models rather than on whether it describes them correctly.

## Three findings

**1. fastapi-mcp 0.4.0 does not construct against the current MCP SDK.** It
declares `mcp>=1.12.0` with no upper bound, so a plain `pip install fastapi-mcp`
resolves `mcp` 2.2.0 and then raises:

```
TypeError: Server.__init__() takes 2 positional arguments but 3 were given
```

from `FastApiMCP.__init__`, before the server starts. Everything here was
measured with `mcp` pinned to 1.30.0. `run.sh` reproduces the failure first.

**2. 210 of the official SDK's 1,129 tokens are Pydantic's auto-generated
`title` keys** — `"title": "Task Id"` next to a property already named
`task_id`. `mcp` 2.2.0 exposes no option to suppress them;
`server_sdk_trimmed.py` strips them from the registered tool's `parameters`
dict, which is a private attribute. That brings the block to 919 tokens and all
25 probes still return the correct verdict.

**3. fastapi-mcp publishes schemas narrower than the API they wrap.** For an
optional field it emits an `anyOf` permitting null *and* a sibling `"type"`
forbidding it, so the null branch is unreachable. Four of the eight tools lose
the ability to send an explicit null over MCP — while the same FastAPI endpoint
returns 200 for the same body over plain HTTP.

## Reproducing

```bash
./run.sh
```

Needs `uv` and Python 3.14. Writes every artefact into `results/`.

## Layout

| File | What it does |
|---|---|
| `toolspec.py` | models, docstrings and function bodies |
| `tools_shared.py` | registers the eight tools on FastMCP and the official SDK |
| `server_fastmcp.py` / `server_sdk.py` / `server_fastapi.py` | the three servers |
| `server_sdk_trimmed.py` | the SDK with `title` keys stripped |
| `server_fastmcp_refs.py` | FastMCP with `dereference_schemas=False` |
| `capture.py` | raw `tools/list` wire capture |
| `probes.py` / `analyze.py` / `runtime_probe.py` | schema and server agreement |
| `decompose.py` / `sensitivity.py` | attribution and stability checks |
| `latency.py` | 300 `tools/call` round trips per tool |
| `summarize.py` / `tables.py` / `charts.py` | `results/summary.json`, tables, figures |

## Limits

Localhost, one machine, one run. Absolute milliseconds are a property of this
laptop; token counts and schema shapes are properties of the libraries and
should reproduce anywhere. Eight tools is a small server. `latency.py` compares
fastapi-mcp on `mcp` 1.30.0 against the other two on 2.2.0, because that is the
only configuration in which it runs.

## Licence

MIT.