mcp-policy-gateway

Runtime policy enforcement for Model Context Protocol tool calls, and a deterministic benchmark that measures which controls actually stop which attacks.

On a 47 case corpus the gateway catches 25 of 26 attacks and wrongly blocks 2 of 21 legitimate calls. A keyword filter, the alternative teams actually reach for, catches 10 and blocks 7. Every number here is reproduced by python evaluation/benchmark.py.

The problem

An MCP client hands a model a set of tools, calls them on the model's behalf, and feeds the results back into context. Three things in that loop are untrusted, and they are untrusted at different times:

1. Tool declarations. A server describes its own tools. A description is prose that most clients concatenate into a prompt, so a server can put instructions there.

2. Call arguments. Built by a model from a schema, then sent to a server that may or may not validate them.

3. Tool results. Whatever comes back is read into context. It is a document, a search result, a database row. It is content the server did not write and cannot vouch for.

The security tooling that exists for MCP today is mostly static scanners: point them at a server, they read the declarations and report suspicious ones. That is worth doing and it covers exactly one of the three. It cannot cover the third at all, because a poisoned search result does not exist until someone runs the search.

This project puts the enforcement point at the protocol boundary instead, where all three are visible, and then measures how much that is worth.

Related MCP server: SafeNode MCP Gateway

What it is

An MCP server that fronts another MCP server. Your client connects to the gateway; the gateway connects to the real server. Both sides speak ordinary MCP, so it works with a server whose source you do not have.

Nine controls run at those three stages:

A control never decides on its own. It reports findings; the engine combines them, and the strongest action wins: allow, sanitise, hold for approval, or block.

Results

Deterministic. 47 cases, no model calls, no network. python evaluation/benchmark.py reproduces every rate in this table exactly on any machine. The decision-time column is the exception: it is a property of the machine it ran on, not of the system, and it moves by tens of microseconds between runs.

The keyword filter is in there because it is the real alternative. "We added a filter" is what actually happens when a team decides to do something about prompt injection, and comparing only against nothing would flatter the result. It catches a third of the attacks and refuses one in three legitimate calls, which is the profile of a control that gets switched off in week two.

How much of that gap the corpus can support

Every rate above is a count over 26 attacks or 21 benign cases. At those sizes a percentage to one decimal place reads as more precise than it is, so the benchmark also prints a 95% Wilson interval for each. Wilson rather than the usual normal approximation, which at 25 of 26 puts the upper bound past 1 and at 0 of 21 claims no uncertainty at all.

The two columns say different things and should be read differently.

On recall the gap is real. The gateway's [81.1%, 99.3%] does not come near the keyword filter's [22.4%, 57.5%]. Twenty-six cases are few, but not so few that this separation could be luck.

On false refusals it is not established. [2.7%, 28.9%] against [17.2%, 54.6%] overlap across most of their width. The gateway refused two legitimate calls where the filter refused seven, and on 21 cases that is the direction the evidence points rather than a finding to quote. Claiming otherwise would be the error this project exists to avoid.

The intervals cover sampling error only: how much these rates would move on another 47 cases written the same way, and nothing about real traffic.

Per control, on the same run:

Every benign case that gets touched is touched by the one control that has to make a judgement call. The eight deterministic controls have no false positives on this corpus, which is the argument for keeping the judgement in exactly one place.

Where it fails

Three cases out of 47 are not handled, and all three were written before the controls were.

inject-006 used to be a fourth. It was a base64 override instruction with no plaintext around it, and the stated reason for leaving it was sound: decoding every base64-looking span before matching would flag legitimate encoded attachments. What closed it was narrowing the decode rather than accepting the cost. A span is only re-scanned if it decodes to valid UTF-8 that reads as prose, so an image, an archive, a key and a digest are all dropped before any rule sees them. Three benign cases were added at the same time to measure that: an encoded attachment, encoded release notes carrying the override rule's entire vocabulary without an imperative, and an advisory quoting the payload it warns about. None of them is a false positive, and the advisory is sanitised rather than refused. The decode can be turned off, and the benchmark can be run either way.

The two false positives are the honest cost of the injection control, and they are the reason it is the only control allowed to be uncertain. Both survived the decode: they are plaintext, and nothing about reading one encoding layer deeper helps with a sentence whose meaning depends on who is being addressed. Full write-up in docs/learning/06_failures.md.

Scored against a corpus I did not write

Everything above is measured on 47 cases I wrote, which is the standard weakness of a self-built benchmark. So the gateway was also run against MCPTox (AAAI 2026), 485 poisoned tool declarations harvested from real MCP servers by people who had never seen this project.

The first number is the useful one. mandatory_tool_precondition bounded the gap between "before" and the obligation at 40 characters, which fits how I write a case and not how anyone else does; 72 of the 103 missed declarations carrying that pair exceeded it. A second attack class — a declaration claiming priority over the user, then coercing an argument value — had no rule at all. Neither could have been found on 47 cases of mine.

The internal numbers did not move through either change, including the benign cases that caused that rule to be narrowed in the first place.

The benign half is external too, because recall without a false-positive rate is the shape of result this project criticises the keyword filter for. Scored against 660 real tool declarations — harvested from the official MCP reference servers and from MCP-Guard's benign set — the gateway blocks 0 and touches 0, interval [0.0%, 0.6%].

That includes the fetch server's own description, which tells the model it was "advised to refuse and tell the user this" and that it may now do otherwise. Official prose with the shape of an injection, and it passes.

A second external source, MSB (ICLR 2026), reaches the stage none of the above does: fifteen attacks arriving inside tool results, which is the case a static scanner structurally cannot cover. The gateway caught 0 of 15. Three wrappers failed for three reasons, the first being that mandatory_tool_precondition knew call, invoke, execute and run but not use — the most ordinary word for invoking a tool. After fixes taken from threat rules ATR-2026-01302 and ATR-2026-01751 it catches 15/15, and MCPTox rose 49.5% → 55.1% at no false-positive cost. That transfer, from a fix found on one corpus to a corpus by different authors, is the least circular evidence here.

55.1% should not be read as generalisation. A large part of the gain is one rule whose trigger phrase appears in 0% of their first two generators and 35% of the third — one generator's opening sentence, not a property of tool poisoning. 245 of 485 are still missed and the three generators disagree at 29.9%, 45.4% and 59.6%. These rates also are not comparable to the ones in their paper: MCPTox scores whether a model complies, this scores whether a control fires.

The changes that produced it came from reading published work rather than inventing rules: two detection rules transcribed from MCP-Guard, the localisation gap closed from threat rule ATR-2026-00148, and argument_coercion rebuilt as weighted scoring against a threshold after MCP-Guard's shadow_rules.json — which alone was worth ten points of recall at no false-positive cost. Both decision conventions are now reported, following CASCADE.

Nothing of theirs is redistributed here — they publish no licence, so the corpus is pinned by commit and digest and fetched at evaluation time. make check and make bench stay offline.

make bench-external     # fetches once, then deterministic

Full write-up, including what it does not establish, in docs/learning/08_external_validation.md.

What it cannot see

A recall number over a corpus of tool poisoning is silent about the other sixteen things on the published list. So the nine controls are also mapped against MCPSecBench's taxonomy of 17 attack vectors across four surfaces: 8 covered, 3 visible but uncovered, 6 structurally out of reach from a position at the protocol boundary.

The three uncovered ones are a single missing capability. Package name squatting by server name, configuration drift and rug pull are all visible here, and none is addressed, because every control is a pure function of one event: nothing remembers what a server declared last time. A definition that was benign on Monday and hostile on Friday is two independent evaluations, both correct, and the change between them is the attack.

make coverage-map

The map is data in evaluation/coverage.py, not prose, and tests hold it to the taxonomy and to the control set in both directions. Full reasoning per vector in docs/learning/09_coverage.md.

Run it

python -m venv .venv && . .venv/Scripts/activate     # or bin/activate
pip install -e ".[dev]"

python -m mcp_policy_gateway.cli demo                # real MCP process, real stdio
python evaluation/benchmark.py                       # the table above
pytest                                               # the test suite

demo launches a deliberately hostile MCP server from examples/, drives it through the gateway and prints every decision. It needs no API key, no network and no configuration. That is the point: an evaluation that needs a key is an evaluation nobody re-runs.

To put the gateway in front of a server you already use:

python -m mcp_policy_gateway.cli proxy \
  npx -y @modelcontextprotocol/server-filesystem /srv/docs \
  --sandbox /srv/docs \
  --allow read_file --allow list_directory \
  --destructive write_file \
  --allow-host docs.internal.example.com

Upstream foundation

This is an original implementation. It is not a fork and no third-party source is vendored here.

• MCP Python SDK (MIT) is a dependency. The gateway speaks MCP through it rather than reimplementing the wire format. Targets the 2.x API, where FastMCP became MCPServer.

• Prior art, read but not used as code: OWASP MCP Tool Poisoning for the threat vocabulary, and Snyk agent-scan (Apache-2.0) as the reference example of the static-scanning approach this measures itself against.

Full attribution in NOTICE.

What is mine

The gateway, all nine controls, the policy engine, the corpus, the benchmark, the trace format and the CLI. Specifically:

• Three-stage enforcement. Splitting discovery, request and response, so response- stage attacks are reachable at all.

• A corpus with ground truth per case. 26 attacks, 21 benign near-misses. Each case states the weakest acceptable response, because "block everything" is wrong for a credential inside a legitimate document.

• Benign near-misses as a first-class half. The false-positive rate is what decides whether a control survives contact with an operator, and it is the number the keyword baseline loses on.

• Context-demotion in the injection control. Quoted, fenced and reported text demotes a finding from block to redact, which is what makes a security advisory readable.

• Per-control attribution. Every control runs even after another has blocked, so the effectiveness table is not an artefact of ordering.

• Fail-closed proxying over real stdio MCP, with a JSONL trace that stores digests rather than payloads.

Figures

Generated from assets/results.json and the control registry, so they cannot describe a system the code does not have:

python scripts/figures/generate_diagram.py   # docs/architecture.svg
python scripts/figures/generate_figures.py   # docs/figures/*.png

Documentation

docs/learning/ works from the problem up: what MCP is and why its trust boundary is unusual, the threat model, the architecture, the implementation, how the evaluation is constructed, where it fails, and what to try next.

Licence

MIT. See LICENSE.