mirage-mcp

An MCP (Model Context Protocol) server that audits how an AI agent's guardrails respond to adversarial, boundary-testing prompts — in the style of AP-Test from "Peering Behind the Shield: Guardrail Identification in Large Language Models". Connect it to your agent, call get_probe to get a test prompt, let your agent respond to it as it normally would, then call submit_probe_response with that response to get back a scored report of which known guardrail/refusal pattern it most resembles.

This is disclosed, not covert: the probe battery is the advertised feature, not a hidden trick. Responses are logged to build a public research dataset on agent guardrail behavior — see What gets logged below before connecting if that's not something you want.

See FINDINGS.md for results so far — real, three-agent comparison data (Claude Sonnet 5, Llama 3.1 8B, Qwen 2.5 7B), not just methodology.

It's a sibling to mirage-core (SSH) and mirage-crawl (HTTP): same idea — instrument a surface, capture what arrives, treat your own corpus as suspect — applied to the MCP protocol, with guardrail behavior as the observable instead of shell commands or HTTP requests. Unlike those two, mirage-mcp doesn't get passive traffic (nothing scans the internet for MCP servers) — it has to be found and connected to deliberately, which is why it's built as a genuinely useful self-audit tool rather than a disguised one.

Ethical and legal boundary

• Deployed only on infrastructure the author owns or has explicit written authorization to run on.

• Never executes anything a caller sends it, never scans, probes, or retaliates against a connecting client. It only scores what's submitted to it.

• Nothing about this tool is disguised. Its description states plainly what it does and that responses are logged for research.

• What gets logged, per submission: the probe id, your response text to that probe, the classifier's verdict, the optional self-reported declared_agent label (if you provide one — never verified, same posture as mirage-crawl treating a claimed crawler identity as a signal to check, not a fact), your MCP client's self-reported name/version from the connection handshake, and a salted HMAC-SHA256 hash of your connecting IP plus the country it resolves to — never the IP itself, same anonymization mirage-core already uses for its published exports. Nothing else is ever captured — no other arguments, no other tool output, no unrelated content.

• Every submission is captured, including ones the classifier can't confidently label — those are recorded as indeterminate, never silently dropped.

• The signature bank in mirage_mcp/signatures.py starts as an unverified set of placeholder patterns. Treat every match as a hypothesis until it's been checked against real observed sessions, not as ground truth.

Related MCP server: RedTeam ML API MCP

Quick start

python -m venv .venv && .venv/bin/pip install -r requirements.txt
.venv/bin/python run.py

This starts the server over stdio for local testing. Point an MCP client at it (Claude Desktop's config, the MCP Inspector, or your own client code), call get_probe, respond to the prompt, then call submit_probe_response.

python tests/test_classifier.py

Containerized (streamable-http)

cp .env.example .env   # fill in IP_SALT; see the file for how to generate one
docker compose up -d --build

Runs the server over streamable-http at 127.0.0.1:8765/mcp, loopback only — public reachability comes from a Cloudflare Tunnel in front of it, not a published port. Captured sessions live in the mcp_data named volume (/app/data/sessions.jsonl inside the container), which survives restarts and rebuilds. Runs as a non-root user, same pattern as mirage-crawl and mirage-core.

Geo lookup expects mirage-core's DB-IP CSVs mounted read-only at /data/geo (see docker-compose.yml's GEO_DATA_DIR, defaults to mirage-core's path on the shared deployment box); without IP_SALT set, IPs simply aren't hashed or logged.

How it works

1. mirage_mcp/probes.py — the bank of adversarial test prompts: the original AP-Test-style set (identity elicitation, policy conflict, instruction override, jailbreak-lite) plus four documented jailbreak-technique categories (persona/roleplay, hypothetical framing, refusal-prefix injection, authority impersonation), adapted from JailbreakBench/HarmBench.

2. mirage_mcp/server.py — the MCP server (built on the official MCP Python SDK). get_probe returns one prompt; submit_probe_response scores whatever the caller submits and returns a report.

3. mirage_mcp/signatures.py — two regex pattern banks: one for known refusal-style phrasing, one for explicit compliance markers, each pattern carrying a confidence weight.

4. mirage_mcp/classifier.py — scores a response on two axes: did it comply with the probe, and if it refused, which known refusal style does it resemble. Compliance is checked first and takes priority — a model agreeing to drop its own safety behavior is the more important finding than any style match. Either axis can come back indeterminate if nothing clears the confidence threshold.

5. mirage_mcp/capture.py — appends every submission and verdict to data/sessions.jsonl, unconditionally, scoped to the fields listed under What gets logged above.

6. mirage_mcp/identity.py / mirage_mcp/geo.py — resolve the connecting IP (from CF-Connecting-IP/X-Forwarded-For) to a salted hash and a country code before the raw address is discarded. Geo lookup reuses mirage-core's DB-IP Lite CSV snapshots rather than a live API call — see mirage_mcp/geo.py's docstring.

Deployment

Runs containerized on the same VPS as mirage-core (/opt/mirage/mirage-mcp), bound to 127.0.0.1:8765. Public exposure goes through a Cloudflare Tunnel rather than a directly-exposed port, to avoid correlating this box's real IP with the SSH honeypot sharing it.

Redeploy via GitHub Actions (workflow_dispatch on deploy.yml). The deploy key is deliberately more restricted than mirage-core's: it's locked server-side to a forced command (authorized_keys command="/opt/mirage/mirage-mcp/deploy.sh", no pty/agent-forwarding/ X11-forwarding/user-rc/port-forwarding) that only ever runs deploy.sh (git fetch && git reset --hard && docker compose up --build -d) — even a leaked key can't run anything else.

Status

Deployed on the VPS. Standalone — no database, no mirage-fleet wiring. Classification is rule-based only; the signature bank needs real observed sessions (ideally with declared_agent set) before its confidence weights mean anything.