EchoProbe

Security, privacy and voice-readiness toolkit for MCP tools used by voice assistants (Alexa+ track). EchoProbe is itself a self-hosted MCP server (Streamable HTTP, MCP spec 2025-11-25) and a one-command audit.

Quick start (macOS)

Double-click start.command (first run: right-click, Open). It sets up a virtual environment, installs the requirements and opens the EchoProbe web app in Chrome at http://127.0.0.1:8080. Put your API key in the app via Settings. Tabs: Try it, Full audit, MCP server, Checklist. Manual alternative: python3 -m venv .venv && source .venv/bin/activate && pip install -r requirements.txt && python app.py.

Related MCP server: meok-mcp-injection-scan-mcp

Features

• Prompt-injection firewall: rules + LLM judge (Gemini), incl. paraphrased, hidden, multi-step and multilingual (Bangla, Hindi, Spanish, French) attacks; custom rules in rules.json.

• Tool-poisoning audit: scans tool names/descriptions of any MCP server for hidden instructions (audit_server).

• Live exploit demo: a deliberately naive LLM agent uses poisoned tools; leaks of planted secrets are measured with and without protection (incl. a multi-step attack and a repeated-call loop guard).

• Detection benchmark (20 attacks + 20 harmless outputs, per category) and an LLM red team that writes fresh attacks.

• Confirmation gate for risky actions (send_message, purchases, deletes).

• Voice readiness: voice score, rule-based spoken rewrite (voice_fix), latency check, personal-data detection (email, phone, card, PIN/password), link allow-list.

• Simulated users: hurried, typo-prone, attacker.

• Report: report.html (A-F heuristic grades, charts), results.json, append-only echoprobe_audit.log, LLM call/token counters.

• Extras: voice_demo.py (macOS say: hear before/after), proxy.py (experimental drop-in protection proxy), GitHub Action running the offline tests.

Setup

python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

Put your Google AI Studio key in .env (GEMINI_API_KEY=...); model defaults to gemini-3.6-flash (GEMINI_MODEL). Without a key everything still runs; LLM-based steps are skipped. Never commit .env.

Run

python app.py          # web app (Chrome opens automatically)
python tests.py        # offline tests (no key, no network)
python check_llm.py    # is the key/model working?
python run_all.py      # runs everything, writes report.html + results.json, opens the report (a few minutes)
python server.py       # EchoProbe as an MCP server: http://127.0.0.1:8000/mcp
python voice_demo.py   # macOS: hear the voice rewrite

MCP tools exposed by EchoProbe

scan_response, scan_hybrid, check_tool_call, voice_check, voice_fix, pii_check, check_links, run_probe, audit_server

Limitations (honest)

• Rules miss paraphrased/multilingual injections; the LLM judge catches more but can itself be fooled, costs API calls and can be rate limited.

• The benchmark is a small self-written set; grades are heuristic summaries, not a standard.

• The live agent is deliberately naive and LLM behaviour varies between runs; the agent is not a real Alexa+ device.

• proxy.py is experimental.

What was built during the hackathon

Everything in this repository was created during the submission period.