MCP Vetting Agent

An AI security auditor for AI tools. An agent, running on TrueForge, that vets third-party MCP servers for security vulnerabilities before you connect them to your agent — then pauses and asks a human before filing a public security report.

TrueForge connects agents to any MCP server. Community servers are shared as gists, templates, and side projects — with hardcoded keys, eval() on tool args, and no auth boundaries. Agents don't know to be suspicious. This agent is the suspicious one.

How it works

User: "audit https://github.com/someone/some-mcp-server"
  └─ TrueForge agent (mcp-vetting)
       ├─ clone_target          ── shallow-clones the GitHub URL onto the probe host
       ├─ read_target_manifest   ── declared tools & permission boundaries
       ├─ subagent: static_audit ── AST rules + pattern matching (VULN-001..007)
       ├─ subagent: full_audit   ── AI review + Docker probes (VULN-008..011)
       ├─ Synthesizes verdict (HIGH/MEDIUM/LOW, OWASP Agentic Top 10 mapped)
       └─ ⏸ PAUSES before filing the GitHub security issue → human approves → files

• Probes run in isolation: Dynamic probes execute the target server inside throwaway Docker containers.

• Approval gate is native TrueForge HITL: issue creation is a write/destructive action, so the harness pauses for Allow/Deny.

• Sessions survive reconnects: refresh mid-audit; the agent keeps working.

Related MCP server: GitHub PR Review MCP Server

What the agent does

The MCP Vetting Agent audits third-party MCP servers for security vulnerabilities before anyone connects them to an AI agent. It answers one question: "Is this server safe to trust?"

Given a GitHub URL, the agent clones the repository, reads its declared tools and permissions, then runs two parallel security scans — static analysis (pattern matching for known vulnerability classes) and dynamic analysis (executing the server in isolated Docker containers to observe real behaviour). Findings are mapped to the OWASP Agentic Top 10 and synthesised into a risk verdict: HIGH, MEDIUM, LOW, or CLEAN.

If the verdict is HIGH, the agent drafts a GitHub security issue with per-finding details and remediation hints, then pauses and waits for human approval before filing anything. Nothing irreversible happens without a person saying yes.

How it uses TrueForge

TrueForge is the agent harness — the runtime layer between the model and the tools it calls. Every part of the audit runs through TrueForge:

| Subagents | Static and dynamic audits run in parallel as delegated subagent tasks, keeping the main agent context clean. | | Session persistence | The audit session survives browser refreshes and reconnects. If the connection drops mid-audit, the agent continues and the UI reattaches to the running session. | | Model flexibility | Runs on a local Ollama model (qwen2.5:7b) — no API key required, no data leaves the machine. |

The harness is doing the real work: reaching tools, running code safely, and stopping for a person before anything irreversible.

Tech Stack

Quick start

Deployed services:

• Frontend (Vercel): https://ui-blond-six.vercel.app/

• Probe server (Render): https://mcp-vetting-probe.onrender.com

# 1. Probe server (the security scanning engine)
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
uvicorn probe_server.server:mcp_app --host 127.0.0.1 --port 8000   # serves /mcp

# 2. TrueForge (separate terminal; needs Node 22+)
npx @truefoundry/trueforge@latest                                   # UI at http://localhost:8790

# 3. In the TrueForge UI:
#    Settings → Models      : configure a provider (API key)
#    Settings → Connectors  : Add MCP Server → http://127.0.0.1:8000/mcp
#                             (+ GitHub connector from the catalog, OAuth)
#    Settings → Sandbox     : optional (skills/code mode); cloning runs on the probe host
#    Create agent           : import deploy/agent-manifest.json via API, or compose in UI

# 4. Chat: "audit ./fixtures/vulnerable_server"

Try it against the bundled fixtures first:

The console

There are two ways to drive the agent: the TrueForge chat above, or the web console in ui/ — a static React SPA that speaks MCP straight to the probe server, with no backend in between.

# terminal 1 — the probe server
uvicorn probe_server.server:mcp_app --host 127.0.0.1 --port 8000

# terminal 2 — the console
cd ui && npm install && npm run dev     # http://localhost:5173

The console presents the audit as a live graph with two lanes: reading the source and running it sealed in a container keep separate visual languages and only merge at synthesis, because a defect is fact only where both agree. It shows what the agent is doing, what it is waiting on, and what it did — and asks before the irreversible step rather than after it. Filing goes through file_github_issue on the probe server, which reads GITHUB_TOKEN from its own environment; the browser never handles a credential.

If the security_scanner engine is not installed, start the probe server with VETTING_DEV_FIXTURES=1 to replay a captured report. Replayed reports are tagged sample_data: true and the console labels them everywhere they appear.

See ui/README.md for the architecture and ui/DESIGN_DIRECTION.md for the locked visual direction.

Repository layout

probe_server/    MCP server exposing the scanning engine as agent tools
ui/              Web console (React + Vite + Tailwind + Zustand + MCP SDK)
fixtures/        Vulnerable + hardened reference MCP servers
deploy/          TrueForge agent manifest (agent spec via API)
docs/            PRD, architecture, week plan, setup guide

Qodo Code Review Evidence

Every substantive change in this repository went through a pull request reviewed by Qodo before merge — starting from the first day of the hackathon.

Representative reviewed PR: #1 — feat: probe server, fixtures, tests, TrueForge agent spec

What Qodo surfaced and what we did about it (full trail visible on the PR):

One finding was dismissed with a recorded reason: the clean fixture's zero-value integrity digest (comment on the PR) — it ships that way upstream, and the code path involved is never exercised by our tools, so we kept our fixtures identical to upstream rather than forking them.

The PR history shows each review, the commits addressing its findings, and follow-up reviews confirming resolution against the final code.

🎬 Demo

Watch the 3-minute walkthrough: MCP Vetter Agent Demo on YouTube

See the system in action:

• Parallel security probes running in real-time

• Real vulnerabilities detected with OWASP Agentic Top 10 mapping

• Human approval gate demonstrated (3-second hold before irreversible action)

• GitHub issues filed to real repositories

Try it yourself:

• Live Console: https://ui-blond-six.vercel.app/

• Probe Server Health: https://mcp-vetting-probe.onrender.com/health

• Test Fixture Repo: https://github.com/hemv-857/mcp-vetter-fixture-target

Enter the fixture repo URL in the live console to see a real security audit in action.

AI tooling disclosure

This project was built with the assistance of AI coding assistants (Claude Code, Cursor). All AI-generated code was reviewed, tested, and understood by the contributors before merging.

Credits

Built for The Agent Harness Hackathon (WeMakeDevs × TrueFoundry, Aug 2026). See fixtures/LICENSE for bundled reference-code licensing.