The MCP QA Lab server offers a comprehensive set of tools for evidence-based quality testing of MCP servers:
Target Management: Register stdio or Streamable HTTP targets securely by referencing environment variable names (never values) and list them without exposing secrets. Local targets are restricted to safe working directories.
Live Inspection: Connect to a target and capture its complete, paginated model-facing contract (tools, prompts, resources, and resource templates).
Static Analysis: Deterministically analyze schemas, descriptions, annotations, naming, and safety hints to identify usability risks without invoking tools.
Context-Cost Measurement: Measure serialized size and approximate token cost of model-facing metadata, and detect duplicate descriptions.
Scenario Generation: Use host-model sampling (with fallback) to propose task-oriented, multi-tool test journeys based on a user-defined objective.
Controlled Tool Execution: Invoke a target tool with reviewed JSON arguments, requiring explicit approval for tools not annotated as read-only or with side effects.
QA Reports: Build redacted Markdown reports from live inspection and deterministic checks, with credentials, tokens, and headers automatically removed.
Safety Defaults: Deny remote targets unless explicitly allowed, restrict local working directories, never persist environment variable values, and redact sensitive information.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@MCP QA Lab Register a new MCP server for testing"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
English | 한국어
Evidence-driven quality testing for Model Context Protocol servers. MCP QA Lab connects as a real client, captures the complete model-facing contract, finds agent-usability risks, measures context cost, and runs only explicitly approved target calls.
A server can have perfect handler-level tests and still be difficult for an agent to use. Tool names may be ambiguous, input fields may be undocumented, pagination may be incomplete, and safety intent may be missing. MCP QA Lab tests that public protocol surface separately from implementation code.
flowchart LR
A[Register target] --> B[Live protocol inspection]
B --> C[Static contract checks]
B --> D[Context-cost measurement]
C --> E[Scenario generation]
E --> F[Approved calls only]
D --> G[Evidence report]
F --> GRelated MCP server: MCProbe
Complete paginated discovery of tools, prompts, resources, and resource templates
Deterministic checks for schema quality, naming, descriptions, and safety annotations
Model-facing contract size and approximate context-cost measurement
Host-model sampling for task-oriented scenario proposals
Explicit approval boundary for target tool calls with possible side effects
Secret-free target registration using environment variable names, never values
Redacted Markdown reports with reproducible evidence
Stdio and Streamable HTTP target support
Install directly from the public repository:
uv tool install "git+https://github.com/efficjump/mcp-qa-lab.git"
mcp-qa-lab --transport stdioFor source development:
git clone https://github.com/efficjump/mcp-qa-lab.git
cd mcp-qa-lab
uv sync --all-extras --locked
uv run mcp-qa-lab --transport stdio{
"mcpServers": {
"mcp-qa-lab": {
"command": "mcp-qa-lab",
"args": ["--transport", "stdio"]
}
}
}Client configuration formats vary; the command and arguments above do not require a local repository path after tool installation.
Tool | Purpose |
| Persist a secret-free stdio or Streamable HTTP target definition |
| List registered targets |
| Capture the live, paginated MCP contract |
| Find deterministic contract and usability issues |
| Measure serialized contract size and approximate token cost |
| Ask the host model for evidence-oriented test journeys |
| Invoke one reviewed target tool with safety approval |
| Write a redacted Markdown QA report |
Remote targets are denied unless MCP_QA_ALLOW_REMOTE=1.
Local target working directories must stay inside MCP_QA_ALLOWED_ROOTS when configured.
Registrations reference inherited environment variable names; values are never persisted.
Inspection and static checks never invoke target tools.
Tools without readOnlyHint: true require explicit side-effect approval.
Reports redact credentials, authorization headers, tokens, and URL user information.
The process boundary is not an operating-system sandbox. Run untrusted targets in an isolated environment.
mcp-qa-lab --transport streamable-http --host 127.0.0.1 --port 8765The default endpoint is http://127.0.0.1:8765/mcp. Keep the listener on loopback unless a separate
authentication and network boundary is in place.
uv sync --all-extras --locked
uv run ruff format --check .
uv run ruff check .
uv run mypy src
uv run pytest --cov --cov-report=term-missing
uv buildThe coverage gate is 85%. See architecture, security policy, and contribution guide.
Create a redacted Markdown report from a new live inspection and deterministic checks.
Inputs: target_id (required). Returns: object
Generate task-oriented multi-tool journeys using sampling, with an explicit fallback.
Inputs: objective (required), target_id (required), maximum_scenarios. Returns: object
Connect to a target and capture its complete paginated model-facing MCP contract.
Inputs: target_id (required). Returns: object
List registered targets without resolving or exposing environment values.
Inputs: no named parameters. Returns: result
Measure serialized model-facing tool metadata and exact duplicate descriptions.
Inputs: target_id (required). Returns: object
Register a secret-free stdio or Streamable HTTP target after boundary validation.
Inputs: name (required), transport (required), cwd, url, +4 more. Returns: object
Analyze live schemas, descriptions, annotations, and size without tool calls.
Inputs: target_id (required). Returns: object
Execute one call; non-read-only tools require explicit side-effect approval.
Inputs: arguments (required), target_id (required), tool_name (required), allow_side_effects. Returns: object
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
Scored across 8 tools
Most tools have clearly distinct purposes (listing, registering, inspecting, checking, measuring, generating, executing, reporting). The only potential confusion is between run_static_checks and measure_context_cost, but their descriptions clarify that one focuses on quality/size analysis and the other on serialized metadata cost and duplicate descriptions.
All tool names follow a consistent verb_noun pattern (list_targets, register_target, inspect_target, run_static_checks, measure_context_cost, generate_scenarios, run_target_tool, build_report). The naming is uniform and predictable.
Eight tools is an appropriate scope for a QA lab. The set covers the main workflow areas without being bloated or sparse, aligning well with the typical 3-15 tool sweet spot.
The toolset covers the core workflow: register, inspect, run checks, measure cost, generate scenarios, execute a tool, and build a report. A minor gap is the absence of a delete/unregister target operation, which could make target lifecycle management incomplete.
Pre-execution governance for AI agents. Deterministic PASS/FAIL/REVIEW verdicts, replayable proof.
LLM cost optimization gateway, repo token-waste auditor & model arbitrage MCP server.
MCP server for building and testing AI agents with multi-model experimentation and insights.
Scan prompts for injection attacks, redact PII, and audit LLM SDK usage from any MCP client
