qa-probe
OfficialAn MCP server that lets an AI assistant inspect and act on qa-probe QA results for a project.
Get the frontend-to-backend dependency graph, optionally filtered to a frontend route.
Fetch QA report scores and root causes, filtered by route status.
Live-probe a single backend endpoint with an optional HTTP method.
Explain why a frontend route is failing or showing no data.
Get step-by-step fix guidance for supported root-cause categories.
Find the blast radius of a backend endpoint outage across frontend routes.
Trigger a full analyze + probe + report run and return a summary.
Record feedback on endpoint diagnoses to suppress or confirm findings in future runs.
Probes and analyzes Express backend endpoints to diagnose blank pages, empty data, and contract mismatches, providing root cause and fix hints.
Probes and analyzes FastAPI backend endpoints to diagnose blank pages, empty data, and contract mismatches, providing root cause and fix hints.
Probes and analyzes GraphQL endpoints to diagnose blank pages, empty data, and contract mismatches, providing root cause and fix hints.
Probes and analyzes Next.js backend endpoints to diagnose blank pages, empty data, and contract mismatches, providing root cause and fix hints.
Probes and analyzes tRPC endpoints to diagnose blank pages, empty data, and contract mismatches, providing root cause and fix hints.
qa-probe
Find out exactly why your pages are blank - in under 5 minutes.
qa-probe is a Node.js command-line tool that reads your React frontend, maps every API call to its backend route, probes those endpoints live with real auth, and tells you the root cause when something is wrong - not just a status code.
Overall score: 74/100
Root causes detected:
empty_db: 5 route(s) seed the database
feature_flag_disabled: 3 route(s) enable HAS_BILLING, HAS_REPORTS
contract_mismatch: 1 route(s) frontend calls /api/users/, backend serves /users
Route Score Status Root cause
----------- ----- -------- ----------------------
/dashboard 100 healthy
/users 80 healthy empty_db
/billing 0 broken feature_flag_disabled
/reports 50 degraded empty_db
/settings 85 healthyIt works best on React apps backed by FastAPI, Express, Next.js, GraphQL, or tRPC with an OpenAPI spec available. Built by the LightShield SIEM team and battle-tested against a production SIEM.
The problem it solves
Your dashboard loads. Nothing crashes. But every table is empty, every chart shows zero, and you have no idea why. The cause is almost always one of:
The frontend calls
/api/users/but the backend route is/api/users(trailing slash)A feature flag (
HAS_BILLING=false) silently disabled an entire routerThe database is connected but the table has no rows
A backend refactor renamed a field and the frontend still reads the old name
An endpoint is missing its auth guard, or leaks data without a token
qa-probe builds a source-aware map from frontend calls to live backend responses, then explains each failure in one line with a fix hint. It complements Playwright (user journeys) and Schemathesis (contract fuzzing) - it is the layer that answers "why is this page blank?" first.
Related MCP server: ProxyPI
Quick start
1. Install
npx qa-probe run # use without installing
# or
npm install -g qa-probe2. Create a config
qa-probe.config.js in your project root:
module.exports = {
baseUrl: 'http://localhost:8000', // your backend
frontendSrc: './frontend/src', // your React source
routerFile: './frontend/src/App.tsx', // your router file
auth: {
type: 'bearer',
loginUrl: '/auth/login',
credentials: {
username: process.env.QA_USER, // never hardcode credentials
password: process.env.QA_PASS,
},
tokenPath: 'access_token',
},
};3. Run it
QA_USER=tester QA_PASS=secret npx qa-probe runqa-probe will parse your frontend, fetch /openapi.json, probe every endpoint with
real auth, and write a full report to .qaprobe/ (report.md, report.json,
ai-context.md, report.html). A summary table prints to the terminal.
Understanding the report
Route status
Status | Meaning |
healthy | endpoints respond, data present, schema valid |
degraded | works but has an issue (empty data, slow response) |
broken | unreachable, 404, 5xx, or a security/logic failure |
Common root causes
Root cause | What happened | What to do |
| router not registered (a | enable the flag and restart |
| a similar route exists (slash/casing) | align the frontend path |
| 404, not in the OpenAPI spec | fix the typo or |
| 200 OK but empty array | seed the database |
| 428 - terms/onboarding/MFA gate not satisfied | clear the gate for the probe account |
| 401/403 - wrong role/scope | use a user with the required permissions |
| a field was renamed/removed vs the spec | align frontend, backend, and spec |
| endpoint returns 200 without auth | add the missing auth guard |
| response violated a declared invariant | fix the response or the assertion |
| 5xx | check backend logs |
Every result is verifiable. Each diagnosis carries its evidence (the request,
a snapshot of the response, timing) and a confidence level - high, medium, or
none. A confidence: none / unknown result is explicitly not a confirmed pass.
Score: each route is 0-100; the overall score is the average. Use
--fail-under 80 to make CI fail when the score drops.
Use cases
"Why is this page blank?" - the everyday case. Get a root cause and a fix hint in one run instead of digging through network tabs and logs.
CI gate -
qa-probe run --fail-under 80blocks a deploy when pages regress.Demo / staging sign-off - confirm an environment actually works before a customer call, not just that it returns 200s.
Catch contract drift - a backend refactor renames a field or moves a route; qa-probe flags the exact endpoint and frontend call affected.
Security smoke - find endpoints missing an auth guard or leaking PII (opt-in, read-only, see Security checks).
Logic checks - assert response invariants (valid enum values, non-negative counts, required fields) without writing a test suite.
AI-assisted debugging - let Claude, Cursor, or Codex query the QA state and explain failures in plain English (see below).
Use it with an AI assistant (Claude Code, Cursor, Codex)
qa-probe ships a Model Context Protocol (MCP) server. Point your AI assistant at it and the assistant can read your QA data and explain failures directly - no manual report-reading.
Setup
Run qa-probe once to populate the cache:
npx qa-probe runThen add the MCP server. For Claude Code / Cursor, add to .mcp.json at your
repo root:
{
"mcpServers": {
"qa-probe": {
"command": "npx",
"args": ["qa-probe", "mcp"]
}
}
}Restart the assistant. Now you can ask in plain English:
"Why is /reports showing no data?"
"Which pages break if the users endpoint goes down?"
"Show me all broken routes."
"Mark the empty /alerts result as expected." (records feedback, reapplied next run)
Available MCP tools
Tool | Ask it |
| "Show me all broken routes and their scores" |
| "Why is /reports showing no data?" |
| "Is GET /users returning data right now?" |
| "Which backend routes does /dashboard call?" |
| "What pages break if the users endpoint goes down?" |
| "What should I do about feature_flag_disabled?" |
| "Run a full QA check and give me the summary" |
| "Mark this finding as expected / a real bug" |
| "I changed Alerts.tsx, does the app still work?" |
| "What does /alerts depend on, and how do we know?" |
MCP output is sanitized - SQL errors, stack traces, and table names are redacted
before they reach the assistant. Raw data stays on disk. Results carry their
evidence and confidence, and explain_failure returns a plain-English trust note
when a finding is unverified, so the assistant does not report guesses as passes.
You can also simply open the qa-probe folder (or your project with .qaprobe/) in
an AI coding tool and ask it to read ai-context.md - a compact, LLM-oriented summary
written on every run.
CLI commands
qa-probe run [--config <path>] [--fail-under <N>] analyze + probe + report
qa-probe analyze [--config <path>] build the dependency graph
qa-probe probe [--config <path>] probe all endpoints
qa-probe report [--config <path>] [--fail-under <N>] score, classify, write reports
qa-probe label <endpoint> <verdict> [-r <reason>] record feedback (reapplied later)
qa-probe fix [--config <path>] [--apply] [--pr] generate remediation diffs
qa-probe mcp [--config <path>] start the MCP server (stdio)
qa-probe doctor [--config <path>] check setup before you run anythingqa-probe doctor checks the Node version, that the config loads, that credentials are
present (it never prints them), that the output folder is writable, and that the backend and
OpenAPI spec answer. If the MCP server cannot start because of a bad or incomplete config, it
now stays up and reports the reason through its setup_status tool, so an MCP client shows the
real problem instead of only "connection closed".
--fail-under <N> exits non-zero when the overall score is below N (a CI gate).
Capabilities
Source-aware mapping - Babel parses your frontend; detects axios/fetch calls and common hook patterns; extracts routes from React Router (JSX and
createBrowserRouter) and TanStack Router.Live probing with real auth - bearer, cookie, api-key, or none. Concurrency-limited, with a hard per-request deadline so a streaming endpoint can never hang the run.
Root-cause classification - around 25 causes, each with a fix hint.
ID chaining - probes collections first, harvests a real id, then probes detail routes with it instead of a guessed value (kills unseeded-DB noise). On by default.
Adaptive baselines - learns each endpoint's normal latency and row counts from recent runs and flags deviation from itself. No model; fully inspectable.
Feedback -
qa-probe label(or the MCP tool) records a verdict that is reapplied on future runs. Scoped labels auto-revoke if behavior changes, so they cannot hide a regression.Security checks (opt-in) - anonymous auth-bypass re-probe, PII scan, and a per-persona privilege-escalation matrix. GET-only; never issues a write.
Response assertions - declare invariants (
in,type,gte,pattern,present,minItems, ...) and qa-probe verifies them on every 2xx response.Write-flows (opt-in, mutating) - full create/read/update/delete chains with guaranteed cleanup. Off by default; for a disposable / test-tenant environment only.
SSE and WebSocket checks, schema-drift detection, blast radius, regression diff, coverage, and HAR import for dynamic frontends.
OpenAPI auto-discovery - tries common spec paths and supports a local spec file, so fewer apps fall back to headless mode.
Configuration
The example above is enough to start. Key options (all optional unless noted):
module.exports = {
baseUrl: 'http://localhost:8000', // required
frontendApiPrefix: '/api', // stripped when matching calls to routes
framework: 'fastapi', // fastapi | express | nextjs | graphql | trpc | generic
openApiUrl: '/openapi.json', // auto-discovers fallbacks if missing
openApiFile: null, // or load the spec from a local file
frontendSrc: './frontend/src',
routerFile: './frontend/src/App.tsx',
auth: { type: 'bearer', loginUrl: '/auth/login',
credentials: { username: process.env.QA_USER, password: process.env.QA_PASS },
tokenPath: 'access_token' },
probe: {
concurrency: 5,
timeoutMs: 10000,
idDiscovery: true, // ID chaining (on by default)
safePosts: [], // POST endpoints that are safe reads
pathParamValues: { id: '1' }, // fallback values for {id} etc.
ignoreHTTPSErrors: false, // dev/staging only
},
// Opt-in extras:
security: { enabled: true, piiAllow: ['email'] },
assertions: { 'GET /alerts': [{ field: 'items[].severity', in: ['low','high','critical'] }] },
writeFlows: { enabled: false, flows: [] },
analyze: { har: { enabled: false, harFile: './traffic.har' } },
output: { dir: '.qaprobe', formats: ['json', 'markdown', 'ai-context', 'html'] },
seedCommand: 'npm run db:seed', // shown in the empty_db fix hint
};qa-probe.config.js is executed as JavaScript (like ESLint or Jest configs). Never
hardcode credentials - use environment variables.
How it works
Three phases, each cached so you can re-run them independently.
Analyze - parse the frontend AST, extract routes, fetch the OpenAPI spec (or import a HAR), and build a dependency graph with blast radius.
Probe - authenticate once, then probe every discovered endpoint live: HTTP, SSE, WebSocket, with hard deadlines and ID chaining.
Report - classify root causes, apply baselines and feedback, score routes 0-100, and write JSON / Markdown / ai-context / HTML reports.
Check your own change (verify)
After an AI assistant, or you, edits code, qa-probe verify answers one question with
evidence: does the running system still work?
qa-probe verify # files changed in git
qa-probe verify --files frontend/src/components/Alerts.tsx,backend/app/routers/alerts.py
qa-probe verify --all # every known endpointIt maps each changed file to the pages and endpoints it affects (an API call in a frontend file, a page component, or routes declared in a FastAPI or Express file), live-probes those endpoints, and returns:
pass: at least one affected endpoint was verified healthy and none is broken.
fail: something it probed live is broken. This is the only result that exits non-zero.
inconclusive: nothing affected could be verified. It never reports a pass it has not earned.
It is honest about what it could not check: write endpoints are never exercised (it does not change your data), an endpoint that returns 200 with no data is "not verified" (working and empty look the same), and a 403 for the test account or a long-lived stream is "not verified", not a failure. Files that map to nothing are listed, not guessed at.
The MCP tool qa_probe_verify_change does the same for an AI assistant, and
qa_probe_get_claims lists what each page depends on and how strong the evidence is:
runtime-observed (seen in real traffic) versus static-only (found in source, never seen
running), with the latest live status. Pair it with a code-intelligence tool: that tells you
what a change could affect, this checks that the running system still works.
Try it in two minutes
A tiny runnable demo lives in examples/demo-app:
cd examples/demo-app
node server.js & # demo backend, no dependencies
npx qa-probe doctor
npx qa-probe run
DEMO_BREAK=1 node server.js # restart it broken, then:
npx qa-probe verify --files frontend/src/components/Alerts.tsx # FAIL, with the evidencevs other tools
Capability | qa-probe | Schemathesis | Playwright | Postman |
Frontend to backend route mapping | yes | no | no | no |
Root-cause labels with fix hints | yes | no | no | no |
Live probe with real auth | yes | yes | yes | yes |
Property-based contract fuzzing | no | yes | no | no |
User-journey E2E | no | no | yes | no |
MCP server for AI assistants | yes | no | no | no |
CI exit-code gate | yes | yes | yes | yes |
Use Schemathesis for thorough contract fuzzing and Playwright for user journeys. qa-probe maps and explains; reach for the others to go deeper on what it surfaces.
Security and safe use
Credentials are held in memory for the run and never written to any report.
Config is executed as JavaScript - only run qa-probe with a config you trust.
ignoreHTTPSErrors: truedisables TLS verification; dev/staging only.Security re-probing is GET-only and never writes. Write-flows are off by default and clean up after themselves; use them only against a disposable environment.
To report a vulnerability, see SECURITY.md.
Known limitations
Detects visible HTTP calls, common hook patterns and generated SDK clients (hey-api and openapi-typescript-codegen styles). GraphQL clients, tRPC routers and custom service layers may need an adapter - or use HAR import to discover them from real traffic.
Pages are found from a
<Route>tree, TanStack Router file routes, or the Next.js App and Pages routers. Other file-based routers (Remix, Nuxt, SvelteKit, Astro) are not read yet.Calls made in shared components are attributed to every page that imports them, by following imports from each page file. Anything still unattributed is listed, not guessed.
Only calls recorded from real traffic (
--har,--runtime) areruntime-observed; everything found in source isstatic-onlyand says so.The probe checks responses against the spec. It does not read which fields your frontend code expects, so a frontend that reads
displayNamewhile the spec saysdisplay_nameis not caught until the spec and the real response disagree.If discovery finds 0 routes or 0 API calls,
analyzewarns that the result is not a clean bill of health. Record real traffic and import it (analyze.har) for any stack.Dynamic URLs built by arbitrary factory functions are not detected.
verifymaps backend files by the routes they declare (FastAPI decorators, Expressrouter.get(...)). Routes registered another way, or business logic in files that declare no routes, will show as "not mapped" rather than being guessed at.It checks that endpoints respond healthily. It does not check that a page renders them correctly; use a browser tool such as Playwright for that.
Best results need an OpenAPI spec; headless mode loses the spec-dependent rules but still detects observed-shape drift.
Contributing
See CONTRIBUTING.md. Contributions require a
Developer Certificate of Origin sign-off
(git commit -s).
Branding and trademarks
"qa-probe", "LightShield", and "LS-SIEM" are trademarks of LS-SIEM LLP. The Apache License does not grant permission to use these names or logos.
License
Copyright (c) 2026 LS-SIEM LLP - created and maintained by the LightShield SIEM team.
Licensed under the Apache License, Version 2.0 - see LICENSE and NOTICE. You may use, modify, and redistribute qa-probe under the Apache-2.0 terms; copyright and the trademarks above remain with LS-SIEM LLP.
Available Tools
8 toolsqa_probe_explain_failureA
Explain in plain English why a specific frontend route is showing no data or failing. Ask: "Why is /rules showing no data?"
Inputs: route (required).
qa_probe_get_blast_radiusA
Find out how many frontend routes would break if a backend endpoint goes down. Ask: "What pages break if GET /alerts goes down?"
Inputs: endpoint (required).
qa_probe_get_graphA
Get the dependency graph showing which backend API routes each frontend route calls. Ask: "Which backend routes does /dashboard call?"
Inputs: route.
qa_probe_get_reportA
Get the QA report with route scores and root causes. Ask: "Show me all broken routes" or "What is the overall score?"
Inputs: filter.
qa_probe_labelA
Record feedback on an endpoint diagnosis so qa-probe reapplies it on future runs. Use this to teach the tool: mark a result as expected/known (suppresses the finding) or confirm it as a real bug. Example: "mark the empty GET /alerts result as expected".
Inputs: verdict (required), endpoint (required), reason, signal.
qa_probe_probe_endpointA
Live-probe a single endpoint right now and return its status. Ask: "Is GET /alerts returning data right now?"
Inputs: path (required), method.
qa_probe_run_analysisA
Trigger a full qa-probe pipeline (analyze + probe + report) and return a summary. Ask: "Run a full QA check and give me the summary."
Inputs: headless.
qa_probe_suggest_fixA
Get a step-by-step fix guide for a root cause category. Ask: "What do I do about feature_flag_disabled issues?"
Inputs: rootCause (required).
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 8 tools
Each tool targets a distinct aspect of QA probing: explaining failures, impact analysis, dependency graphs, reports, feedback labeling, live probing, full analysis runs, and fix suggestions. No two tools overlap in purpose.
All tools share the 'qa_probe_' prefix and follow a consistent verb_noun pattern in snake_case (e.g., explain_failure, get_blast_radius). The minor deviation of 'label' as a single verb is still semantically clear and consistent with the pattern.
With 8 tools, the server is well-scoped for its domain—covering analysis, probing, reporting, labeling, and fixes—without being too sparse or overwhelming.
The tools cover the full workflow: triggering analysis, probing live endpoints, retrieving reports and graphs, explaining failures, assessing impact, suggesting fixes, and recording feedback. No obvious gaps exist for the intended use case.
Maintenance
Related MCP Connectors
Free MCP debugging tools for AI agents and developers. Diagnose failed API and MCP tool calls, identify HTTP errors, validate tool arguments against supported JSON Schema rules, and get actionable troubleshooting recommendations.
MCP observability. Query live traffic, errors, duration, and alerts from your AI agent.
AI-callable tools for API mocking, testing, monitoring, security, and automation.
Build, validate, deploy — HTTP APIs, cron jobs, webhooks and MCP tools — from your AI client.
Related MCP Servers
- AlicenseAqualityBmaintenanceLocal-first production-readiness MCP server for AI-built apps. It runs read-only checks, produces an evidence-based readiness score, and guides fixes before launch.11224 npm7Apache 2.0
- AlicenseAqualityDmaintenanceSelf-healing API proxy MCP that sends REST requests, diagnoses failures using Claude, patches requests, and retries. Successful fixes are remembered for instant reuse.45 npmMIT
- AlicenseAqualityDmaintenanceAI-powered backend diagnostics MCP server that statically analyzes frontend and backend code to catch cross-boundary bugs, contract drift, missing validation, security gaps, and performance anti-patterns before deployment.413MIT
- FlicenseNot gradedqualityDmaintenanceA local-first MCP server that gives AI coding agents runtime visibility and AI-managed debug logging. It replaces blind print() debugging by turning runtime execution into causal chains, allowing agents to instantly locate bugs by finding missing .success events in Python and TypeScript code. Single binary with MCP, CLI, and HTTP interfaces.-