1. Click on "Install Server". 2. Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state. 3. In the chat, type @ followed by the MCP server name and your instructions, e.g., "@qa-mcp run the full QA suite and diff against last run" That's it! The server will respond to your query, and you can continue using it as needed. Here is a step-by-step guide with screenshots.

qa-mcp

by shahinnr

Overview Schema Related Servers Score Discussions

TypeScript

Local

qa-mcp

Automated QA for a .NET microservice backend (several independent services, each with its own Swagger/OpenAPI document) and a React frontend, running against stage, exposed to an MCP client over stdio.

The one architectural rule

MCP is a thin layer on top of the test engine. The engine does not depend on the model.

Tests are never generated by an LLM at call time. They are either derived deterministically from OpenAPI, or written by hand as YAML flows / Playwright specs. That is what makes two consecutive runs comparable — and comparability is the entire value of the tool. If every run produced a different set of assertions, qa_diff could not tell a regression from noise.

Consequence: everything under src/api, src/flows, src/ui and src/report runs perfectly well from CI without an MCP client. src/index.ts only registers tools and shrinks the output.

Related MCP server: AutoDev MCP

Install

npm install
npm run build

Requires Node ≥ 20. npm install also pulls Playwright; download the browser once with:

npx playwright install chromium

Configure

qa.config.json is the single source of truth. Its path comes from QA_MCP_CONFIG, defaulting to ./qa.config.json. Start from the example:

cp qa.config.example.json qa.config.json

Section	What it does
`environment`	Free-form label stored in every run report.
`services[]`	`{ name, baseUrl, openapi, tags[], headers{} }`. `openapi` is either an absolute URL or a path relative to `baseUrl`.
`frontend`	`baseUrl`, `loginPath`, `loginSelectors{username,password,submit,successUrlPattern}`, `crawl{maxPages,ignore[],viewport}`.
`auth`	`tokenUrl`, `grantType`, `clientId`, `clientSecret`, `tokenField`, and one entry per role under `roles`.
`policy`	`readOnly`, `allowedMutationPaths[]`, `excludePaths[]`, `maxResponseMs`, `concurrency`, `requestTimeoutMs`.
`samples`	Real values used to fill required parameters, keyed by parameter name.
`artifactsDir`	Where runs, HTML reports and screenshots are written. Default `.qa-runs`.

Secrets never live in the config

Any string starting with env: is read from the environment; if the variable is unset, startup fails with an explicit message naming the field and the variable.

"password": "env:QA_ADMIN_PASSWORD"

export QA_ADMIN_PASSWORD=...
export QA_CUSTOMER_PASSWORD=...
export QA_CLIENT_SECRET=...

Register the server in an MCP client

command + args, transport stdio:

{
  "mcpServers": {
    "qa-mcp": {
      "command": "node",
      "args": ["G:/mcp/qatester/dist/index.js"],
      "env": {
        "QA_MCP_CONFIG": "G:/mcp/qatester/qa.config.json",
        "QA_ADMIN_PASSWORD": "...",
        "QA_CUSTOMER_PASSWORD": "...",
        "QA_CLIENT_SECRET": "..."
      }
    }
  }
}

Claude Code CLI equivalent:

claude mcp add qa-mcp -- node G:/mcp/qatester/dist/index.js

Verify the handshake at any time without touching stage:

npm run smoke

Tools

Tool	Input	What it does
`qa_discover`	`services?`	Fetches every Swagger document, reports endpoints per service, planned assertions, operations skipped by policy with the reason, and services that could not be reached. Sends no request to the endpoints.
`qa_run_api`	`services?`, `role?`, `pathContains?`	Generates and runs the API matrix. Mutating endpoints are skipped while `policy.readOnly` is on.
`qa_run_flows`	`names?`, `flowsDir?`	Runs `tests/flows/*.yaml` sequentially.
`qa_run_ui`	`mode: specs\|crawl\|both`, `role?`, `maxPages?`, `startPaths?`, `grep?`	Playwright specs and/or the automatic crawl.
`qa_run_all`	`role?`, `services?`, `skipUi?`, `maxPages?`	Full pass, then an automatic diff against the previous stored run. Release gate.
`qa_report`	`runId?`, `status?`, `severity?`, `suite?`, `service?`, `limit`	Reads a stored run with filters. Use it when a digest says results were truncated.
`qa_diff`	`baselineRunId?`, `currentRunId?`	Compares two runs by stable test id.

Every run tool returns a digest, not the raw results: the summary, failures grouped by service, at most 40 failures sorted critical-first, how many were truncated, and the runId to drill into with qa_report. A full pass over several services produces thousands of assertions; returning all of them would bury the signal.

What gets tested

Generated API tests

Per operation, from the OpenAPI document:

Check	Assertion	Severity
`contract`	Valid request → status must be declared in the spec and the body must validate against that status' schema	major
`authz`	Secured operation called with no token → anything other than 401/403 is a leak	critical
`robustness`	Malformed value in a required typed parameter → expect 400/404/422	major
`perf`	Response time over `policy.maxResponseMs`	minor

Any 5xx is an unconditional critical failure, including when the input was deliberately broken. A well-behaved service answers 400, not 500. On a .NET backend this single rule is the highest-yield automated check there is — it drags unhandled exceptions straight out of the framework.

Parameter values come from config.samples (by parameter name) first, then enum[0], default, example, then a format/type heuristic (uuid → zero-uuid, date → today, integer → 1, …). Malformed values match the type: uuid → not-a-uuid, numeric → not-a-number, date → 31-31-9999.

Each test carries a stable id — service:METHOD:/path#check — with no timestamp and no random value in it. qa_diff depends entirely on that.

One valid request produces both the contract and perf results, so an operation costs at most three HTTP calls (valid, unauthenticated, malformed).

Safety model — read this before pointing it at anything

Running generated tests against stage can destroy data or fire notifications at real users.

With policy.readOnly: true (the default) only GET/HEAD/OPTIONS are executed. Every mutating verb is refused unless its path starts with one of policy.allowedMutationPaths.
policy.excludePaths is honoured always, including when readOnly is off.
Every refused operation is reported with its reason by qa_discover, so you always know what is not covered.
Destructive operations are only allowed inside YAML flows, where a human wrote the steps.

Cross-service flows

tests/flows/*.yaml. Real microservice bugs live here, not in single-service tests: each service passes its own tests while the order id never reaches the notification consumer.

name: order-lifecycle
description: why this scenario matters
role: customer
severity: critical
steps:
  - name: create order
    service: orders          # must match a service name in qa.config.json
    method: POST
    path: /api/orders
    role: customer           # optional, overrides the flow role
    body: { productId: "{{productId}}" }
    expectStatus: [200, 201]
    expectBody: { status: Pending }   # dot-path -> expected value
    capture: { orderId: id }          # variable name -> dot-path in the response
    waitMs: 0                         # pause before the step, for eventual consistency

{{var}} interpolation works in path, body, headers and expectBody. Dot-paths understand array indexes (items[0].status). Flows run sequentially — they mutate shared state. One flow produces exactly one result: the business scenario either completes or it does not. On failure the evidence holds the failing step name, the trace of the steps that did succeed, and every captured variable. waitMs exists for the case where the target service only learns about the change through a message broker.

Frontend — two layers

Fixed specs (tests/ui/, standalone playwright.config.ts) are run as a child process with --reporter=json and normalised into results. QA_BASE_URL, QA_LOGIN_PATH and QA_LOGIN (credentials for the requested role) are passed in as environment variables, so the project also runs directly in CI. If playwright.config.ts is missing you get a skip with a clear message, not a crash. Put @critical or @minor in a test title to set its severity.

The three shipped specs assert business outcomes rather than "did the page load": a successful login, a list that must contain rows (an empty table where data is expected is the classic silent data-layer failure), and one that fakes a 500 with page.route to prove the UI shows the user an error instead of hanging on a spinner.

Automatic crawl covers the pages nobody wrote a spec for. It optionally logs in with a role; if that login fails it returns one critical result and stops immediately, because every later result would be meaningless without a session. It then BFS-crawls same-origin links up to maxPages, skipping crawl.ignore prefixes. A route fails on navigation error, HTTP ≥ 400, console errors, an XHR ≥ 400, or an effectively empty render. Console noise (React DevTools, HMR, React Router future flags, unused-preload warnings, favicon 404s) is filtered out — without that the output is unusable. Every failure gets a full-page screenshot whose path lands in the evidence.

Runs, reports and diff

Each run is written to artifactsDir as run-<timestamp>.json plus a readable .html next to it, with a severity-coloured failure table. Colons and dots in the timestamp are replaced with - (illegal in Windows filenames); the ids stay lexicographically sortable.

qa_diff compares two runs by stable id and splits the delta:

Bucket	Meaning
`regressions`	Passed before, fails now. The only bucket that should block a release.
`fixed`	Failed before, passes now.
`newFailures`	Test did not exist in the baseline and fails now (new endpoint, new page).
`stillFailing`	Already broken before.
`removed`	Was in the baseline, gone now (endpoint or page deleted).

When no baselineRunId is given, the baseline is the most recent earlier run covering the same suites — comparing an API-only run against a UI-only run would mark every test as removed and mean nothing.

Expect the first run to be noisy

This is normal and it is not a bug. The first pass against a real stage environment will report plenty of failures that are configuration, not defects: endpoints needing an id you did not provide, admin-only paths, export jobs, health probes.

Work through it in this order:

Run qa_discover and read what was skipped by policy — that is your coverage gap.
Run qa_run_api, then push obvious non-defects into policy.excludePaths.
Fill samples with real stage ids (orderId, customerId, productId, …) so path and required-query parameters resolve to rows that actually exist. Most 404 noise disappears here.
Raise maxResponseMs if stage is simply slower than production.
Re-run until what remains is genuinely interesting.
Keep that run as the baseline. From then on qa_diff / qa_run_all answer the only question that matters before a release: did anything that used to work stop working?

Layout

src/
  config.ts               zod config schema + env: resolution
  types.ts                TestResult / RunReport / summarize()
  discovery/openapi.ts    fetch + normalise specs, resolve $ref
  auth/session.ts         per-role token with cache
  api/generator.ts        test matrix per operation + safety gate
  api/runner.ts           bounded-concurrency execution + schema validation
  flows/runner.ts         cross-service YAML scenarios
  ui/crawler.ts           automatic Playwright crawl
  ui/specs.ts             runs Playwright specs, normalises the JSON report
  report/store.ts         store, diff, render HTML
  index.ts                MCP tool registration
tests/
  flows/*.yaml
  ui/                     playwright.config.ts + *.spec.ts

Notes for whoever extends this

stdio transport: never write to stdout. console.log corrupts JSON-RPC and the client disconnects with an opaque error. Every log goes to process.stderr (see log() in src/index.ts). npm run smoke fails if anything non-protocol reaches stdout.
ajv/ajv-formats are CommonJS. Under NodeNext the constructable sits on .default, and import type Ajv from "ajv" is unusable as a type — src/api/runner.ts shows the working pattern.
Relative ESM imports need the .js extension even from .ts sources.
Every tool handler is wrapped in try/catch and returns isError: true rather than throwing.
Every fetch carries AbortSignal.timeout(policy.requestTimeoutMs).
Uncompilable response schemas are treated as "no schema", not as an endpoint failure — the defect is in the spec, not in the service.
$ref resolution has both a depth cap and a per-branch seen-set; self-referencing DTOs are common in .NET and would otherwise hang the process.

Available Tools

7 tools

qa_diffCompare two runsA

Compares two stored runs by stable test id and splits the delta into regressions (passed before, fails now — the release blocker), fixed, newFailures (test did not exist in the baseline), stillFailing and removed (endpoint or page gone). Defaults to comparing the latest run against the most recent earlier run that covered the same suites.

ParametersJSON Schema

Name	Required	Description	Default
`currentRunId`	No	Current run id. Defaults to the latest run.
`baselineRunId`	No	Baseline run id. Defaults to the run before the current one.

TDQS

A4.4/5.0

Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Since no annotations are provided, the description carries the full burden. It discloses the matching key ('stable test id'), the delta categories, and the default run selection logic, adding behavioral context beyond the schema. It does not mention side effects, but this is a read-only comparison so it is not a major gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the primary action, and every clause adds value. It is compact without being under-specified.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity (two optional params, no output schema), the description covers the core functionality, output categories, and defaults. It could explicitly describe the return format, but the listed categories imply the response structure.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so baseline is 3. The description enriches parameter semantics by stating that currentRunId defaults to the latest run and baselineRunId defaults to the most recent earlier run covering the same suites, which the schema does not include.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb 'Compares' and resource 'two stored runs', and goes beyond by explaining the output categories (regressions, fixed, newFailures, etc.), which clearly distinguishes it from sibling tools like qa_run_* and qa_report.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context by calling regressions 'the release blocker' and specifies default behavior for run selection. It does not explicitly name alternative tools or exclusions, but the context and sibling tool names make the use case clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qa_discoverDiscover API surfaceA

Fetches the OpenAPI/Swagger document of every configured service and reports the testable surface: endpoint count per service, how many assertions would be executed, which operations are skipped by the safety policy (and why), and which services could not be reached. Runs no test and sends no request to the endpoints themselves. Use this first to check coverage before running anything.

ParametersJSON Schema

Name	Required	Description	Default
`services`	No	Service names from qa.config.json. Omit to discover every service.

TDQS

A4.7/5.0

Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Since no annotations are provided, the description carries the full burden. It discloses that it fetches OpenAPI/Swagger documents, never sends requests to the endpoint APIs themselves, reports on services that could not be reached, and explains safety-policy skips. This is a comprehensive behavioral profile.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the core action and ending with usage advice. It includes only relevant details and no filler, making it efficient and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description adequately summarizes the return values (endpoint count, assertion counts, skipped operations, unreachable services). It also includes safety behavior and usage context, making the description self-sufficient for an agent to understand the tool's outcomes.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already fully documents the single optional 'services' parameter with its description. The tool description adds no additional information about the parameter beyond the schema, so a baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies this as a discovery tool for the OpenAPI/Swagger documents of configured services. It specifies the verb 'fetches' and the resource, and reports a specific set of metrics (endpoint count, assertions, skipped operations, unreachable services). This distinguishes it from sibling tools that run tests or generate reports.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly instructs to 'Use this first to check coverage before running anything', which tells the agent when to invoke this tool relative to the test-running siblings. It also notes that it 'runs no test and sends no request to the endpoints themselves', clarifying that it is safe for pre-flight checks.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qa_reportRead a stored runA

Reads the full detail of a stored run with filters, for drilling into what a run digest truncated. Filter by status, severity, suite or service. Omit runId to read the most recent run.

ParametersJSON Schema

Name	Required	Description
`limit`	No	Max results to return (default 50).
`runId`	No	Run id. Omit for the latest run.
`suite`	No	Filter by suite.
`status`	No	Filter by result status.
`service`	No	Filter by service name.
`severity`	No	Filter by severity.

TDQS

A3.9/5.0

Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Without annotations, the description carries the transparency burden. It discloses that this is a read operation, supports filtering, and defaults to the most recent run when runId is omitted. However, it doesn't describe the return format, pagination behavior, or error handling, leaving some gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, front-loaded with purpose, then filters, then default behavior. No redundant words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the absence of an output schema and annotations, the description adequately communicates the tool's purpose and common usage, but it leaves out details about the response structure and how limit interacts with results, which would be valuable for a tool meant to drill into full details.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already provides descriptions for all six parameters (100% coverage), and the description's mention of filters adds no new meaning beyond the schema. The only additional note about omitting runId is already present in the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses the verb 'Reads' and identifies the resource as 'a stored run', specifying it provides 'full detail' in contrast to a truncated digest. This clearly distinguishes it from sibling tools that run or discover tests.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states the use case: 'for drilling into what a run digest truncated', and provides clear filter guidance. It doesn't explicitly mention alternatives, but the contrast with siblings is implicit; this is sufficient context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qa_run_allFull QA pass with regression diffA

One complete pass: discovery, generated API tests, YAML flows, Playwright specs and the frontend crawl — then an automatic comparison against the previous stored run. Returns the digest plus the diff buckets, of which regressions (passed before, fails now) is the one that should block a release. Mutating endpoints are still governed by policy.readOnly. Use this as the release gate.

ParametersJSON Schema

Name	Required	Description
`role`	No	Role used for API calls and UI login. Defaults to the first configured role.
`skipUi`	No	Skip both UI layers (useful when no browser is available).
`maxPages`	No	Crawl page cap for this run.
`services`	No	Limit the API part to these services.

TDQS

A4.2/5.0

Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of disclosure. It discloses the components included, the automatic comparison, the returned digest and diff buckets, and the policy constraint on mutating endpoints. It does not mention whether the run updates the stored baseline or the exact structure of the digest, but overall it provides solid transparency for a complex orchestration tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise and front-loaded, using two sentences to convey the core functionality and usage guidance. It avoids fluff, with every phrase earning its place—listing components, mentioning the diff, identifying the blocking bucket, and noting the policy constraint.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (an orchestrator running multiple QA layers) and no output schema, the description adequately explains what the tool does and what it returns (digest plus diff buckets). It highlights the critical 'regressions' bucket but does not detail the full diff bucket structure or potential side effects (e.g., updating the stored run). This is a minor gap, but the description is largely complete for an agent to understand and invoke the tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema description coverage is 100%, so parameters (role, skipUi, maxPages, services) are fully documented in the schema. The description adds some context by mentioning the components (e.g., UI layers, API part), which aligns with skipUi and services, but it does not add new semantic meaning beyond what the schema already provides. This is the baseline for high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states what the tool does: a complete QA pass covering discovery, generated API tests, YAML flows, Playwright specs, and frontend crawl, followed by a regression diff against the previous run. This is specific and distinguishes it from the sibling tools (which cover individual components like qa_discover, qa_run_api, etc.).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit usage guidance, stating 'Use this as the release gate.' It also implies when to use the full run versus the individual sibling tools (e.g., qa_run_api or qa_run_flows), though it doesn't explicitly state when not to use it. The mention of 'regressions' as the blocking bucket further signals its role in release decisions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qa_run_apiRun generated API testsA

Generates deterministic tests from OpenAPI and executes them against stage: contract (status declared in the spec + response body matches its schema), authz (secured endpoint called without a token must answer 401/403), robustness (malformed required parameter must answer 400/404/422), and perf (policy.maxResponseMs). Any 5xx is an unconditional critical failure. IMPORTANT: while policy.readOnly is true, every mutating endpoint (POST/PUT/PATCH/DELETE) is skipped unless its path is listed in policy.allowedMutationPaths — call qa_discover to see exactly what was skipped. Returns a digest; the full run is stored under artifactsDir.

ParametersJSON Schema

Name	Required	Description
`role`	No	Auth role from qa.config.json used for authenticated calls. Defaults to the first configured role.
`services`	No	Limit to these service names.
`pathContains`	No	Only test operations whose path contains this substring, e.g. "/api/orders".

TDQS

A4.5/5.0

Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full responsibility. It discloses deterministic behavior, exact test criteria, 5xx critical failures, mutation-skipping logic, return digest, and artifactsDir storage. This is comprehensive and leaves no major behavioral ambiguity.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but well-organized, starting with the main action, then test categories, a critical caveat (IMPORTANT), and ending with return/storage info. Every sentence adds unique value with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex tool with no output schema, the description is remarkably complete: it covers test types, failure conditions, special policy behavior, return digest, artifact location, and even points to a related tool. It leaves little to no ambiguity about what the tool does and what to expect.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema covers all three parameters with descriptions (role, services, pathContains). The tool description adds no parameter-specific detail beyond the schema, which already fully documents them, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it generates and executes deterministic API tests from OpenAPI, listing four test categories (contract, authz, robustness, perf). This specific verb+resource+scope distinguishes it from sibling tools like qa_run_ui and qa_run_flows.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use this tool (for API testing) and explicitly directs the agent to call qa_discover to see skipped mutations under a readOnly policy. It does not explicitly contrast with qa_run_all, but it provides strong contextual guidance on the workflow.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qa_run_flowsRun cross-service YAML scenariosA

Executes the hand-written business scenarios in tests/flows/*.yaml sequentially (they share state, so never in parallel). Each flow chains calls across services with {{variable}} interpolation, capture, expectStatus/expectBody and waitMs for eventual consistency. This is the only place where destructive operations are allowed, because a human wrote the steps. One flow = one result: on failure you get the failing step name, the trace of successful steps and every captured variable.

ParametersJSON Schema

Name	Required	Description	Default
`names`	No	Flow `name` values to run. Omit to run every flow.
`flowsDir`	No	Directory of YAML flows. Defaults to ./tests/flows.

TDQS

A4.5/5.0

Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full behavioral burden. It discloses sequential execution, state sharing, the ability to chain calls with interpolation/capture/expectations/waitMs, the fact that destructive operations are allowed only here, and the failure result format (failing step name, trace, captured variables).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences packed with essential information, no fluff. Front-loaded with the main action, then covers semantics, safety, and result format efficiently.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite no output schema, the description explains the result format on failure and implies success results. It covers execution mode, state sharing, destructive operation safety, and flow semantics. Adequate for a complex tool with no annotations.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so parameters are already documented in the schema. The description adds no extra parameter details, but it doesn't need to. Baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool 'Executes the hand-written business scenarios in tests/flows/*.yaml sequentially', which is a specific verb+resource. It also distinguishes itself from siblings by noting it is 'the only place where destructive operations are allowed' and emphasizes the cross-service flow chaining.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides strong context: flows share state and must never be run in parallel, and this is the only tool where destructive operations are allowed. It implies when to use it over alternatives but does not explicitly name sibling tools or state when not to use it beyond the parallel warning.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qa_run_uiRun frontend testsA

Two layers against the React frontend. mode=specs runs the fixed Playwright specs in tests/ui (skipped with a clear message if playwright.config.ts is missing). mode=crawl logs in with a role and BFS-crawls same-origin links, failing a route on navigation error, HTTP >= 400, console errors, failing XHRs or an effectively empty render, with a full-page screenshot for every failure. If the login itself fails the crawl stops immediately with one critical result. mode=both runs the two in order.

ParametersJSON Schema

Name	Required	Description
`grep`	No	Playwright --grep filter for the specs layer.
`mode`	Yes	Which UI layer to run.
`role`	No	Role used to log into the UI. Defaults to the first configured role.
`maxPages`	No	Crawl page cap. Defaults to frontend.crawl.maxPages.
`startPaths`	No	Crawl entry points, e.g. ["/", "/orders"]. Defaults to ["/"].

TDQS

A4.4/5.0

Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It discloses numerous behaviors: missing config handling, failure criteria (navigation error, HTTP >= 400, console errors, failing XHRs, empty render), screenshot on failure, and login failure handling. This is exceptionally transparent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single well-organized paragraph, front-loaded with 'Two layers against the React frontend.' Each sentence provides essential behavioral details without redundancy, making it appropriately concise for the complexity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The behavior is thoroughly covered for both modes, including error conditions and outcomes like screenshots. However, it never states what the tool returns upon completion (e.g., a test report or exit codes), which leaves a gap given no output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and every parameter has a description, so baseline is 3. The description adds detail for mode values (specs/crawl/both) but does not explain grep, role, maxPages, or startPaths beyond their schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it runs frontend tests with two modes (specs and crawl), distinguishing it from sibling API test tools like qa_run_api. The title 'Run frontend tests' matches the content, and the description elaborates on the exact scope against the React frontend.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

While it doesn't explicitly name alternatives, it establishes its domain as frontend testing ('against the React frontend') and explains when to use each mode (specs for fixed Playwright specs, crawl for link navigation). It does not mention when not to use it versus qa_run_api or qa_run_all, so some explicit guidance is missing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Install Server

RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

TDQS

A4.4/5.0

Disambiguation5/5

Each tool targets a distinct task: discovery, API tests, UI tests, flow tests, combined run, report reading, and diffing. There is no overlap in their purposes, and descriptions clearly delineate their roles.

Naming Consistency5/5

All tools use a consistent qa_ prefix followed by a clear verb (discover, run_flows, run_api, run_ui, run_all, report, diff). The pattern is uniform and predictable, making it easy to infer tool functions from names.

Tool Count5/5

Seven tools is well within the ideal range for a QA-focused server. Each tool covers a necessary step in the workflow without redundancy or bloat, making the set tightly scoped.

Completeness5/5

The tool surface covers the full QA lifecycle: discovering testable surface (qa_discover), executing different test types (flows, API, UI), running all together (qa_run_all), and then analyzing results via qa_report and qa_diff. There are no obvious dead ends or missing operations for the server's purpose.

Maintenance

ActivityMaintained

ResponsivenessSyncing

Resources

Need Help?

Related Servers

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

aiqaramba
io.github.ajrogier
Browser-based QA for AI-built software. Test pages with real browsers via agents.
Testing & QA Tools Browser Automation Agent Orchestration
2
Qlane
ai.qlane
AI QA that runs your app in a browser on every pull request: projects, test targets, test cases.
Testing & QA Tools CI/CD & DevOps Browser Automation
1
ProdPoke
io.github.prodpoke
AI QA tester — real browsers scan sites for bugs, SEO, perf, and accessibility issues via chat.
Testing & QA Tools Web Scraping Monitoring
21
Keploy A
io.github.keploy
End-to-end API testing — generate and run tests from OpenAPI, curl, Postman, or real user traffic.
API Testing Testing & QA Tools Developer Tools
7217,094

Related MCP Servers

Debugg AI MCPofficial
Browser Automation Testing & QA Tools
debugg-ai
A
license
A
quality
A
maintenance
Zero-Config, Fully AI-Managed End-to-End Testing for all code gen platforms.
Updated 13 days ago (2026-08-20 03:46 UTC)
8
639
68
Apache 2.0
AutoDev MCP
Testing & QA Tools CI/CD & DevOps Developer Tools
rookiejefren
F
license
Not graded
quality
D
maintenance
A dual-track testing server that combines CLI test execution with Playwright-based browser testing and persistent SQLite logging. It enables automated test pipelines, Git integration, and evidence-based requirement generation to streamline the development lifecycle.
Updated 7 months ago (2026-02-10 06:12 UTC)
Argus
Testing & QA Tools Browser Automation Autonomous Agents
chriswu727
A
license
A
quality
A
maintenance
AI-powered exploratory QA agent. Explores web apps like a real user — 18 MCP tools for clicking, filling forms, and navigating. Automatically verifies that actions persist (fake deletes, failed edits). Runs 16 detection types including dead links, SEO, accessibility, and performance checks.
Updated 24 days ago (2026-08-09 23:47 UTC)
29
2
MIT
qa-ai-agents
Testing & QA Tools Developer Tools
abchahal
F
license
Not graded
quality
B
maintenance
Enables automated QA testing by running a pipeline of AI agents that generate test scenarios, architect test layers, write Playwright tests, and review code, all grounded in feature requirements and API contracts.
Updated 2 months ago (2026-06-25 09:58 UTC)

Latest Blog Posts

Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)
By Om-Shree-0709 on July 25, 2026.
mcp
Agent Identity
OAuth 2.1
Your AI Chatbot Just Exposed Your CEO's Salary to an Intern
By Om-Shree-0709 on July 2, 2026.
Agent Identity
MCP Security
OAuth Delegation
Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)
By Om-Shree-0709 on June 30, 2026.
Agentic Ai
Prompt Injection
WebAssembly

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/shahinnr/mcp_qa'

If you have feedback or need assistance with the MCP directory API, please join our Discord server

qa-mcp

The one architectural rule

Install

Configure

Secrets never live in the config

Register the server in an MCP client

Tools

What gets tested

Generated API tests

Safety model — read this before pointing it at anything

Cross-service flows

Frontend — two layers

Runs, reports and diff

Expect the first run to be noisy

Layout

Notes for whoever extends this

Available Tools

Maintenance

Resources

Looking for Admin?

Related MCP Connectors

Related MCP Servers

Debugg AI MCPofficial

Latest Blog Posts

MCP directory API