Skip to main content
Glama
Sign In
qai-consultant-mcp

qai-consultant-mcp

by gvasile29
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python

review_qa_document

Scores an existing test plan, test strategy, or test case list against a six-dimension ISTQB/IEEE-829 rubric, returning findings, dimension scores, and cited sources to write your own review narrative.

Instructions

Deterministically review an existing QA document (Test Plan, Test Strategy, or a test case list) against a six-dimension ISTQB/IEEE-829- grounded rubric (structure completeness, objectives & scope clarity, entry/exit criteria, traceability, measurability, risk coverage) — no LLM anywhere in this call path; write your own narrative from the returned findings. doc_type must be one of "auto", "test_plan", "test_strategy", "test_cases" — "auto" runs a cheap heading-keyword classifier and reports which type it assumed; an unrecognized value returns a structured {"error": "invalid_argument", ...} rather than raising. Documents under ~200 characters (after stripping this app's own AI-disclosure front matter/footer) return doc_type="insufficient_content" with overall_score=0 rather than an error, unless they carry an explicit QA signal (test plan/strategy/case wording, "Test N:" lines) and enough words to score — those get a (low) score with findings. In "auto" mode a weak but genuine QA document (a QA signal plus a core testing term) is scored as its most likely type instead of being rejected. Rejections carry a hint telling you to retry with an explicit doc_type. Each finding carries kb_citations resolved from the knowledge base for its citation queries — a finding with no resolvable source is returned with an empty kb_citations list rather than a fabricated one. Non-QA input returns status="not_a_qa_document" (doc_type equals the status, no score); oversize input returns status="input_too_large". Non- English input is still scored but score_reliable=false and language_warning explains the rubric is English-keyword based. detection_confidence says how sure the "auto" type guess was. Returns {doc_type, overall_score, dimension_scores, findings, stats, status, language, language_warning, score_reliable, detection_confidence, detection_note, hint, kb_version}.

Input Schema

NameRequiredDescriptionDefault
doc_typeNoauto
document_textYes

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv3.1.4

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With zero annotations the description carries the full burden and delivers: no LLM in the call path (determinism), structured errors instead of exceptions for invalid doc_type, short-document handling with overall_score=0, explicit QA-signal exception, non-QA and oversize statuses, non-English scoring with score_reliable=false, and no fabricated citations. Nearly every failure mode an agent could hit is disclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Core purpose and rubric are front-loaded before the edge-case detail, and nearly every sentence carries decision-relevant information. It is, however, a single dense block of considerable length with some hedging qualifiers that could be tightened into a short bulleted edge-case list.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema exists, yet the description enumerates the return contract ({doc_type, overall_score, dimension_scores, findings, stats, status, language, language_warning, score_reliable, detection_confidence, detection_note, hint, kb_version}). Combined with status semantics and citation-resolution behavior, nothing an agent needs to call this correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate and does: all four doc_type values are enumerated with behavior ('auto' runs a keyword classifier and reports its assumption; invalid values return a structured error). document_text constraints are also implied through the ~200-character threshold and front-matter/footer stripping rule.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource — deterministically review an existing QA document (Test Plan, Test Strategy, or test case list) — and names the exact rubric dimensions used. This is unmistakably distinct from siblings like estimate_qa_effort or analyze_test_results, which operate on different inputs and outputs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Clear context for use: reviewing an existing document, with the contract that the caller writes its own narrative from the returned findings. Doc_type selection guidance is explicit, including when 'auto' is appropriate. It stops short of naming sibling alternatives or stating when not to use this tool (e.g., versus retrieve_qa_knowledge for source lookup).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools