Skip to main content
Glama

assess_qa_maturity

Assess QA process maturity from free-text project descriptions; get TMMi process-area scores and EU AI Act readiness checks for AI systems.

Instructions

Deterministically assess QA process maturity from a free-text project/ process description (or a pasted existing Test Strategy/Risk Register) — no LLM anywhere in this call path; write your own narrative from the returned findings. Scores 10 TMMi process areas (Level 2 Managed + Level 3 Defined) and returns an indicative_tmmi_level (1-3 — NEVER higher; Levels 4-5 require quantitative evidence a text description cannot substantiate, per TMMi's own no-skip rule — always read the returned disclaimer and never claim a certified TMMi level yourself). When the description signals an AI/ML system, also scores 7 EU AI Act Articles 9-15 readiness checks (ai_act_relevant=true, ai_act_dimension_scores populated); otherwise that dimension is omitted entirely (ai_act_relevant=false, empty dict) rather than scored as a false gap. Descriptions under ~200 characters (after stripping this app's own AI-disclosure front matter/footer) return indicative_tmmi_level=0 with empty findings rather than an error (status="insufficient_content"). Input with essentially no software- testing/QA vocabulary (a recipe, meeting notes) returns status="not_a_qa_description" with indicative_tmmi_level=null, a message and no findings; a successful assessment has status="ok". Mentions of AI that are negated ("no machine learning is used", "AI-free") do not switch the EU AI Act dimension on. Each finding carries kb_citations resolved from the knowledge base for its citation queries — a finding with no resolvable source is returned with an empty kb_citations list rather than a fabricated one. When ai_act_relevant=true, ai_act_note carries a fixed caveat that risk-tier classification (whether the system is legally "high-risk") is a determination this tool does not make — the user/team must confirm that independently; ai_act_note is empty when ai_act_relevant=false. Returns {status, indicative_tmmi_level, tmmi_dimension_scores, ai_act_relevant, ai_act_dimension_scores, ai_act_note, findings, disclaimer, detection_caveat, language, language_warning, score_reliable, keyword_list_warning, stats, kb_version}. Detection is keyword-based and can under-score paraphrased practices (see detection_caveat); non-English input sets score_reliable=false.

Input Schema

NameRequiredDescriptionDefault
project_descriptionYes

Schema Changelog

Changes observed during successful MCP inspections.

  1. Addedv3.5.2

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does so thoroughly: it discloses determinism/no-LLM, the indicative_tmmi_level cap at 3 with the TMMi no-skip rationale, the branch behavior for AI/ML mentions including negated ones, keyword-detection limits, and non-English handling. The three distinct status outcomes (ok, insufficient_content, not_a_qa_description) are spelled out precisely.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core purpose is front-loaded and every sentence carries substantive information about a non-obvious edge case, so the length is defensible. The structure is a single dense paragraph with heavy parenthetical nesting, which makes the branch conditions harder to scan than they need to be.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite having no output schema, the description enumerates the full return shape (status, indicative_tmmi_level, both dimension score dicts, notes, findings, disclaimer, caveats, stats, kb_version) and explains the semantics of the conditional fields. An agent has everything needed to call it and interpret the result without guessing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0% and the single parameter has no schema description, so the description must compensate — and it does: it defines what project_description should contain, the ~200-character minimum after stripping AI-disclosure boilerplate, the vocabulary requirement that separates a valid input from a recipe or meeting notes, and the language constraint that flips score_reliable.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description leads with a specific verb+resource: deterministically assess QA process maturity from a free-text project/process description, naming the exact scope (10 TMMi process areas, Level 2 Managed + Level 3 Defined). It explicitly distinguishes itself from the sibling tools by framing the output as findings the caller must narrate themselves, versus review/estimate tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It clearly states the valid input context (a free-text project/process description or a pasted Test Strategy/Risk Register) and the conditions under which it does not apply (under ~200 characters, no QA vocabulary). It does not, however, explicitly compare itself to siblings like review_qa_document or estimate_qa_effort, leaving the agent to infer which to choose.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.