Skip to main content
Glama
gabriel-tbc

qa-toolkit-mcp

by gabriel-tbc
OverviewSchemaRelated ServersScoreDiscussions
Python
Local

Server Quality Checklist

67%
Profile completionA complete profile improves this server's visibility in search results.

  • Latest release: v0.1.0

  • Disambiguation5/5

    The three tools have clearly distinct purposes: listing runs, getting a single run's details, and comparing two runs. There is no overlap or ambiguity.

    Naming Consistency5/5

    All tool names follow the consistent pattern 'qa_<verb>_runs' (list_runs, get_run, compare_runs), using snake_case and a clear verb_noun structure.

    Tool Count4/5

    Three tools is on the lower end but still reasonable for a focused test run analysis toolkit. The count matches the scope of listing, retrieving, and comparing runs.

    Completeness3/5

    The set covers the core operations of browsing and comparing test runs, but it lacks tools for creating, updating, or deleting runs, and flakiness detection is explicitly omitted. There are notable gaps for full lifecycle management.

  • Average 4.6/5 across 3 of 3 tools scored.

    See the Tool Scores section below for per-tool breakdowns.

    • No community issues in the last 6 months
    • 2 commits in the last 12 weeks
    • No stable releases found
    • No critical vulnerability alerts
    • No high-severity vulnerability alerts
    • No code scanning findings
    • CI status not available

  • This repository is licensed under MIT License.

  • This repository includes a README.md file.

  • No tool usage detected in the last 30 days. Usage tracking helps demonstrate server value.

    Tip: use the "Try in Browser" feature on the server page to seed initial usage.

  • Add a glama.json file to provide metadata about your server.

  • If you are the author, simply .

    If the server belongs to an organization, first add glama.json to the root of your repository:

    {
  "$schema": "https://glama.ai/mcp/schemas/server.json",
  "maintainers": [
    "your-github-username"
  ]
}

    Then . Browse examples.

  • Add related servers to improve discoverability.

How to sync the server with GitHub?

Servers are automatically synced at least once per day, but you can also sync manually at any time to instantly update the server profile.

To manually sync the server, click the "Sync Server" button in the MCP server admin interface.

How is the quality score calculated?

The overall quality score combines two components: Tool Definition Quality (70%) and Server Coherence (30%).

Tool Definition Quality measures how well each tool describes itself to AI agents. Every tool is scored 1–5 across six dimensions: Purpose Clarity (25%), Usage Guidelines (20%), Behavioral Transparency (20%), Parameter Semantics (15%), Conciseness & Structure (10%), and Contextual Completeness (10%). The server-level definition quality score is calculated as 60% mean TDQS + 40% minimum TDQS, so a single poorly described tool pulls the score down.

Server Coherence evaluates how well the tools work together as a set, scoring four dimensions equally: Disambiguation (can agents tell tools apart?), Naming Consistency, Tool Count Appropriateness, and Completeness (are there gaps in the tool surface?).

Tiers are derived from the overall score: A (≥3.5), B (≥3.0), C (≥2.0), D (≥1.0), F (<1.0). B and above is considered passing.

Tool Scores

  • Behavior4/5

    Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

    Annotations already declare readOnlyHint=true, destructiveHint=false, and idempotentHint=true, so the safety profile is clear. The description adds valuable behavioral context: returns metadata only, filter application order, sorting by started_at ascending, pagination via limit/offset, and return format options. No contradictions.

    Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

    Conciseness5/5

    Is the description appropriately sized, front-loaded, and free of redundancy?

    The description is concise at around 200 words, well-structured into logical paragraphs: purpose, clarification, filter order, return format, error handling. Every sentence serves a purpose with no redundancy.

    Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

    Completeness5/5

    Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

    Given the presence of an output schema (JSON shape provided), the description covers input, behavior, and output completely. It distinguishes from siblings, specifies error responses, and provides all necessary context for an agent to use the tool correctly.

    Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

    Parameters3/5

    Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

    Schema description coverage is 100%, so the schema documents all parameters. The description adds context about filter order (suite, since, until) and sorting, but does not significantly enhance meaning beyond what schema descriptions already provide. Baseline of 3 is appropriate.

    Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

    Purpose5/5

    Does the description clearly state what the tool does and how it differs from similar tools?

    The description clearly states the verb 'List' and resource 'available test runs', and specifies it returns metadata only. It explicitly distinguishes itself from the sibling tool `qa_get_run`, which provides full test case details. The purpose is unambiguous and well-scoped.

    Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

    Usage Guidelines4/5

    Does the description explain when to use this tool, when not to, or what alternatives exist?

    The description provides explicit when-to-use guidance: for listing run metadata, and directs users to `qa_get_run` for full test cases. It does not explicitly mention when not to use compared to `qa_compare_runs`, but that sibling has a distinct purpose, so the guidance is sufficient.

    Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

  • Behavior4/5

    Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

    Annotations already indicate readOnly, idempotent, and non-destructive behavior. The description adds details about error responses and default test inclusion, providing useful extra context beyond annotations.

    Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

    Conciseness5/5

    Is the description appropriately sized, front-loaded, and free of redundancy?

    The description is concise with three short paragraphs: purpose, parameter guidance, and return/error info. No unnecessary words, well-structured.

    Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

    Completeness5/5

    Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

    Given the simple retrieval nature, annotations, and full schema coverage, the description provides all necessary context including error response format and reference to output schema. Complete for the tool's complexity.

    Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

    Parameters4/5

    Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

    Schema description coverage is 100%, so baseline is 3. The description adds value by explaining the default behavior of include_passed and the return format options, which goes beyond the schema descriptions.

    Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

    Purpose5/5

    Does the description clearly state what the tool does and how it differs from similar tools?

    The description clearly states 'Return a single test run by id' with a specific verb and resource. It is easily distinguishable from sibling tools qa_list_runs and qa_compare_runs.

    Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

    Usage Guidelines4/5

    Does the description explain when to use this tool, when not to, or what alternatives exist?

    The description explains when to use include_passed and response_format parameters, but does not explicitly contrast with siblings. It provides clear context on default behaviors, which is helpful.

    Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

  • Behavior5/5

    Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

    Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false. The description adds detailed behavioral context: categorization logic with priority ordering, that error responses start with 'Error:', and the return formats. No contradiction with annotations.

    Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

    Conciseness5/5

    Is the description appropriately sized, front-loaded, and free of redundancy?

    Description is well-structured with bullet points and sections, each sentence adds value. It is concise yet comprehensive, front-loading the purpose and then detailing categories.

    Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

    Completeness5/5

    Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

    For a tool with moderate complexity (multi-category comparison), the description covers the full logic, return formats (Markdown/JSON), error handling, and even mentions what is not covered (flakiness). No output schema provided but the description explains what is returned.

    Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

    Parameters4/5

    Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

    Schema description coverage is 100%, so baseline is 3. The description adds context for run_a and run_b as baseline/newer, reinforcing schema statements, and explains how the parameters drive the categorization logic, which adds meaning beyond the schema.

    Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

    Purpose5/5

    Does the description clearly state what the tool does and how it differs from similar tools?

    The description clearly states it compares two test runs and categorizes differences, listing all categories. It distinguishes itself from sibling tools (qa_get_run, qa_list_runs) by focusing on comparison rather than retrieval or listing.

    Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

    Usage Guidelines5/5

    Does the description explain when to use this tool, when not to, or what alternatives exist?

    Explicitly explains that run_a is baseline (older) and run_b is newer, provides guidance on flakiness detection requiring N>2 runs and directs to an alternative (weekly_regression_review prompt). This covers when to use and when not to use.

    Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

GitHub Badge

Glama performs regular codebase and documentation scans to:

  • Confirm that the MCP server is working as expected.
  • Confirm that there are no obvious security issues.
  • Evaluate tool definition quality.

Our badge communicates server capabilities, safety, and installation instructions.

Card Badge

qa-toolkit-mcp MCP server

Copy to your README.md:

Score Badge

qa-toolkit-mcp MCP server

Copy to your README.md:

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/gabriel-tbc/qa-toolkit-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server