openqa-mcp

Server Quality Checklist

Profile completionA complete profile improves this server's visibility in search results.

Latest release: v0.6.0
Disambiguation4/5
Tools are generally distinct, targeting specific resources and actions. However, list_jobs and list_jobs_overview serve similar purposes with different output detail, and find_jobs_by_setting overlaps with list_jobs filtering, causing slight ambiguity.
Naming Consistency4/5
All tools use snake_case with a verb-first pattern (e.g., delete_job, list_machines). A few exceptions like find_jobs_by_setting and list_jobs_overview deviate slightly but remain consistent in style.
Tool Count3/5
40 tools is relatively high for an MCP server, indicating a broad scope. While each tool serves a specific function, the count may be overwhelming for simple use cases, though justified for a full-featured openQA client.
Completeness4/5
The tool surface covers major operations for jobs, groups, comments, assets, and scheduling. Minor gaps exist, such as missing update/delete for bugs and direct job creation, but core workflows are well-supported.
Average 2.8/5 across 38 of 40 tools scored. Lowest: 1.9/5.
See the Tool Scores section below for per-tool breakdowns.
- No community issues in the last 6 months
- 32 commits in the last 12 weeks
- Last stable release on July 20, 2026
- No critical vulnerability alerts
- No high-severity vulnerability alerts
- No code scanning findings
- CI is failing
This repository is licensed under GPL 2.0.
This repository includes a README.md file.
No tool usage detected in the last 30 days. Usage tracking helps demonstrate server value.
Tip: use the "Try in Browser" feature on the server page to seed initial usage.
Add a glama.json file to provide metadata about your server.
If you are the author, simply .
If the server belongs to an organization, first add glama.json to the root of your repository:
```
{
  "$schema": "https://glama.ai/mcp/schemas/server.json",
  "maintainers": [
    "your-github-username"
  ]
}
```
Then . Browse examples.
Add related servers to improve discoverability.

How to sync the server with GitHub?

Servers are automatically synced at least once per day, but you can also sync manually at any time to instantly update the server profile.

To manually sync the server, click the "Sync Server" button in the MCP server admin interface.

How is the quality score calculated?

The overall quality score combines two components: Tool Definition Quality (70%) and Server Coherence (30%).

Tool Definition Quality measures how well each tool describes itself to AI agents. Every tool is scored 1–5 across six dimensions: Purpose Clarity (25%), Usage Guidelines (20%), Behavioral Transparency (20%), Parameter Semantics (15%), Conciseness & Structure (10%), and Contextual Completeness (10%). The server-level definition quality score is calculated as 60% mean TDQS + 40% minimum TDQS, so a single poorly described tool pulls the score down.

Server Coherence evaluates how well the tools work together as a set, scoring four dimensions equally: Disambiguation (can agents tell tools apart?), Naming Consistency, Tool Count Appropriateness, and Completeness (are there gaps in the tool surface?).

Tiers are derived from the overall score: A (≥3.5), B (≥3.0), C (≥2.0), D (≥1.0), F (<1.0). B and above is considered passing.

Tool Scores

Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. 'Delete' implies a destructive operation, but no information is given about side effects (e.g., cascading deletes, error states, permission requirements). This is severely lacking for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise (two words), but this brevity sacrifices necessary detail. While conciseness is generally good, here it results in under-specification that hinders correct tool usage.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite the presence of an output schema, the description is incomplete. It omits behavioral context, parameter semantics, and usage guidance, making it insufficient for an agent to safely and correctly invoke the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%. The description fails to explain the singular parameter (job_id), its format, or how to acquire it. It adds no value beyond the schema's type definition.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Delete a job' clearly states the verb (Delete) and direct object (a job), distinguishing it from siblings like cancel_job or list_jobs. However, it lacks nuance about what 'delete' entails compared to 'cancel', which is a minor gap.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines1/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., cancel_job), prerequisites (e.g., job state), or scenarios to avoid. The presence of many sibling tools makes this omission critical.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It only states that it triggers scheduling, but does not disclose side effects, constraints, authorization requirements, or what happens to existing schedules. This is insufficient for an agent to understand the tool's impact.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is only one sentence, which is concise but at the expense of including essential details. While it is front-loaded, it does not earn its place as it misses critical information about parameters and behavior.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite the presence of an output schema, the description lacks context about the tool's behavior, preconditions, and effects. It is not complete enough to guide an agent on how to use the tool effectively, especially given the complexity of scheduling operations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 5 parameters with 0% description coverage (no parameter descriptions), and the tool description does not mention any parameter meaning, defaults, or allowed values. The description fails to compensate for the lack of parameter documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'trigger' and the resource 'ISO test scheduling' with the context 'for a product'. It effectively distinguishes this tool from sibling tools that perform other actions like listing or querying, avoiding tautology.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines1/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, no prerequisites, and no exclusions. It does not mention any context that would help an agent decide between this and similar tools such as 'get_iso_job_stats'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden, but it only says 'Duplicate (clone) a job.' It fails to disclose behavioral traits such as whether it creates a full copy, preserves dependencies, or is destructive. Score 1.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely short (5 words) but under-specifies to the point of being unhelpful. Not a model of conciseness; lacks necessary detail. Score 2.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 3 parameters, 0% description coverage, no usage guidance, and existence of an output schema not explained, the description is grossly incomplete for a mutation tool. Score 1.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description adds no meaning to parameters like job_id, prio, or dup_type_auto. The agent must infer from names alone. Score 1.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'duplicate' and the resource 'job' with a synonym 'clone'. However, it lacks differentiation from sibling tools like cancel_job or delete_job, earning a 4.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. No mention of prerequisites, permissions, or context for duplication. Score 2 for minimal implied usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose any behavioral traits such as idempotency, error conditions, side effects (e.g., job state changes), or rate limits. The description fails to inform the agent about potential consequences.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise (one sentence) but lacks necessary detail. It is under-specified, making it unhelpful despite its brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the three parameters (with zero coverage), no annotations, and an output schema that is not described, the description is severely incomplete. It does not explain what the tool returns or how parameters affect behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage and no additional parameter explanations in the description, the agent receives no semantic context for parameters like 'prio', 'force', or 'job_ids'. The description adds no value beyond the raw schema types.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('restart') and resource ('jobs') and specifies the bulk nature. It is specific enough to distinguish from single-job restart, though it could explicitly mention the difference from the sibling 'restart_jobs' tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like 'restart_jobs' or when not to use it. The description lacks context for appropriate usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description bears full burden. It only states 'List comments' implying read-only, but fails to disclose pagination, ordering, permissions, or any side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no fluff, but lacks structure. No front-loading of key details like return type or required parameters.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema present, description omits any reference to return values. For a 1-param simple tool, key context is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% and description does not explain the required group_id parameter. No information on its meaning, format, or valid values.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose3/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description states 'List comments on a job group,' which is a clear verb+resource. However, it does not differentiate from similar sibling tools like list_job_comments or list_parent_group_comments, leaving ambiguity about the exact scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. No mention of prerequisites, context, or when-not-to-use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description does not disclose behavioral traits such as immediate vs. async cancellation, reversibility, permissions required, or impact on running jobs. The verb 'Cancel' implies destruction but lacks detail.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single concise sentence, but too brief to convey necessary detail. Appropriate length for a simple purpose, but under-informative given parameter complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, the description lacks explanation of behavior (e.g., return value, confirmation, side effects). With 10 undocumented parameters and no annotations, the tool is underspecified.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description does not explain any of the 10 parameters (e.g., arch, test, build). Schema has zero description coverage, leaving the agent without meaning for each filter field.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Cancel all jobs matching the given filters' clearly states the action (cancel) and resource (all jobs) with a scope (matching filters). It distinguishes from sibling 'cancel_job' implicitly via plural vs singular, but lacks explicit differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like cancel_job (single job) or restart_jobs. No context about conditions for execution or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must convey behavioral traits. 'Delete' implies mutation and irreversibility, but no details are given about permissions, side effects, or confirmation. The agent cannot assess the impact of using this tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is too brief (four words) and under-specifies the tool's behavior. While it is front-loaded, it lacks critical detail that warrants additional sentences, making it insufficient rather than concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having a simple interface with two parameters and an output schema, the description omits any explanation of return values, error conditions, or parameter constraints. The agent is left with minimal context to correctly invoke the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has two required integer parameters with no descriptions, and the tool description offers no explanation of what job_id and comment_id represent or how they relate. Since schema description coverage is 0%, the description fails to add any meaning to the parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Delete' and the resource 'a job comment', which differentiates it from sibling tools like add_job_comment, update_job_comment, and get_job_comments. However, it lacks specificity such as indicating that the comment is identified by job_id and comment_id.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like update_job_comment or cancel_job. There is no mention of required prerequisites or context, leaving the agent to infer usage from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must convey behavioral traits. It only says 'Get' (implying read-only) and mentions 'result of a prior ISO trigger', but lacks details on auth needs, side effects, or response size.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, but it's not well-structured: no logical flow and omits key information. Conciseness is good but at the expense of clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With only one parameter and an output schema present, the description is still incomplete: it does not explain the parameter, usage context, or relationship to siblings. The tool is simple but could be more helpful.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the description does not mention the parameter 'scheduled_product_id' at all. The single required parameter is left entirely unexplained.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves a scheduled product resulting from a prior ISO trigger, distinguishing it from generic 'get_product' or sibling tools like 'get_job'. However, 'ISO trigger' is not explained and may be unclear to some agents.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus siblings like 'cancel_scheduled_product' or 'get_job'. Also, no prerequisites or context provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description bears full responsibility for behavioral disclosure. It fails to mention any behavioral traits such as pagination, ordering, access permissions, or whether comments include replies. The description is essentially a tautology of the tool name.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is overly brief, consisting of a single clause. While concise, it sacrifices informativeness; every word should add value, but here it simply restates the tool's purpose with no additional detail or structure.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity (1 required parameter, no annotations, output schema exists), the description should provide a minimal but complete context. It fails to explain what 'comments' entail, any default behavior, or how to interpret the output. The agent lacks sufficient information to use the tool reliably.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, yet the description adds no meaning for the 'parent_group_id' parameter. It doesn't explain what a parent group is, how to obtain its ID, or any constraints. The parameter is left entirely to the agent's inference.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'List' and the resource 'comments on a parent job group'. It distinctly identifies what the tool does and differentiates from siblings like 'list_group_comments' or 'add_parent_group_comment'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. No mention of prerequisites, typical use cases, or exclusions. The description is purely functional without contextual advice.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure. It merely says 'Restart' without explaining side effects, required job states, permissions, or the effect on running jobs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with only one sentence, but it sacrifices informativeness. It is not particularly well-structured for efficient parsing.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the existence of sibling tools like restart_jobs_bulk and an output schema, the description fails to explain how this tool differs, what the output contains, or the operational context. It is incomplete for effective tool selection.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The parameter job_ids is not described in the schema (0% coverage), and the description adds no meaning about its format or purpose beyond implying it lists jobs to restart.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (restart) and resource (jobs). It is specific enough to distinguish from sibling tools like cancel_jobs, though it does not elaborate on the exact meaning of restart.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like restart_jobs_bulk or cancel_jobs. No prerequisites or context for invocation are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description bears full responsibility for disclosing behavioral traits. It only states the basic action without mentioning side effects, permissions, error conditions, or idempotency. This is insufficient for an AI agent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise (one sentence), which is good for front-loading. However, it is too brief and lacks necessary details, making it minimally adequate rather than well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of a write operation and the existence of an output schema (details not provided), the description is incomplete. It does not explain what the tool returns or how to handle errors, leaving the AI agent with insufficient context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 2 parameters with 0% description coverage. The description does not elaborate on 'text' or 'job_id' beyond their names, leaving their semantics underspecified. For example, it's unclear if 'text' has length limits or format requirements.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Add a comment to a job' clearly states the action (add) and the resource (comment to a job). It differentiates from sibling tools like add_group_comment or add_parent_group_comment. However, it doesn't specify the parameters or context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like add_group_comment. The description does not mention exclusions or prerequisites for using this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must fully disclose behavioral traits. It only states 'cancel' implying destruction, but lacks details on permissions, reversibility, or what happens to associated resources. This is insufficient for safe usage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely brief (one sentence), which is concise but at the cost of clarity and completeness. It reads as a terse label rather than a helpful guide, missing important details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity (1 param, output schema exists), the description should cover basic usage and outcomes. However, it lacks any information about return values, error conditions, or prerequisites, leaving the agent underinformed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description mentions canceling 'by name' which corresponds to the single required parameter, but it does not explain what format the name should follow or where to obtain it. Since schema coverage is 0%, the description fails to add meaningful semantics beyond the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('cancel') and the resource ('scheduled product / ISO'), and it distinguishes from sibling tools like cancel_job, which target different entities. The mention of 'by name' indicates the primary input. However, the term 'ISO' is ambiguous without further context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., cancel_job), nor any prerequisites or side effects. The description simply states the action without context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description carries the full burden. It only mentions the scope of search but does not disclose behavior like read-only, pagination, query syntax, or result limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence. It is front-loaded but lacks details; however, it does not waste words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the context (1 required param, no annotations, output schema exists), the description is insufficient. It does not explain query syntax, result format, or other essential details for effective use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage. The description does not explain the 'q' parameter (e.g., format, operators, escaping), leaving the agent without guidance on how to construct queries.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it performs full-text search across specific resources (jobs, groups, test modules). The verb 'search' is appropriate and distinguishes from sibling tools that are CRUD operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. For instance, it could be used instead of specific list tools when querying across multiple types, but this is not mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description lacks any behavioral traits beyond the action itself. There is no mention of side effects, limitations, permissions, or whether the change is immediate or reversible. With no annotations provided, the description fails to disclose important behavioral aspects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
While the description is concise (one sentence), it is underspecified and lacks structure. It does not earn its place as it provides minimal helpful information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (mutation, integer parameters) and no output schema details, the description is incomplete. It does not explain return values, success conditions, or behavior for invalid inputs.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, yet the description adds no meaning to the parameters. It does not explain what 'prio' means (e.g., valid range, effect) or what 'job_id' refers to. The description is insufficient to aid agent understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Set the priority of a job.' uses a specific verb and resource, clearly indicating the action and target. It distinguishes from sibling tools as no other sibling explicitly sets job priority.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, such as when jobs can have priorities set, or any prerequisites. No exclusions or contexts are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided. The description simply says 'update', which is a mutation, but does not disclose any side effects, authorization requirements, or idempotency. For a mutation tool, this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is one short sentence with no waste. However, it is under-specifying; conciseness should not compromise completeness. It could benefit from additional context without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (3 required parameters, mutation), the description is too minimal. An output schema exists, so return values are covered, but important context like order of parameters or validation rules is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%. The description does not explain the meaning of parameters beyond their names. 'text' is the comment content, but no format or constraints are given. The description adds no value over the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Update an existing job comment' clearly indicates the verb and resource. It distinguishes from 'add_job_comment' and 'delete_job_comment' by specifying 'update existing', but does not differentiate from other update tools like 'set_job_priority'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool vs alternatives such as 'add_job_comment' or 'delete_job_comment'. There is no mention of prerequisites like requiring job_id and comment_id.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; the description only says 'Add a comment' implying mutation but no details on side effects, permissions, or return value. The presence of an output schema is not mentioned.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with one short sentence, but it sacrifices necessary information for brevity. It is not verbose but lacks detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With two required parameters, zero schema descriptions, and no behavioral or output details, the description is inadequate for proper tool usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not explain the meaning or constraints of the 'text' and 'group_id' parameters beyond their names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'Add' and resource 'comment to a job group', clearly distinguishing it from siblings like add_job_comment and add_parent_group_comment.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool over alternatives such as add_job_comment or add_parent_group_comment. The description lacks any contextual help.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure. It only says 'Create a tracked bug reference' without mentioning side effects, permissions, error cases, or return value. This is insufficient for an agent to understand the tool's effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise (one sentence), but it sacrifices necessary detail. It is front-loaded with the verb and resource, but lacks parameter information and behavioral context.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has two parameters and an output schema, the description is incomplete. It does not explain what the created bug reference looks like, what the output contains, or any constraints. A more complete description would address these gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description does not mention or explain any of the two parameters (bugid, title). Schema coverage is 0%, so the description should compensate but fails to do so. The agent gets no help understanding what values to provide.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Create') and the resource ('tracked bug reference'). However, 'tracked bug reference' is somewhat vague compared to simply 'bug', but it distinguishes the tool from listing bugs. The name 'create_bug' reinforces the purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives. However, the sibling tools list does not include any other bug creation tool, so usage is implied. The description lacks context such as prerequisites or scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden but fails to disclose behavioral traits such as ordering, pagination, authentication requirements, or read-only nature. It only states the basic action.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence with no redundancy. However, it is under-specified for the tool's purpose, bordering on insufficient rather than efficiently informative.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema (though not shown), the description does not hint at return structure or content. For a listing tool, details on pagination or format expectations are missing, making it incomplete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, yet the description provides no additional meaning for the 'job_id' parameter beyond its integer type. The agent gains no clarity on format, constraints, or semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'List comments on a job' uses a specific verb 'List' and resource 'comments', clearly identifying the tool's action and target. It distinguishes itself from siblings like add_job_comment (adds) and get_job (retrieves job details).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, required context (e.g., job existence), or exclusions. The description is only a statement of function without any usage hints.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description must fully disclose behavior. It lacks details on what 'aggregated' means, potential side effects (rare for a get), or data limitations. The tool returns results but the description does not explain pagination or performance implications.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The single-sentence description is very concise but too brief for a tool with 5 parameters. It front-loads the purpose but omits essential details, making it under-specified.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema (not shown) and 5 parameters, the description is insufficient. It does not explain the aggregation behavior, parameter usage, or any context about job groups. Without annotations, more depth is needed for a complete understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, meaning no parameters are described in the schema. The description adds no information about the 5 parameters, leaving their meaning entirely implicit from names like show_tags, limit_builds, etc.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's action (Get), resource (aggregated build results), and scope (for a job group), distinguishing it from siblings like get_job_group or list_jobs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as list_job_group_jobs or get_job_group. The description does not mention prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits beyond 'get'. It does not mention read-only nature, safety, or any side effects. The description is insufficient for a tool with zero annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, efficient but overly terse. It lacks substance needed for effective tool selection.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool (single required parameter, output schema exists), the description is adequate but minimal. It could benefit from specifying the return value or example usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has one parameter (group_id) with no description, and the tool description does not explain its meaning or format. Schema coverage is 0%, and the description fails to compensate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves a single parent job group, using a specific verb and resource. However, it does not differentiate from sibling tools like get_job_group or list_parent_groups.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, such as list_parent_groups. No prerequisites or exclusions mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description carries the full burden. It only states the action without any details on side effects, pagination, error behavior, or authentication needs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short (one sentence), which is concise but lacks informative content. It could be more detailed without being verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given a simple tool with one parameter and an output schema, the description does not explain the output structure or behavioral details like pagination. It is minimally complete but insufficient for effective agent usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The sole parameter 'group_id' is defined in the schema with type integer but the description adds no semantic meaning. Schema description coverage is 0%, and the description fails to compensate, e.g., by clarifying the group_id's role or format.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'List jobs belonging to a job group.' clearly states the verb (list) and resource (jobs), differentiating it from sibling tools like list_jobs (which likely lists all jobs) and get_job_group (which gets the group itself).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. There is no mention of prerequisites, limits, or contextual triggers.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. Description does not disclose side effects, reversibility, or dependencies of cancellation. For a destructive action, more context is needed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence with no redundant words. Efficient but lacks detail; conciseness is appropriate for the information provided.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, description does not mention return value or error conditions. Simple tool but incomplete context for effective use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 0% description coverage for parameter job_id (integer). Description adds no meaning beyond the schema; does not explain the parameter's role.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the verb 'Cancel' and the resource 'a running or scheduled job'. It distinguishes from sibling tools like cancel_jobs (plural) and restart_jobs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like cancel_jobs or restart_jobs. No prerequisites or conditions provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description must fully disclose behavioral traits. It omits any details about permissions, system effects, or response characteristics, leaving the agent blind.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise but borderline underspecified. It could benefit from a sentence clarifying the scope or use case without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The existence of an output schema reduces the need to describe return values, but the description still lacks contextual cues to differentiate from siblings and inform selection.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With zero parameters and 100% schema coverage, the description adds no parameter information but is not required to. The baseline for 0 params is 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists registered worker instances, but it does not differentiate from sibling tools like list_machines or list_jobs, which could cause confusion for the agent.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. The agent has no context to determine if list_workers is appropriate over similar list tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. It only states the basic operation, omitting error handling, idempotency, permissions, or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise, single sentence. Front-loaded with key information. However, it could include more useful details without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a retrieval tool with no annotations and low schema coverage, the description is too minimal. It doesn't address return values, error states, or asset semantics, even though an output schema exists.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must explain the parameter. It merely restates 'by id' without adding meaning, format, or constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly specifies the action (get), resource (asset), and identifier (by id). It distinguishes from the sibling list_assets tool, which retrieves multiple assets.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like list_assets. No context on prerequisites or scenarios where get_asset is preferred.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description only implies a read operation through the verb 'Get'. No annotations are provided, and the description does not disclose any behavioral traits such as required permissions, error conditions, or idempotency. The presence of an output schema is not referenced.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at 4 words, with no wasted text. However, it could be slightly more descriptive without losing efficiency, such as mentioning that the output is a job group object.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple get tool with one parameter and an output schema, the description is adequate but incomplete. It does not explain what a 'job group' is or how the returned data relates to other tools. More context would be helpful for the agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not add meaning beyond the input schema. The parameter 'group_id' is not described, so the agent must infer its purpose from the tool name alone. This fails to compensate for the low schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get a single job group' is specific, using a verb ('Get') and a resource ('single job group'). It clearly distinguishes from list tools like 'list_job_groups' and other get tools like 'get_parent_group'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. For example, it does not explain when to use 'get_job_group' instead of 'get_parent_group' or 'list_job_groups'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description only states the action without disclosing behavioral traits such as required permissions, idempotency, or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
One short sentence, no wasted words, but could be slightly expanded to include usage hints without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate for a simple write tool with output schema, but lacks usage guidelines and parameter details, making it minimally complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%; description adds no additional meaning beyond param names (parent_group_id, text) and does not explain constraints like text format or length.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description uses specific verb 'Add' and resource 'comment to a parent job group', clearly distinguishing from siblings like add_group_comment and add_job_comment.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool over siblings like add_group_comment or add_job_comment, nor any prerequisites or context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden, but it only states the basic purpose. No information about authentication, potential side effects, rate limits, or what exactly 'statistics' entails is given.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence with no redundant information. However, it could be slightly expanded to improve clarity without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the zero parameters and existence of an output schema, the description is minimally complete but lacks contextual details about the statistics provided. It does not leverage the schema or annotations to enrich understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, so the description naturally cannot add parameter semantics. The absence of parameters is clear from the schema, and the description does not mislead.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'job statistics for scheduled products', but it does not explicitly distinguish from siblings like 'get_job_status' which might also return statistics. However, the qualifier 'for scheduled products' adds specificity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool over alternatives or any conditions. There are many sibling tools for job retrieval, and without context, an agent cannot decide when to use this one.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description must disclose behaviors. It only restates the tool's name without detailing idempotency, authentication needs, rate limits, or what 'full details' entails. The agent learns nothing beyond the name.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence with no wasted words. However, it is slightly under-specified, just barely meeting the needs.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the output schema exists, return values need not be explained. However, with 12 siblings and no behavioral or usage guidance, the description is only minimally adequate for the tool's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, meaning no parameter details in the description. The only parameter job_id is implicit from its name, but the description adds no explanation of its role or format, failing to compensate for the schema gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Get' and the resource 'full details for a single job', which is specific and distinct from siblings like list_jobs or get_job_comments.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. Siblings such as list_jobs, find_jobs_by_setting, or get_job_comments exist, but no context for differentiation is given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description bears full burden. It implies a read-only list operation but omits details like pagination, sorting, or behavior with large datasets.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise (three words). Every word is necessary, but the description is perhaps too terse and could include more context without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given zero parameters and presence of output schema, the description is minimally adequate. However, it does not explain what constitutes a job group or if there are any special behaviors.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist; schema coverage is 100%. Description adds nothing beyond schema, but baseline for zero-parameter tools is 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb ('List') and resource ('job groups'), making purpose obvious. It distinguishes from siblings like 'list_jobs' and 'list_parent_groups' by resource name, though it does not explicitly differentiate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. Does not mention scenarios, prerequisites, or exclusions. With many sibling list tools, more context would help.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden but only states 'List', implying a read-only operation. No details about authentication, side effects, or output are given beyond what an output schema might provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise (4 words) and front-loaded. It is clear but could benefit from a bit more detail without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters and an output schema exists, the description is adequate for a simple list operation. However, it lacks context about the meaning of 'configured' and how this tool fits into the workflow among sibling tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are zero parameters, and schema coverage is 100%, so the description does not need to add parameter meaning. The baseline of 4 is appropriate as there is nothing to describe.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('List') and resource ('configured worker machines'), distinguishing it from sibling tools like list_jobs and list_products. However, it is brief and could specify what constitutes 'configured'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool, nor are any alternatives or prerequisites mentioned. The description offers no context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It states 'list' implying read-only, but does not disclose behavioral traits such as whether all test suites are returned, pagination, or access control. Minimal transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at four words, with no wasted text. It is front-loaded and clear.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity, the description is adequate but lacks context about the scope of 'configured' test suites and how this tool differs from sibling listing tools. The presence of an output schema partially compensates, but more context would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no parameters and schema coverage is 100%. The description adds no further meaning beyond the schema, which is acceptable for a no-parameter tool.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'List configured test suites' clearly states the action (list) and the resource (test suites). It is straightforward but does not differentiate from sibling tools that list other resources, though the resource name itself provides distinction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like list_jobs or list_machines. The description lacks context about the intended use case or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description only states what the tool does without disclosing behavioral traits like idempotency, error handling, or authorization requirements. Minimal transparency for a read operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no wasted words. Every part adds value, clearly conveying the tool's purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the existence of an output schema, the return details are covered. However, the description lacks context on error conditions or prerequisites, which is notable given the absence of annotations and sibling differentiation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description adds no information about the job_id parameter beyond its type and requirement. The agent must infer its meaning from context, which is insufficient.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it retrieves a single job with full test-module/step details, using a specific verb and resource. This distinguishes it from siblings like get_job or get_job_status.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when-to-use or alternatives are mentioned. The description implies it's for detailed job retrieval, but does not differentiate from similar tools like get_job or provide context on when to choose it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, and the description does not disclose behavioral traits like read-only nature, side effects, authentication needs, or rate limits. For a list tool, it likely is safe, but the description fails to confirm this.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with verb and resource, no unnecessary words. Extremely concise and easy to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is minimal. While output schema exists to explain return values, the description lacks context about what 'tracked bugs referenced by jobs' means, e.g., whether they are system-wide or job-specific. Adequate but could be improved.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, and the input schema is fully covered (empty). With 0 parameters, baseline is 4. The description does not need to add parameter meaning, but it also does not add any value beyond the schema. Still, it meets the baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description states verb 'List' and resource 'tracked bugs referenced by jobs.' It clearly indicates what the tool does and distinguishes from sibling tools like create_bug (creation) and list_jobs (jobs). However, it could be more specific about the scope of bugs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, such as create_bug or other list tools. No context about prerequisites or typical use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description only restates the name. It fails to disclose any behavioral traits such as pagination, sorting, or whether the list is exhaustive. The existence of an output schema is not communicated in the description.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no superfluous words. It is front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the absence of parameters and presence of an output schema, the description is minimally adequate. However, it could add context about the scope (e.g., 'top-level groups') or behavior (e.g., alphabetical order) to improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, and schema coverage is 100% trivially. Per guidelines, a baseline of 4 is appropriate since the description does not need to add parameter information.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and resource 'parent job groups', making the tool's purpose immediately understandable. However, it does not explicitly differentiate from sibling tools like list_job_groups, which might also list parent groups by default.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. With many sibling list tools, the agent receives no context to choose appropriately.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It does not disclose that the tool is read-only, safe, or any auth requirements. The behavior is simple, but transparency is lacking.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence, no fluff. Front-loaded with the action and resource. Perfectly concise for what it describes.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With zero parameters and an output schema present, the description is minimally adequate. However, it does not explain what 'configured products (mediums)' are, leaving some ambiguity. Could be improved with a brief definition.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters; schema coverage is 100% by default. The description adds no extra meaning beyond the empty schema, which is acceptable given zero params. Baseline score of 4 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clear verb+resource: 'List configured products (mediums).' It specifies exactly what the tool does and the resource type. It is distinct from sibling tools which are about jobs, machines, etc., so no confusion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs. alternatives. The description provides no context about prerequisites, typical use cases, or when not to use it. While siblings are unrelated, explicit usage hints are missing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description bears full burden. It indicates a read operation returning status fields, but lacks disclosure of any side effects, authentication requirements, or error conditions. It is adequate but not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence containing only essential information: verb, resource, and key fields returned. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the output schema exists, the description naming return fields is helpful. However, it omits parameter semantics and usage context relative to numerous sibling tools, leaving gaps for a complex domain like job management.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, meaning the input schema has no parameter descriptions. The tool description only mentions return fields and does not explain the 'follow' parameter or further clarify 'job_id', leaving the agent without guidance on how to use these parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the tool retrieves a lightweight job status with specific fields (id, state, result, blocked_by_id). The term 'lightweight' implies a simpler, faster alternative to more detailed job retrieval tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for quick status checks via 'lightweight', but does not explicitly state when to use this over siblings like get_job or get_job_details, nor when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, and the description does not disclose any behavioral traits beyond the basic read operation. There is no mention of permissions, side effects, or what 'known to the system' entails.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence. It could be slightly more informative, but it is not verbose and is appropriate for the tool's simplicity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no parameters and an output schema exists, the description is minimally complete. However, it lacks context about what constitutes an 'asset' or how the list is ordered.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has no parameters, and schema coverage is 100% (implicitly). The description does not add semantics but also does not need to, as there are no parameters to document.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (list) and the resource (assets). It distinguishes from sibling tools like list_bugs, list_jobs, etc., by specifying a different resource type.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. There is no mention of prerequisites or conditions under which list_assets is preferred.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It explains the equality condition but does not address case sensitivity, the meaning of 'setting', or behavior with no matches. Basic transparency is achieved, but depth is lacking.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence without superfluous content. It could be slightly more structured (e.g., adding a phrase about the search behavior), but it earns a high score for efficiency.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema present, return values are covered elsewhere. The description covers the basic query logic, but it omits details like handling multiple matches or setting key domains. It is minimally complete for a simple retrieval tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, so the description adds meaning by naming the parameters as setting key and list value. However, it does not explain valid values or constraints beyond the schema's type, providing only minimal semantic value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: finding jobs based on a setting key-value pair. It uses a specific verb ('Find') and resource ('jobs by setting'), and it distinguishes itself from siblings like 'list_jobs' or 'get_job' which likely serve broader or different purposes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use when filtering jobs by a specific setting, but it lacks explicit guidance on when not to use it or alternatives. Given sibling names, the context is clear, but no exclusions are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses that results can be very large and may be truncated, and explains the summary parameter's effect. However, it does not explicitly state that the tool is read-only, leaving some behavioral ambiguity.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences: purpose, warning, advice. It is front-loaded with the primary action and contains no redundant or vague statements. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (14 parameters, no annotations, output schema exists), the description covers purpose, size warning, and summary usage. However, it lacks explanation for most parameters and does not mention authentication or permissions. It is adequate but not fully comprehensive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It only explains the 'summary' parameter, ignoring the other 13 parameters. The description adds minimal value beyond the schema for most parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists a 'condensed jobs overview' matching filters, distinguishing it from siblings like list_jobs. The verb 'list' and resource 'jobs overview' are specific, and the warning about large results indicates the tool's scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance on when to use summary=True for triage and suggests saving to file for full data, along with a jq example. However, it does not explicitly exclude alternatives or state when not to use this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses key behaviors: result size, truncation risk, and workarounds (summary, file processing). No contradictions, and it adds actionable information beyond the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short paragraphs: first states purpose, second provides a critical warning and action. Every sentence is valuable, though the warning could be slightly condensed. Front-loaded with purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Output schema exists, so return values are documented. The description covers the most important behavioral context (size, truncation). However, it lacks any parameter-level guidance, which is a significant gap given 15 parameters and 0% schema coverage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 0% description coverage for 15 parameters. Description only mentions the summary parameter (value true/false) without details. Does not explain other parameters like ids, arch, test, etc., meaning the description adds almost no semantic value to the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List jobs matching the given filters.' with a specific verb and resource. It distinguishes from siblings like list_jobs_overview by mentioning the summary parameter for compact output.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides clear context: warns about large results (1.5 MB, 150+ jobs) and potential truncation, suggests using summary=True for triage or saving to file for full data. Does not explicitly compare to alternatives like list_jobs_overview, but offers practical usage guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

GitHub Badge

Glama performs regular codebase and documentation scans to:

Confirm that the MCP server is working as expected.
Confirm that there are no obvious security issues.
Evaluate tool definition quality.

Our badge communicates server capabilities, safety, and installation instructions.

Card Badge

Copy to your README.md:

[![openqa-mcp MCP server](https://glama.ai/mcp/servers/mimi1vx/openqa-mcp/badges/card.svg)](https://glama.ai/mcp/servers/mimi1vx/openqa-mcp)

Score Badge

Copy to your README.md:

[![openqa-mcp MCP server](https://glama.ai/mcp/servers/mimi1vx/openqa-mcp/badges/score.svg)](https://glama.ai/mcp/servers/mimi1vx/openqa-mcp)

Latest Blog Posts

Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)
By Om-Shree-0709 on July 25, 2026.
mcp
Agent Identity
OAuth 2.1
Your AI Chatbot Just Exposed Your CEO's Salary to an Intern
By Om-Shree-0709 on July 2, 2026.
Agent Identity
MCP Security
OAuth Delegation
Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)
By Om-Shree-0709 on June 30, 2026.
Agentic Ai
Prompt Injection
WebAssembly

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/mimi1vx/openqa-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server