qarunbook
Server Details
Shared app-testing runbook: read the plan, record results per platform, raise and fix issues.
- Status
- Healthy
- Last Tested
- Transport
- Streamable HTTP · MCP 2025-06-18
- URL
- Repository
- Ifeanyiejindu/qarunbook
- GitHub Stars
- 0
TDQS
Scored across 14 tools
Every tool targets a distinct action on a distinct resource: app creation, section/check management, result recording, issue lifecycle, and summary queries. There is no overlap between tools like add_issue and resolve_issue or set_result and list_checks; each has a clear and unique purpose.
Most tools follow a verb_noun pattern in snake_case (add_section, get_check, list_issues, resolve_issue). Minor deviation: 'progress' is a single noun rather than verb_noun, and 'create_app' uses 'create' while others use 'add' or 'list', but the overall style is consistent and predictable.
14 tools is well within the ideal range for a domain-specific server. Each tool addresses a necessary part of runbook management—creation, querying, updates, issue tracking, and coverage reporting—without bloat or redundant utilities.
The tool surface covers the full lifecycle of creating, reading, updating, and reporting on checks and issues. Missing features include deletion (e.g., remove_app, delete_check) and a dedicated get_issue tool, but these are not core to typical runbook workflows and can be worked around with existing tools.
Available Tools
14 toolsadd_checkAdd a checkBInspect
Add a check to a section. Its reference is generated as the next number in that section's series. Steps should be followable by someone who has never seen the code.
| Name | Required | Description | Default |
|---|---|---|---|
| app | Yes | App name. | |
| steps | No | Numbered steps to follow. | |
| title | Yes | What is being checked, in the user's words. | |
| section | Yes | Section title or reference. | |
| expected | No | What should happen. Concrete and checkable — never 'it works'. | |
| preconditions | No | Who is signed in, what state things must be in. | |
| not_applicable_on | No | Platform codes this check does not apply to. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint=false, destructiveHint=false, which means it's a mutation but not destructive. The description adds that the reference is auto-generated from the section's series, which is useful. However, it does not disclose other behaviors like authentication requirements, reversal possibilities, or impact on existing data. With annotations providing baseline safety, the description adds some value but not comprehensive behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, concise and front-loaded with the primary action. The second sentence about steps is an additional guideline but not necessary for the core purpose. No waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool that adds a check, the description covers the main purpose and reference generation. However, it lacks output details (no output schema) and does not explain what happens when the check is added (e.g., confirmation, where it appears). The complexity is moderate with 7 params, but schema covers parameter semantics. It's adequately complete but not rich.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so every parameter has a description in the schema. The description adds only the reference generation context, not parameter-specific details. Baseline 3 is appropriate since schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states the tool adds a check to a section and mentions reference generation, which is a specific function. However, it does not explicitly differentiate from the sibling 'update_check' or 'set_result', so it lacks explicit sibling distinction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives a clear context (adding a check to a section) and a behavioral note about steps followability, but it does not explicitly state when to use this tool versus adding an issue or updating a check. No exclusions or alternatives are mentioned, so guidance is implied rather than explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
add_issueRaise an issueAInspect
Raise an issue against a check. This takes the pass away on the platforms it affects, until it is marked fixed. Leave platforms empty to mean every platform.
| Name | Required | Description | Default |
|---|---|---|---|
| by | Yes | Who is reporting it. | |
| app | Yes | App name. | |
| ref | Yes | Check reference, e.g. 'AUTH-01'. | |
| text | Yes | What went wrong. Include the exact error text if there is one. | |
| platforms | No | Platform codes affected, e.g. ['WL','WM'] for web only. Empty means all. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate a mutating operation (readOnlyHint: false). The description adds context by explaining the effect: 'This takes the pass away on the platforms it affects' and the reversible nature until fixed. This goes beyond the annotations without contradicting them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with no redundancy. The purpose and the key behavioral note are front-loaded, making it easy to scan.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 5 parameters, no output schema, and the annotations cover safety, the description is fairly complete. It explains the effect and platform behavior, but does not mention return values or prerequisites like the need for an existing check. Minor gaps for a mutation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so all parameters are documented. The description repeats the 'empty means all' guidance already present in the schema, adding no new meaning. It does not explain parameter formats or relationships beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the specific action: 'Raise an issue against a check' and explains the consequence (takes the pass away). It distinguishes from siblings like edit_issue and resolve_issue by describing the lifecycle effect.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives a specific usage note for the platforms parameter ('Leave platforms empty to mean every platform') but does not explicitly mention when to use alternatives like edit_issue or resolve_issue. The phrase 'until it is marked fixed' implies resolve_issue but does not name it directly, leaving the agent to infer.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
add_sectionAdd a sectionAInspect
Add a section to an app — a module or feature area. Its reference is the prefix its checks are numbered against, so 'SUP' gives SUP-01, SUP-02. No two sections in an app may share one.
| Name | Required | Description | Default |
|---|---|---|---|
| app | Yes | App name. | |
| ref | No | Reference prefix, e.g. 'SUP'. Derived from the title if omitted. | |
| title | Yes | Section name, e.g. 'Support'. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=false and destructiveHint=false, so the description doesn't need to restate safety. The description adds valuable behavioral context: the reference prefix is used for numbering checks (SUP-01, SUP-02) and the uniqueness constraint across sections in an app. This goes beyond the schema and annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loads the core purpose, and every sentence adds value. The reference-prefix explanation is concise and directly relevant to correct usage.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a creation tool with no output schema, the description covers the key behavioral aspects: what a section is, how ref works, and the uniqueness constraint. It doesn't mention error cases or whether the operation is reversible, but annotations already cover the safety profile and the schema covers parameters.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all three parameters. The description adds context about the 'ref' parameter's role in check numbering and its derivation from title if omitted, which is useful. However, it doesn't add much beyond what the schema already provides for 'app' and 'title'.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Add') and resource ('a section to an app'), and clarifies what a section is ('a module or feature area'). It also distinguishes the tool from siblings by explaining the reference prefix behavior, which is unique to this tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use this tool: when creating a section within an app, and it clarifies the uniqueness constraint ('No two sections in an app may share one'). It doesn't explicitly name alternatives or exclusions, but the sibling list and the specific resource make the usage context clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
create_appCreate an appAInspect
Add an app to the runbook. Give it a name and the platforms it is tested on. Then use add_section and add_check to fill it in — or upload a markdown test plan through the web app, which is faster for a whole plan.
| Name | Required | Description | Default |
|---|---|---|---|
| by | Yes | Who is adding it. | |
| name | Yes | App name, e.g. 'Errbud'. | |
| platforms | No | Platforms this app is tested on. Defaults to web laptop, web mobile, Android and iOS. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already establish that the tool is not read-only and not destructive; the description adds the 'to the runbook' scoping and clarifies that the app is created as a shell to be filled in later. It does not discuss duplicate handling, idempotency, or return value, but for a simple create tool the core side effect is clear.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no filler: the first states the core purpose, and the second gives the workflow and a bulk alternative. The most important information is front-loaded and every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple three-parameter create tool with full schema coverage, the description is mostly adequate and includes useful workflow guidance. The main gap is that the bulk-import alternative is described as 'through the web app' rather than pointing to the available sibling import_plan tool, which is the actionable option in this MCP context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents name, by, and platforms. The description only lightly restates 'name and platforms' and does not add meaning beyond the schema; the required by parameter is omitted from prose but is fully covered in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Add an app to the runbook', a specific verb and object that clearly identifies what the tool does. It also distinguishes itself from add_section and add_check by framing those as follow-up steps rather than alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives a clear workflow: create the app first, then use add_section and add_check to fill it in, or upload a markdown plan for a whole plan. It names alternatives but routes the bulk upload through the web app instead of the sibling import_plan tool, so it is not fully explicit about the when-not-to-use case.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
edit_issueEdit an issueADestructiveInspect
Reword an issue, or change which platforms it affects — when a report is vague, wrong, or missing the exact error. The original reporter stays on it, you are recorded as the editor, and the previous wording is kept in its history.
| Name | Required | Description | Default |
|---|---|---|---|
| by | Yes | Who is making the edit. | |
| text | No | The corrected description. Leave out to keep the current wording. | |
| issue_id | Yes | Issue id from list_issues or get_check. | |
| platforms | No | Platform codes it affects. Leave out to keep them; an empty list means all. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark this as destructive (destructiveHint: true). The description adds valuable behavioral context: 'The original reporter stays on it, you are recorded as the editor, and the previous wording is kept in its history.' This goes beyond the annotations and clarifies side effects and ownership semantics.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two short sentences with no filler. It leads with the primary action, then states the use case, and finally the behavioral effects. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 4-parameter tool with no output schema, the description covers purpose, use case, and behavioral side effects. All parameters are documented in the schema. It doesn't mention error cases or permissions, but given the simplicity and annotations, it is adequately complete for an agent to call it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and each parameter already has a clear description, including the semantics of omitting 'text' or 'platforms' (e.g., 'Leave out to keep the current wording'). The tool description adds no extra parameter meaning, so it meets the baseline of 3 for well-documented schemas.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a clear verb and resource: 'Reword an issue, or change which platforms it affects.' It also provides a concrete use case ('when a report is vague, wrong, or missing the exact error'). While it doesn't explicitly name sibling tools, the action is distinct from adding or resolving issues, so it is not a tautology.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives a clear 'when' condition: 'when a report is vague, wrong, or missing the exact error.' This is explicit context for when to use the tool. However, it does not mention when not to use it or suggest alternatives, so it falls short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_checkGet one checkARead-onlyInspect
One check in full by its reference, e.g. 'AUTH-01': preconditions, steps, expected result, status on each platform, and every issue against it.
| Name | Required | Description | Default |
|---|---|---|---|
| app | Yes | App name. | |
| ref | Yes | Check reference, e.g. 'AUTH-01'. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false, covering the safety profile. The description adds value by detailing what the full check includes, which is especially important because there is no output schema. It doesn't describe error handling or nonexistent-reference behavior, but those are secondary for a read-only lookup.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One efficiently structured sentence that front-loads the operation ('One check in full by its reference') and uses a colon to list the included components. Every phrase earns its place with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Since there is no output schema, the description compensates by listing the return contents, giving the agent a clear expectation of what it will receive. It does not address what happens if the reference doesn't exist, but for a read-only single-item lookup, the essential calling context (app and ref) and return contents are covered.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% with both app and ref documented. The description reinforces the ref example ('AUTH-01') but doesn't add meaning beyond what the schema already provides. With full coverage, the baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a clear verb and resource: retrieving one check in full by its reference. It enumerates what 'full' includes (preconditions, steps, expected result, platform statuses, issues), making the purpose specific. However, it doesn't explicitly distinguish itself from the sibling list_checks, relying on the tool name and context to convey the difference.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the appropriate use case: when you have a specific check reference and need complete details. It does not explicitly state when not to use it or mention alternatives such as list_checks for summary-level queries. The guidance is implied but not explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
import_planImport a test planAInspect
Create a whole app from a markdown test plan, in one call. This is how to add a real runbook — a plan runs to hundreds of checks and add_check one at a time is not the way. Sections come from the '##' headings, checks from the table rows beneath them, and the platform columns from the check table's header. Pass the file's contents as text.
| Name | Required | Description | Default |
|---|---|---|---|
| by | Yes | Who is importing it. | |
| name | Yes | App name to create, e.g. 'Errbud'. | |
| markdown | Yes | The whole markdown document. Each section is a '##' heading with one table under it; the first column of every check row is a reference like AUTH-01; trailing columns are the platforms, and a cell reading N/A marks that platform not applicable. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate the operation is not read-only and not destructive; the description adds meaningful behavioral context by explaining how the markdown is parsed (## headings, table rows, header columns) and that the entire app is created in a single call. It does not mention potential collisions or error handling, but the annotations plus parsing detail cover the main behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences with zero filler. The purpose is front-loaded, the alternative is named early, and parsing rules are packed compactly. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a moderately complex import operation, the description covers purpose, usage condition, parsing semantics, and input format. The schema provides full parameter documentation. The main gap is the lack of any description of the return value, but with no output schema and rich surrounding context, this is a minor omission, not a blocking one.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, and the schema already fully documents all three parameters, including the markdown format. The main description adds only a small clarification ('Pass the file's contents as text'), which does not substantially go beyond the schema. Hence the baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Create a whole app from a markdown test plan, in one call.' It clearly distinguishes itself from the sibling add_check by contrasting bulk import with one-at-a-time addition. The parsing rules for sections, checks, and platform columns further pin down exactly what the tool does.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly gives the when-to-use condition: 'This is how to add a real runbook — a plan runs to hundreds of checks and add_check one at a time is not the way.' It names the alternative (add_check) and explains why that alternative is inappropriate for large plans, leaving no ambiguity about which tool to pick.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_appsList appsARead-onlyInspect
List the apps in the runbook, with their platform codes and how many checks each has. Call this first to get an app name to pass to the other tools.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false, so the safety profile is known. The description adds valuable behavioral context by describing the output contents, platform codes and check counts, without contradicting any annotation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two efficient sentences with no redundant phrasing. The core purpose is front-loaded, and the workflow note is essential rather than filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter read-only listing tool, the description is complete: it states the resource, the returned fields, and how the result should be used. Without an output schema, describing the returned content is necessary, and this description does so clearly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the schema is complete and the description needs no parameter documentation. The description adds no parameter detail, but none is needed; baseline for zero parameters is a 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: listing apps in the runbookciation. It also names the returned details (platform codes and check counts), which clearly distinguishes it from sibling tools like list_checks and list_issues.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says to call the tool first and explains why: to obtain an app name to pass to other tools. This provides clear workflow guidance and prevents an agent from skipping a required prerequisite step.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_checksList checksARead-onlyInspect
Checks in an app, with their derived status per platform. Filter to 'failing' to see what is broken, 'retest' for fixes nobody has verified yet, 'untested' for what nobody has looked at.
| Name | Required | Description | Default |
|---|---|---|---|
| app | Yes | App name. | |
| limit | No | Default 50, max 200. | |
| filter | No | Defaults to 'failing'. | |
| section | No | Only checks whose section title or ref matches this. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false, so the read-only safety profile is covered. The description adds useful behavioral context by explaining that statuses are 'derived per platform' and by interpreting what retest and untested mean. It does not describe output shape, pagination, or ordering, but given the annotations the additional context is sufficient for a mid-range score.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with no filler. The first sentence states the core function and scope, and the second sentence provides immediately useful filter guidance. Every word earns its place, and the structure is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only list tool with a well-documented schema, the description covers the key aspects an agent needs: what is listed, the app scope, and the meaning of important filters. There is no output schema, but the phrase 'checks ... with their derived status per platform' gives a sufficient high-level return expectation. It could add pagination details, but those are already in the schema's limit parameter.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description goes beyond the schema by explaining the practical meaning of the filter enum values: failing, retest, and untested. This adds real decision-making value for an agent choosing a filter. It does not cover every parameter, but the schema already handles those, and the filter semantics are the most behaviorally important.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the resource as 'checks in an app' and adds that statuses are 'derived status per platform', which differentiates it from a plain list or a single-check tool like get_check. The title supplies the missing verb 'List', and the plural scope is unambiguous. It stops short of explicitly naming a sibling alternative, so it is clear but not maximally differentiated.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives concrete guidance on when to use specific filter values: 'failing' to see broken items, 'retest' for unverified fixes, and 'untested' for items nobody has looked at. However, it does not contrast this tool with siblings such as get_check or list_issues, nor state when not to use it. The usage context is implied rather than explicitly framed against alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_issuesList issuesARead-onlyInspect
Issues testers raised, newest first — the text they wrote, who raised it, which platforms it affects, and which check it is against. Defaults to open issues only. This is the main thing to read before fixing anything.
| Name | Required | Description | Default |
|---|---|---|---|
| app | Yes | App name. | |
| limit | No | Default 50, max 200. | |
| status | No | Defaults to 'open'. | |
| platform | No | Only issues affecting this platform code, e.g. 'WL'. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark the tool as read-only and non-destructive. The description adds useful behavioral context beyond annotations: results are sorted newest first, they default to open issues only, and the returned fields are listed. Nothing contradicts the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and front-loaded: it opens with the returned content, then states the default filter, then gives strategic usage advice. Every sentence earns its place with no filler or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, but the description compensates by naming the key output fields and the ordering. It doesn't describe response shape or pagination details, but the limit parameter and simple list semantics make the tool sufficiently transparent for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, and all four parameters are documented in the schema. The description mainly repeats the default-open behavior already present in the status parameter, so it adds little parameter-level meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List issues' with a specific verb and resource, and enumerates what the output contains: text, raiser, platforms, and check. It does not explicitly name sibling alternatives, but the phrase 'main thing to read before fixing anything' separates it from mutation tools like add_issue, edit_issue, and resolve_issue.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives an explicit usage context: this is the primary tool to read before making fixes. It doesn't list alternative tools or conditions for when not to use it, but the default-open filter and status enum in the schema provide enough practical guidance for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
progressCoverage summaryARead-onlyInspect
Coverage summary for an app: how many platform-checks are passing, failing, untested or not applicable, and how many issues are open. Use it to answer 'where is testing up to'.
| Name | Required | Description | Default |
|---|---|---|---|
| app | Yes | App name, e.g. 'Smart Invites'. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already establish readOnlyHint=true and destructiveHint=false, so the safety profile is covered. The description adds useful behavioral context by specifying the exact summary metrics returned and confirming the tool is an aggregate read-only view rather than a mutating operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with no filler. It front-loads the core purpose and output, then provides a crisp usage cue. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter read-only tool, the description covers the essential return information (counts by status and open issues) and the intended use case. It does not detail output formatting, but this is a minor gap given the tool's simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already fully documents the single required 'app' parameter with an example, so schema coverage is 100%. The description reinforces that the summary is scoped to an app, but adds no extra semantic detail beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the tool's output: a summary of platform-check counts by status (passing, failing, untested, not applicable) plus open issues. This distinguishes it from sibling list/detail tools like list_checks, which would enumerate individual checks rather than aggregate counts.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use it: 'Use it to answer where is testing up to.' This gives clear context for selection, though it does not explicitly mention alternatives or when not to use it, so it stops short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
resolve_issueMark an issue fixedADestructiveInspect
Mark an issue fixed, or reopen it. Fixing restores whatever mark was underneath — a check that was passing before the issue goes back to passing, and one that was never tested goes back to untested.
| Name | Required | Description | Default |
|---|---|---|---|
| by | Yes | Who is making the change. | |
| fixed | No | true to fix (default), false to reopen. | |
| issue_id | Yes | Issue id from list_issues or get_check. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds a non-obvious behavioral detail beyond the annotations: fixing 'restores whatever mark was underneath,' with concrete examples. The destructiveHint annotation already signals mutation, and the description usefully explains what changes happen to the underlying check state.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with no filler. The core action is front-loaded, and the explanatory example is tightly scoped to the most important behavioral nuance.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple three-parameter mutation tool with no output schema, the description covers the key behavior, the target issue source is documented in the schema, and the toggling semantics are clear. It omits permission requirements and return details, but those are not critical given the annotations and schema richness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds meaning to the `fixed` parameter by explaining that setting an issue as fixed is not a simple boolean flip—it restores the prior mark. This deepens the agent's understanding beyond the schema's 'true to fix (default), false to reopen.'
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Mark an issue fixed, or reopen it.' It also clarifies the dual behavior, which distinguishes this status-toggle tool from generic issue-editing tools like edit_issue.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear context for when to invoke the tool: to fix or reopen an issue. It does not explicitly name alternatives or state when not to use it, but the purpose is unmistakable and there are no misleading instructions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
set_resultRecord a resultADestructiveInspect
Record a tester's result for one check on one platform: 'P' passed, 'F' failed, '' clears it back to untested. An open issue still overrides this while it is open, and recording a result is what clears an 'R' (retest) after a fix.
| Name | Required | Description | Default |
|---|---|---|---|
| by | Yes | Who recorded it. | |
| app | Yes | App name. | |
| ref | Yes | Check reference. | |
| status | Yes | P, F, or empty to clear. | |
| platform | Yes | Platform code, e.g. 'WL'. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already signal a destructive write, and the description adds substantial behavioral detail: clearing with '', the fact that open issues override recorded results, and that recording a result clears an 'R' retest. This goes well beyond what the annotations alone would tell an agent and is fully consistent with them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no wasted words: the first sentence front-loads the core action and the second sentence adds high-value behavioral nuance. Every clause earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple write tool with all parameters documented in the schema, the description covers the important correctness hazards: clearing behavior, issue override, and retest semantics. It does not cover error cases or permissions, but those are not essential for selecting and invoking the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description adds meaningful semantics for the status parameter by explaining what P, F, and '' mean and how recording interacts with retest, which is useful beyond the raw enum listing.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses the specific verb 'Record' and identifies the exact resource: a tester's result for one check on one platform. It enumerates the valid status values and their meanings, which clearly distinguishes it from sibling tools like add_check or update_check.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides clear context for when to use the tool: whenever recording a check result, and it explains the relationship with open issues and the 'R' retest state. It does not explicitly name alternatives or exclusion cases, but the context is sufficient for an agent to select it correctly.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
update_checkCorrect a checkADestructiveInspect
Correct a check — its wording, or which platforms it applies to. Use when a step is wrong or misleading, or when a check does not belong on a platform at all. Pass not_applicable_on to mark it N/A there; it then stops counting towards coverage on those platforms. The reference is not editable; it is a position in the section's series.
| Name | Required | Description | Default |
|---|---|---|---|
| app | Yes | App name. | |
| exp | No | New expected result. | |
| pre | No | New preconditions. | |
| ref | Yes | Check reference. | |
| steps | No | New steps. | |
| title | No | New title. | |
| not_applicable_on | No | Platform codes this check does not apply to, e.g. ['IOS','AND'] for a web-only check. Replaces the current list rather than adding to it — pass [] to make the check apply everywhere again. Omit to leave it alone. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already signal mutation (readOnlyHint=false, destructiveHint=true), and the description adds useful behavioral detail: not_applicable_on marks a check N/A and stops it counting toward coverage, and the reference is not editable. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Four dense sentences, front-loaded with purpose, then usage, then key behavioral constraints. No filler or redundant restatement of schema fields.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given seven parameters, full schema descriptions, and a destructive annotation, the description supplies the missing usage context and key side effects. No output schema exists, but for an update operation the returned value is not critical to invoking it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description adds value beyond the schema by explaining the effect of not_applicable_on on coverage and by stating that ref is a position rather than an editable value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action ('Correct') on a specific resource ('a check') and clarifies the scope: wording or platform applicability. This lets an agent distinguish it from siblings like add_check, get_check, and list_checks without opening the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives explicit trigger conditions: use when a step is wrong or misleading, or when a check does not belong on a platform. It does not explicitly name alternatives or exclusions, but the intended scenarios are clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
14 tool updates
- First observed
add_check - First observed
add_issue - First observed
add_section - First observed
create_app - First observed
edit_issue - First observed
get_check - First observed
import_plan - First observed
list_apps - First observed
list_checks - First observed
list_issues - First observed
progress - First observed
resolve_issue - First observed
set_result - First observed
update_check
Related MCP Connectors
3rd Generation Testing (3TG) — generate deterministic test suites from Markdown spec tables via MCP.
Discover software problems, analyze evidence, and create implementation-ready Build plans.
Manage test suites, run tests, view results, and automate QA workflows via AI with testRigor.
Katalon True Platform MCP: design tests, run with AI, upload reports, assess release readiness.
Related MCP Servers
- AlicenseNot gradedqualityBmaintenanceProvides autonomous QA and a deterministic ship/no-ship verdict for iOS and web apps, enabling coding agents to explore, screenshot, and test apps via MCP tools.563 npm2MIT
- AlicenseNot gradedqualityBmaintenanceEnables agents to author, validate, and execute browser/API test plans with parallel drivers, assertions, and captured evidence such as screenshots and network logs.MIT
- FlicenseNot gradedqualityNot gradedmaintenanceOrchestrates end-to-end testing of AI-powered incident remediation workflows through declarative YAML scenarios, fault injection, AI response evaluation, and automated command execution with comprehensive reporting.-
- FlicenseNot gradedqualityDmaintenanceTest automation across web, mobile, desktop, API, and SAP GUI via ActionTestScript. 30 tools covering the Playwright MCP surface plus native multi-platform reach.-
Glama MCP Gateway
Add one secure layer between your agents and this server.