@checked/mcp
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@@checked/mcpshow me the latest run's report and top findings"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
checkd-mcp
An MCP server (stdio) that lets AI assistants use Check’d: read run results, AI reports and findings with their evidence, and, when you ask, start runs, re-run failed tests or scan a website.
Needs Node.js 20 or newer. Built on the official MCP TypeScript SDK and the checkd-cli core.
Credentials
CHECKED_API_KEY: a workspace API key (Settings, API keys in the app). Reaches that workspace only.or
CHECKED_TOKEN, or the sign-in saved bychecked loginon this machine.CHECKED_API_URL: only if you use a server other thanhttps://getchecked.twelveai.app.
Without a credential the server still starts; every tool answers with how to sign in.
Related MCP server: Autotest AI MCP Server
Add it
Claude Code:
claude mcp add checked --env CHECKED_API_KEY=tfy_your_key -- npx -y checkd-mcpCursor (.cursor/mcp.json, or ~/.cursor/mcp.json for every project):
{
"mcpServers": {
"checked": { "command": "npx", "args": ["-y", "checkd-mcp"], "env": { "CHECKED_API_KEY": "tfy_your_key" } }
}
}VS Code (.vscode/mcp.json; VS Code asks for the key once):
{
"inputs": [{ "type": "promptString", "id": "checked-api-key", "description": "Check’d workspace API key", "password": true }],
"servers": {
"checked": { "type": "stdio", "command": "npx", "args": ["-y", "checkd-mcp"], "env": { "CHECKED_API_KEY": "${input:checked-api-key}" } }
}
}Tools
Read-only (free; readOnlyHint: true):
Tool | What it does |
| Which account or key, workspaces, plans |
| Projects with their last run |
| A project’s environments (secrets masked) |
| Recent runs |
| Status, release gates, AI verdict, top problems with finding ids |
| Waits up to 600 s for a run to finish |
| Individual API test results |
| The AI QA report |
| Failed tests and scan findings; full request, response, checks, stack trace, diagnosis and fix |
| The issue inbox: production errors and test findings, with the cached AI explanation (see the seam below) |
| Session and month usage, and whether work is blocked |
| What an API import would add; nothing saved |
Start work or change things (readOnlyHint: false; descriptions say they use allowance):
Tool | What it does |
| Start a run; optional |
| Re-run only the failed tests of a finished API run |
| Website scan (uses or creates the workspace’s project for that host) |
| Save endpoints from an API document |
| Cancel a run |
Each tool returns a readable text answer plus the JSON it was built from. Errors come back as isError results; a 402 says the allowance is used up, when it resets, and not to retry.
Resources: checked://runs/{run}/report, checked://runs/{run}/summary, checked://findings/{id}. Prompt: fix_failures (look at the latest run of a project and fix what it found).
The server sends instructions to the client: prefer read-only tools, start work only when asked, never loop on work-starting tools, and stop on a usage limit.
Issues seam
The issue tools use issuesProvider() from checkd-cli/core. When /cli/whoami reports capabilities.issuesInbox: true they read the issue inbox (get_issue includes the privacy-safe evidence, the fix pull request and the cached AI explanation; it never starts a new one); on older servers they read findings. list_findings and get_finding always read runs.
Development
npm install # links ../cli; build it first (cd ../cli && npm run build)
npm test # tool schemas and handlers against a mocked API, over an in-memory MCP transport
npm run build
CHECKED_API_KEY=tfy_... CHECKED_API_URL=http://localhost:4300 node dist/bin.jsAvailable Tools
19 toolscancel_runCancel a runADestructiveIdempotent
Cancels a queued or running run. Stops further usage by that run.
| Name | Required | Description | Default |
|---|---|---|---|
| run | Yes | Run ref (e.g. payments-kqzxm) or numeric id |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true, readOnlyHint=false, and idempotentHint=true, so the safety profile is covered structurally. The description adds a genuine consequence beyond those flags—'Stops further usage by that run'—telling the agent what resource consumption ceases, which is the kind of 'what gets destroyed' context the annotations do not spell out. It still omits the outcome for a run that is already terminal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, zero filler, with the action and its scope front-loaded before the effect. Nothing could be trimmed without losing information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter mutation with no output schema, the description plus annotations cover the essentials an agent needs to call it safely. The remaining gap is minor: it does not say what happens when the target run is already complete or unknown to the cancellation path.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%: the single 'run' parameter already documents that it accepts a run ref or numeric id. The description adds nothing about the parameter, so the baseline 3 applies—the schema does all the work.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific verb and resource ('Cancels ... a run') and narrows the scope to 'queued or running' runs, which lets an agent tell it apart from start_run, rerun_failed, and wait_for_run by name alone. It stops short of explicitly contrasting itself with those siblings, so it is clear but not fully differentiated.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Applicability is implied by state: only a queued or running run can be cancelled. There is no explicit when-to-use guidance, no statement of what to do instead once a run has finished or failed, and no mention of the alternatives (rerun_failed, start_run) that an agent might confuse this with.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_findingGet findingARead-onlyIdempotent
Read-only. One finding with its full evidence: the exact request and response, failed checks, stack trace or console errors when captured, the diagnosis and how to fix it.
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | r123 or f456 |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover safety (readOnly, idempotent, non-destructive, closed-world), so 'Read-only' is redundant, but the description adds real value by disclosing exactly what the response contains (raw request/response, failed checks, stack traces, console errors, diagnosis, fix). It omits behavior for a missing/invalid id.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences with the key classification ('Read-only') front-loaded and the content list kept dense and useful. Minor redundancy: 'Read-only' duplicates the readOnlyHint annotation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description carries the burden of describing return values and does so thoroughly, listing every evidence category an agent would receive. Only error/not-found behavior is left unexplained.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and the single id parameter is documented in the schema ('r123 or f456'). The description adds no format or constraint detail beyond that, so baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('One finding with its full evidence'), and enumerates the payload contents, which clearly distinguishes it from the sibling list_findings. It does not name any sibling explicitly, so it stops short of a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The singular 'One finding' implies fetching details for a known id after a listing call, but there is no explicit when-to-use guidance or routing away from list_findings/get_issue. Usage is inferable but not stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_issueGet issueARead-onlyIdempotent
Read-only. One issue: occurrences, users, releases, the privacy-safe evidence (error, stack frames in the app’s code, breadcrumbs, request), its fix pull request, and the cached AI explanation (what is broken, likely cause, where to look, suggested fix). Never starts a new explanation. Also accepts finding ids (r123, f456).
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | An issue ref like kqzxmvbt, or r123 / f456 |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent and non-destructive, so the bar is lower, and the description still adds real context: the cached-vs-generated distinction ('Never starts a new explanation') is a genuine side-effect boundary, and 'privacy-safe evidence' tells the agent what class of data comes back. It stops short of describing pagination or size limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the mutation-safety fact and the single-resource scope, then the payload list. It is one dense paragraph with no filler sentences, though the mid-sentence enumeration of evidence types is long enough to slightly dilute the key routing information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description carries the return-value burden and does so by listing the returned fields, including the AI explanation's sub-parts. Annotations cover the safety profile. It is nearly complete, missing only volume/shape caveats (pagination, evidence truncation).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and the single 'id' parameter already documents the ref format including r123/f456. The description's 'Also accepts finding ids' restates that schema detail rather than adding new syntax or constraints, so the baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The first clause names a specific verb and resource ('Read-only. One issue') and then enumerates the payload: occurrences, users, releases, evidence, fix PR, cached AI explanation. This clearly separates it from list_issues and get_finding, and the 'Also accepts finding ids (r123, f456)' note pins down its scope relative to the finding siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives an important when-not ('Never starts a new explanation'), which prevents an agent from expecting generation on this call. However it never explicitly routes the agent to alternatives (e.g. get_finding for a finding-centric read, or why to pick this over that) despite overlapping id acceptance with get_finding, leaving the sibling choice to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_runGet run resultARead-onlyIdempotent
Read-only. A compact result for one run: status and counts, release gates, the AI verdict and the top problems with ids for get_finding.
| Name | Required | Description | Default |
|---|---|---|---|
| run | Yes | Run ref (e.g. payments-kqzxm) or numeric id |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and destructiveHint/openWorldHint false, so 'Read-only' is partly redundant. The value-add is the return-shape disclosure and the word 'compact', which signals a summarized rather than exhaustive payload — meaningful given there is no output schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence plus a safety lead-in, front-loaded and with zero filler. Every clause either qualifies safety or names returned content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With one fully documented parameter and no output schema, the description does the necessary work of sketching the return payload. It is slightly short on distinguishing this compact view from get_run_results/get_run_report, but nothing required to invoke the call correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and the single 'run' parameter is documented in the schema with accepted forms (slug or numeric id). The description adds nothing beyond that, so the baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Get run', 'a compact result for one run') and enumerates the payload (status, counts, release gates, AI verdict, top problems). It does not, however, differentiate itself from the close siblings get_run_results and get_run_report, leaving the agent to guess which run-scoped reader to pick.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'ids for get_finding' is an explicit hand-off to a sibling, which is genuine routing guidance. But there is no statement of when to use this over list_runs, get_run_results, get_run_report or wait_for_run, so the when-to-use decision is only implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_run_reportGet AI reportARead-onlyIdempotent
Read-only. The AI QA report for a run: verdict (ship, ship with care, hold), findings with impact and fixes, next steps, what was not tested.
| Name | Required | Description | Default |
|---|---|---|---|
| run | Yes | Run ref (e.g. payments-kqzxm) or numeric id |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover readOnly/idempotent/non-destructive, so 'Read-only' is partly redundant, but the description earns credit by disclosing the return shape that no output schema provides: the verdict vocabulary (ship, ship with care, hold), findings carrying impact and fixes, and explicit 'what was not tested' coverage. It omits whether the report may be absent or still generating, which is a real behavioral gap given wait_for_run exists as a sibling.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single compact sentence set, front-loaded with the safety note and then the payload contents; every clause carries information. The leading 'Read-only.' duplicates the annotation and is the only real waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description carries the return-value burden and does so well by naming the verdict enum and finding structure. For a one-parameter read tool with full annotations this is nearly complete; the missing piece is availability semantics — whether a report always exists or may need wait_for_run first.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single 'run' parameter is fully documented in the schema (coverage 100%), including the ref-or-numeric-id format and examples. The description adds nothing about parameter semantics, so the baseline 3 for high schema coverage applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Names a specific resource ('the AI QA report for a run') and enumerates its contents — verdict, findings with impact and fixes, next steps, untested areas — which cleanly separates it from raw-data siblings like get_run_results and list_findings. It stops short of explicitly naming or contrasting those siblings, so an agent must infer the boundary itself.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by the resource and the enumerated contents: fetch this when you want the synthesized verdict rather than raw results. However, the description never states when to prefer this over get_run_results, list_findings, or get_run, nor whether a report is guaranteed to exist for a run — the reader has to infer all of that.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_run_resultsGet test resultsBRead-onlyIdempotent
Read-only. Individual API test results of a run; filter to failures with status "failed,errored".
| Name | Required | Description | Default |
|---|---|---|---|
| run | Yes | Run ref (e.g. payments-kqzxm) or numeric id | |
| limit | No | ||
| status | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint and destructiveHint=false, so the leading 'Read-only' partly repeats structured data. It does add one genuinely useful behavioral detail the annotations lack — the comma-separated multi-value syntax for the status filter — but says nothing about pagination despite a limit parameter, or about result ordering/volume.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short, front-loaded fragments with no filler; the resource and the filtering trick are both immediate. The leading 'Read-only' is slightly wasteful given annotations, but nothing else is padded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, so the description should carry return-value expectations; 'individual API test results' is a thin sketch of the payload. With an undocumented limit parameter and no pagination or shape guidance, it is adequate but leaves real gaps for a results-listing tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is only 33%; 'run' is documented in the schema while 'limit' and 'status' are not. The description partially compensates by showing the status value syntax ('failed,errored'), but limit remains undocumented in both places, so it neither fully compensates nor is redundant.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: reads individual API test results belonging to a named run. This is distinct from get_run_report (aggregate) and list_findings (issue-level), though the description never names those siblings to make the distinction explicit.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The filter hint ('filter to failures with status "failed,errored"') implies the main use case of narrowing to failures, but it never states when to prefer this tool over get_run_report, get_run, or list_findings, nor any exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_usageGet usageBRead-onlyIdempotent
Read-only. How much of this session’s and this month’s allowance the workspace has used, and whether new work is blocked.
| Name | Required | Description | Default |
|---|---|---|---|
| workspace | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, so 'Read-only' merely restates structured data. The description does add real behavioral content by naming what is reported (session vs. month consumption and blocked status), but omits any note on scoping, caching, or refresh behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two compact sentences with the payload information front-loaded. The leading 'Read-only.' fragment is redundant against the annotations, a small amount of waste in an otherwise efficient description.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description carries the burden of describing returns and does so adequately — session and monthly allowance consumption plus a blocked flag. Units, exact fields, and reset semantics are left unspecified, but the shape of the answer is conveyed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% and the sole 'workspace' parameter is never mentioned in the description. The phrase 'the workspace has used' weakly implies the default workspace, but it provides no guidance on what value to pass or what happens when omitted.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific informational resource (allowance/usage) and its scope (this session and this month, plus whether new work is blocked). It's clear what the tool returns, though it makes no attempt to differentiate from siblings — reasonable here since no sibling is a usage/quota tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no explicit when-to-use or when-not-to-use guidance, but 'whether new work is blocked' implies the natural context of checking quota before starting a run. Usage is inferable rather than stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
import_specImport an API specADestructive
Saves endpoints into an API project from an OpenAPI, Postman or other document (merged by method and path; replace swaps the whole list). Call preview_import first and confirm with the user. May start an AI reading of the API, which uses allowance.
| Name | Required | Description | Default |
|---|---|---|---|
| url | No | ||
| content | No | ||
| project | Yes | Project ref (e.g. payments-kqzxm) or numeric id | |
| replace | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already flag destructiveHint=true, openWorldHint=true and idempotentHint=false. The description adds real value beyond them by explaining the mutation model ("merged by method and path; replace swaps the whole list") and a cost side-effect ("May start an AI reading of the API, which uses allowance"). It stops short of stating permissions, reversibility or rate limits, so it is strong but not exhaustive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three tight clauses, all front-loaded: what it does, how merging works, then the required prerequisite. No filler or restated name/title.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive import with no output schema and annotated write behavior, the description covers the merge model, the safety workflow and the cost side-effect. Only the url/content parameter distinction and any failure/partial-import behavior are left unaddressed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is only 25% — just `project` carries a description. The description clarifies the semantic effect of `replace` ("swaps the whole list") and implies url/content by naming source document types, but does not disambiguate `url` vs `content` or address the 8MB content cap and merge-vs-replace interplay with the other params. Baseline 3 given partial schema help.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ("Saves endpoints into an API project") plus the accepted source formats (OpenAPI, Postman, other documents). The merge-vs-replace clause and the preview_import prerequisite make it clearly distinguishable from its sibling preview_import.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly prescribes the workflow: "Call preview_import first and confirm with the user." It names the alternative tool and the gating condition for proceeding, leaving no inference required.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_environmentsList environmentsARead-onlyIdempotent
Read-only. Lists a project’s environments (base URL, auth type). Secrets are masked.
| Name | Required | Description | Default |
|---|---|---|---|
| project | Yes | Project ref (e.g. payments-kqzxm) or numeric id |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, so the leading "Read-only" adds nothing new. However, "Secrets are masked" is genuine behavioral context about the response that annotations do not convey, warning the agent that credential values will not be exposed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three compact clauses with no filler, and the safety/scope constraint is front-loaded. Every clause contributes distinct information: read-only, the resource and its returned fields, and the masking behavior.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description usefully names the returned fields (base URL, auth type) and the masking behavior. It does not mention ordering, pagination, or error behavior when the project ref is invalid, leaving a small gap for a listing tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and the single parameter is fully documented as a project ref or numeric id, so the schema carries the weight. The description only implies the parameter's role ("a project's environments") without adding format or constraint detail, matching the baseline for fully-covered schemas.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (Lists) and resource (a project's environments), and even names the fields returned (base URL, auth type). The resource is naturally distinct from siblings like list_projects or list_runs, but the description never explicitly contrasts with an alternative tool, which keeps it short of a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied: the tool scopes to "a project's environments" and requires a project ref, so an agent can infer it is the lookup for environments of a known project. There is no explicit when-to-use, when-not-to-use, or named alternative, so this is minimum-viable rather than strong guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_findingsList findingsARead-onlyIdempotent
Read-only. Problems from a run (default the latest finished one): failed API tests and website scan findings, worst first.
| Name | Required | Description | Default |
|---|---|---|---|
| run | No | Default: the latest finished run | |
| limit | No | ||
| project | Yes | Project ref (e.g. payments-kqzxm) or numeric id | |
| severity | No | This severity and worse |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover the safety profile (readOnly, idempotent, non-destructive), and 'Read-only' merely repeats that. The description does add genuinely new behavioral context: the default run selection, the result ordering ('worst first'), and the kinds of records returned.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences, front-loaded with the read-only constraint, then scop; no filler and every clause carries information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 4-parameter, read-only query tool with no output schema, the description covers what is returned and in what order. It omits limit/pagination behavior and filtering semantics, which an agent would still need to infer from the schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 75%, so most parameters are already documented, including the run default which the description merely restates. The description adds no format or syntax detail for `severity`, `limit`, or how `project` refs/hashes are accepted.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a clear verb+resource ('list findings') and defines the content precisely: failed API tests and website scan findings from a run. It implicitly distinguishes findings (problems within a run) from the sibling list_issues/get_issue, though it never names those alternatives explicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage context is implied by 'default the latest finished one' and the run-scoped framing, but there is no explicit when-to-use guidance, no exclusion of list_issues, and no pointer to get_finding for single-item retrieval.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_issuesList issuesARead-onlyIdempotent
Read-only. Open issues for a project from the issue inbox: errors real users hit in production and findings from tests and scans, highest impact first, with the one-line AI summary when one exists. On servers without the inbox, the latest run’s findings.
| Name | Required | Description | Default |
|---|---|---|---|
| run | No | Default: the latest finished run | |
| limit | No | ||
| source | No | ||
| status | No | Default open (new, regressed, investigating) | |
| project | Yes | Project ref (e.g. payments-kqzxm) or numeric id | |
| severity | No | This severity and worse |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent and non-destructive, so the safety profile is covered. The description adds genuinely new behavioral context beyond that: the sort order (highest impact first), the inclusion of a one-line AI summary when present, and a fallback data source when the inbox is unavailable. It stops short of documenting pagination or the limit parameter's effect.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences with the 'Read-only' qualifier front-loaded and the core purpose immediately following. The fallback clause is placed last, which is correct emphasis. Nothing is redundant, though the embedded clause listing sources makes the first sentence slightly dense.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description does the right work of summarizing returned content (issues with AI summaries, impact-ordered) and noting the fallback source. Given six parameters with one required, an agent has enough to call this correctly, though return cardinality and pagination are unaddressed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 67%, so several parameters (limit, source, severity) are already documented structurally. The description reinforces the default open-status filter implicitly ('open issues') but adds no syntax or format detail beyond the schema. This matches the baseline 3 when the schema does most of the work.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource ('Open issues for a project from the issue inbox') and qualifies the scope ('errors real users hit in production and findings from tests and scans'). It implicitly differentiates from the raw-findings sibling by describing an aggregated, prioritized inbox view, but never names list_findings explicitly, so sibling routing is left partly to inference.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by the scope ('open issues... from the issue inbox') but there is no explicit when-to-use/when-not guidance and no named alternative among siblings like list_findings or get_issue. The one routing hint is the fallback behavior on inbox-less servers, which addresses environment rather than task selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_projectsList projectsBRead-onlyIdempotent
Read-only. Lists Check’d projects (API projects test endpoints; website projects are scanned in a browser) with their last run.
| Name | Required | Description | Default |
|---|---|---|---|
| workspace | No | Workspace slug or id; default all the credential can see |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so 'Read-only.' is largely redundant. The description does add genuine behavioral context by disclosing that results include each project's last run and by explaining the API-vs-website project distinction, but it says nothing about pagination, ordering, or the default 'all workspaces' scope behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight clauses with no wasted words and the resource up front. The leading 'Read-only.' duplicates the annotations rather than earning its place, a minor inefficiency.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read-only list tool with a fully documented single optional parameter and no output schema, the description covers the key question of what comes back (projects plus last run). Additional detail on result shape or volume is not strictly required here.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% for the single workspace parameter, so the schema already documents it fully (slug or id, defaults to all visible). The description adds nothing about the parameter, which is the expected baseline when the schema does the work.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (Lists) and resource (Check'd projects) and even distinguishes two project subtypes (API vs website) with an important semantic difference in how they are exercised. It does not compare itself to sibling list tools like list_runs or list_environments, so it falls just short of 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description says what the tool returns but gives no when-to-use guidance, no condition for choosing it over siblings such as list_runs or whoami, and no exclusions. Usage must be inferred from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_runsList runsCRead-onlyIdempotent
Read-only. Recent runs of a project, newest first.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | ||
| status | No | Comma separated: queued, running, passed, failed, errored, cancelled | |
| project | Yes | Project ref (e.g. payments-kqzxm) or numeric id |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, non-destructive, and closed-world, so the 'Read-only' prefix is largely redundant. The description's only added behavioral fact is that results are ordered newest first, which is genuinely useful context not present in the annotations or schema, but defaults for limit and any pagination behavior are undisclosed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short fragments, front-loaded with the safety profile followed by scope and ordering, with zero filler. It is efficient, though the terseness borders on under-specification rather than optimal conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a three-parameter list tool with no output schema, an agent still lacks the default and maximum for limit, the accepted status values (only in the schema property, not the description), and whether results are paginated or truncated. Annotations cover the safety profile, but the operational details an agent needs to call this correctly are missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is only 67%: project and status carry descriptions, but limit has none in either schema or description, and its default value is never stated. The description adds nothing about which parameters exist or how to combine them (e.g. that status accepts a comma-separated list and limit caps at 50), so it does not compensate for the coverage gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names the resource (runs) and scopes it to a single project, with ordering ('newest first'), so the agent can tell it is a listing tool rather than a single-record fetcher like get_run. However, it never states the verb explicitly and gives no contrast against siblings such as get_run_results or get_run_report, leaving differentiation to inference.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no when-to-use guidance, no stated prerequisites, and no named alternative. The phrase 'recent runs' hints at a browsing use case, but nothing tells the agent when to prefer this over list_findings, get_run, or wait_for_run. No explicit exclusion of cases where filtering by status or a single run is more appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
preview_importPreview an API importARead-onlyIdempotent
Read-only (nothing is saved). Parses an OpenAPI, Swagger, Postman, Insomnia, HAR, GraphQL or cURL document, given as text or a URL, and says what an import into the project would add.
| Name | Required | Description | Default |
|---|---|---|---|
| url | No | Or a URL to fetch it from | |
| content | No | The document text | |
| project | Yes | Project ref (e.g. payments-kqzxm) or numeric id |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, openWorldHint and destructiveHint=false, so the safety profile is covered. The description's "nothing is saved" usefully reinforces that no state is mutated and implies network fetching for the URL form, but adds nothing about failure modes, size limits, or partial-parse behavior beyond what structured fields give.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences with zero waste; the most decision-relevant fact (read-only, nothing saved) is front-loaded before the format list and outcome.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only, fully-annotated tool with complete parameter documentation and no output schema, the description covers what it does and what it returns at a summary level. It could be stronger by contrasting with import_spec or noting the response shape, but nothing essential is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema documents url, content and project itself. The description's "given as text or a URL" only restates the two input modes already encoded in the schema, adding no new syntax, exclusivity rule, or format guidance.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (preview/parse) and resource (OpenAPI/Swagger/Postman/Insomnia/HAR/GraphQL/cURL document) and immediately scopes it as non-persisting ("Read-only (nothing is saved)"). The phrase "says what an import into the project would add" cleanly separates it from the sibling import_spec without needing the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The "nothing is saved" framing and "what an import would add" make the use case clear: dry-run before importing. However, it never names the alternative (import_spec) or states an explicit when-not-to-use condition, so the routing to the write-side sibling is left to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
rerun_failedRe-run failed testsA
Re-runs only the failed and errored tests of a finished API run, in place, to confirm a fix. Uses the workspace’s usage allowance (session and month).
| Name | Required | Description | Default |
|---|---|---|---|
| run | Yes | Run ref (e.g. payments-kqzxm) or numeric id |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare the mutation profile (readOnlyHint=false, idempotentHint=false, destructiveHint=false), and the description adds two things they don't carry: the operation is 'in place' (existing run is modified) and it consumes the workspace's session and monthly usage allowance. It does not say what happens to prior results or whether the run becomes reusable afterward.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence that leads with the action and scope, then adds the quota caveat. No filler or restated title content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a one-parameter mutation tool with no output schema, the description covers scope, intent, in-place semantics, and billing impact. Minor gaps remain around return value and effect on the original run's results, but nothing an agent needs to invoke it correctly is absent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only one parameter ('run') and schema description coverage is 100%, so the schema already explains the run-ref-or-numeric-id format. The description adds no parameter-level detail, which is the expected baseline when the schema does the work.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Re-runs') and a precisely scoped resource ('only the failed and errored tests of a finished API run'). The scope words 'only', 'failed and errored', and 'finished' implicitly separate it from start_run, which begins a fresh run.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives a clear triggering context, 'to confirm a fix', so an agent knows when this is the right tool rather than re-starting the run. It does not name start_run as the alternative or state when not to use it, so it stops short of full routing guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
start_runStart a test runA
Starts a run on a project: API tests for API projects, a website scan (or AI exploration) for website projects. Uses the workspace’s usage allowance (session and month). Optionally waits for the result (at most 600 s); otherwise use wait_for_run or get_run.
| Name | Required | Description | Default |
|---|---|---|---|
| kind | No | suite = API tests (default for API projects); frontend_health = website scan (default for website projects) | |
| depth | No | API runs: critical is fastest and cheapest; deep (default) runs every test | |
| project | Yes | Project ref (e.g. payments-kqzxm) or numeric id | |
| max_pages | No | ||
| start_url | No | Website runs: the page to start from | |
| suite_ids | No | ||
| environment | No | Environment id or name; default is the project’s default | |
| wait_seconds | No | Wait for the result up to this long (default 0: return at once) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With annotations already declaring non-read-only, open-world, non-idempotent mutation, the description adds genuinely new behavioral context: the run consumes the workspace's usage allowance (session and month) and the optional wait is capped at 600 s. It stops short of stating what a run costs, whether concurrent runs are allowed, or that a run id is produced.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three tight sentences with the core action front-loaded, then consumption cost, then the wait/hand-off pattern. No filler, though the parenthetical '(or AI exploration)' is the only hint that frontend_explore/frontend_flow kinds exist.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with no output schema and openWorld=true, the description covers what triggers the run, its side effect on quota, and how to retrieve the result via siblings. It leaves the return shape (run id) and permission requirements implicit, but the routing to wait_for_run/get_run covers the main gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 75%, so the schema largely carries the parameter burden. The description maps project type to the suite/frontend_health kinds and bounds wait_seconds, but says nothing about depth, max_pages, start_url, suite_ids, or environment beyond what the schema already documents. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Starts a run on a project') and explains what a run means per project type — API tests, website scan, or AI exploration. It does not, however, distinguish itself from the sibling start_scan, which an agent could easily confuse with the website-scan branch of this tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives explicit routing for result retrieval: 'Optionally waits for the result (at most 600 s); otherwise use wait_for_run or get_run.' The kind/scope defaults per project type are also implied. No exclusion guidance against the sibling start_scan, and no prerequisites (auth/permissions) are stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
start_scanStart a website scanA
Scans a website for broken links, console and network errors, accessibility and speed. Uses the workspace’s website project for that host, creating one if needed. Uses the workspace’s usage allowance (session and month).
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | ||
| max_pages | No | ||
| workspace | No | ||
| wait_seconds | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare non-read-only, non-idempotent, open-world behavior. The description adds meaningful context beyond that: it consumes the workspace's usage allowance (session and month) and auto-creates a project if needed. That is real behavioral disclosure – cost and side effects – though it omits whether the call blocks, returns a run id, or how long it takes.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three tight sentences, front-loaded with what is scanned. Every sentence carries information; nothing is padded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
A non-idempotent, quota-consuming scan tool with no output schema and zero parameter documentation. The description covers side effects and quota, which is valuable, but leaves an agent unable to predict return shape (run handle vs inline results) or the role of wait_seconds/max_pages.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate for all 4 undocumented parameters (url, max_pages, workspace, wait_seconds). It mentions 'that host' and 'workspace' implicitly but explains none of them: no crawl limit semantics for max_pages, no meaning for wait_seconds, no format guidance for url. This is a major gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (scans) and resource (website), and enumerates exactly what it checks: broken links, console and network errors, accessibility, and speed. Clearly distinct from sibling run tools like start_run or rerun_failed, which operate on test runs rather than website scans.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies usage by describing what a scan does and mentioning workspace project resolution, but never says when to prefer this over start_run or how it relates to wait_for_run/get_run_results. No explicit exclusions or alternative routing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
wait_for_runWait for a runARead-onlyIdempotent
Read-only. Waits (at most 600 s) for a queued or running run to finish, then returns its result. Does not start anything.
| Name | Required | Description | Default |
|---|---|---|---|
| run | Yes | Run ref (e.g. payments-kqzxm) or numeric id | |
| timeout_seconds | No | Default 300 |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare read-only and idempotent, but they say nothing about blocking. The description adds the critical behavioral trait that this call blocks up to 600 s and never triggers work, which an agent cannot infer from the annotations. It omits what happens if the wait expires (error vs. partial result), keeping it short of a 5.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three short, front-loaded sentences with zero filler; the blocking constraint and the non-starting guarantee are both stated up front. 'Read-only' slightly duplicates the annotation but costs almost nothing.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, so the description is the only place return behavior could be explained, and 'returns its result' is vague about what that result contains. Similarly, timeout-expiry behavior is unstated. It is adequate for calling the tool but incomplete for interpreting the outcome.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so both parameters (run ref/numeric id, timeout_seconds with default 300 and max 600) are fully documented in the schema. The description restates the 600 s cap but adds no new semantics for the run reference. Baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (waits) and resource (run), plus an explicit negative boundary ('Does not start anything') that separates it from start_run/rerun_failed. It does not explicitly name get_run, the obvious non-blocking sibling, but the blocking semantics make the distinction inferable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies the use case (you want to block on a queued or running run rather than poll) but never states when to prefer this over get_run or list_runs polling. No exclusions or alternatives are given, so usage is only implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
whoamiWho am IARead-onlyIdempotent
Read-only. Shows which Check’d account or workspace API key this server uses, its workspaces and plans.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false, and openWorldHint=false, so 'Read-only' in the description is a restatement rather than added value. It does add the scope of the returned identity (account/workspace API key, workspaces, plans), which is useful context, but it omits any note on auth requirements or throttling. With rich annotations covering the safety profile, this is adequate but thin.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, safety attribute front-loaded, then the substance of what is returned. No filler, nothing an agent has to read twice.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter diagnostic with no output schema, the description is the only place to learn what comes back, and it names it precisely: the account or workspace API key in use, plus its workspaces and plans. Nothing needed to call it correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, so the baseline is 4. The description correctly adds nothing about argument syntax because there is none, and it does not mislead about inputs.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description gives a specific verb and resource: it 'shows which Check'd account or workspace API key this server uses, its workspaces and plans.' That is well beyond restating the name 'whoami' and it clearly distinguishes this tool from the list_*/get_* siblings, which enumerate issues, runs, and findings rather than identity/credential scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no explicit when-to-use statement, no prerequisites, and no named alternatives. The 'whoami' convention makes the purpose inferable and there is no competing tool that overlaps, but the description leaves the agent to infer that this is the right call for verifying which key/account is active. Usage is implied, not stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
19 tool updates
v0.1.0- First observed
cancel_run - First observed
get_finding - First observed
get_issue - First observed
get_run - First observed
get_run_report - First observed
get_run_results - First observed
get_usage - First observed
import_spec - First observed
list_environments - First observed
list_findings - First observed
list_issues - First observed
list_projects - First observed
list_runs - First observed
preview_import - First observed
rerun_failed - First observed
start_run - First observed
start_scan - First observed
wait_for_run - First observed
whoami
TDQS
Scored across 19 tools
Most tools have clearly distinct resources/actions (projects, environments, runs, findings, issues, imports, scans, usage). A few could be confused—start_run vs start_scan on website projects, and list_issues/list_findings/get_issue/get_finding around problem retrieval—but the detailed descriptions explain the differences.
All tool names are snake_case and mostly follow a verb_noun pattern (list_projects, get_run, start_run, cancel_run). Minor deviations such as whoami, wait_for_run, and rerun_failed keep it from perfect consistency.
19 tools is in the heavy 16-25 range for a single integration. While most map to distinct entities, several granular run/finding retrieval tools (get_run_results, get_run_report, list_findings, get_finding) could be consolidated.
Core operational workflows are covered: inspect projects/environments/runs/findings/issues, import specs, start/rerun/cancel runs, scan websites, and check usage. However there are notable gaps in setup/admin—no create/update/delete for projects or environments, and issue/finding tools are read-only—so onboarding a new API project or editing configuration is a dead end.
Maintenance
Related MCP Connectors
Direct access to Cypress tests results and accessibility reports in your AI workflow.
AI-callable tools for API mocking, testing, monitoring, security, and automation.
Create and manage test cases, test runs and results in Testiny from your AI assistant.
Read-only AI coding tools for change verification, release readiness, capacity, and guidance.
Related MCP Servers
AlicenseBqualityAmaintenanceEnables AI assistants to interact with SmartBear's suite of testing and monitoring tools including BugSnag, Swagger, Reflect, and more through natural language, allowing querying of testing data, analysis of performance metrics, and management of test automation.30912,469 npm44MIT- FlicenseAqualityDmaintenanceEnables AI agents to manage Autotest AI projects, screens, test cases, and test runs through natural language commands.10-
- AlicenseNot gradedqualityDmaintenanceProvides AI assistants with seamless access to SmartBear's suite of testing and monitoring tools, enabling querying testing data, analyzing performance metrics, and managing test automation through natural language.12,469 npmMIT
- AlicenseAqualityBmaintenanceLets an assistant read recorded production AI agent runs — steps, model and tool calls with arguments, and outcomes — along with the findings of comparison runs that checked a prompt or model change against those recordings, and verify a signed evidence bundle offline with no account or network. It is strictly read-only: no tool starts a replay or comparison, so nothing can spend money or act without a person deciding.536 PyPIApache 2.0