The VisualQ MCP Server provides a comprehensive QA and visual testing platform for web projects, covering visual/functional regression testing, audits, run monitoring, and CI integration.
Project & Scenario Management
List projects, check setup health (identify blockers/warnings), list VRT scenarios (labels, URLs, viewports), and get full scenario configuration details.
Visual Regression Testing (VRT)
Run VRT comparisons against baselines, capture new baselines, list failed scenarios with mismatch percentages, get detailed diff stats, explain VRT failures with structured analysis and recommended actions, approve results to promote to baselines (confirm: true), and create comparison rules (ignore, dynamic, layout-only) (confirm: true).
Functional Regression Testing (FRT)
Run FRT feature test suites (confirm: true).
Run Monitoring
Get run status/summary, poll for run completion, view run history, and compare metrics between two runs.
Quality & Audit Reports
Get composite quality scores with pillar breakdowns, plus detailed performance, SEO, accessibility, and analytics tracking plan reports.
GitHub / CI Integration
Post or update VRT summary comments on GitHub PRs (confirm: true) to gate pull requests based on quality.
Note: All mutating actions (approving results, creating rules, running FRT, posting PR comments) require explicit confirm: true.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@VisualQ MCP Server Run visual regression test on the login page"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
MCP server for VisualQ — the Quality OS agent interface. @visualq/mcp exposes 44 tools (qa profile): multi-pillar audit, VRT/FRT (frt_get_feature), tracking plan + JIRA proof, rolling health. For Cursor, Claude Desktop, or any MCP client.
Sign in to https://visualq.ai
Go to Settings → Agent API Keys (org admin)
Create a key with scope mcp_full
Optionally set a default project slug if you mostly work on one client
Copy the Cursor MCP config snippet
Paste into ~/.cursor/mcp.json (or project .cursor/mcp.json) and restart Cursor
Or one command:
npx @visualq/setup-agent cursor --key vq_org_live_… --project my-siteOr install the Cursor plugin (skills + rules bundled).
{
"mcpServers": {
"visualq": {
"command": "npx",
"args": ["-y", "@visualq/mcp"],
"env": {
"VISUALQ_API_KEY": "vq_org_live_…",
"VISUALQ_BASE_URL": "https://visualq.ai",
"VISUALQ_TOOL_PROFILE": "qa"
}
}
}
}If your org key has a default project (or you set it only in MCP env):
"VISUALQ_DEFAULT_PROJECT": "afp-com"Then tools can omit project when the key has defaultProject on the server or this env var is set in the MCP config.
Pass project on every tool call (slug or id), e.g. "project": "afp-com".
Related MCP server: Artillery MCP Server
Variable | Required | Default | Description |
| yes | — | Org agent key |
| no |
| VisualQ instance (use your origin for self-hosted) |
| no | — | Default project slug injected into tool args |
| no |
|
|
| no | — | Set |
| no |
| HTTP mode port |
Scope | MCP read | MCP write ( | CI |
| yes | no | no |
| yes | yes | no |
| no | no | yes |
Org agent keys support mcp_read and mcp_full only.
Quality MCP PR gate: gate_pr_quality → fix with explain_vrt_failure / frt_heal_step_def → run_vrt or run_full_audit
Onboard a site: create_project → crawl_site → create_scenario → run_baseline → create_frt_scenario → run_frt_feature
Full QA: run_full_audit (optional pillars[]) → wait_for_run → gate_pr_quality → get_site_health
Pillar-only audit: run_full_audit with pillars: ["a11y"], ["tracking"], etc.
Pre-merge VRT: list_scenarios → run_vrt → get_run_failures → explain_vrt_failure
FRT in CI: GitHub Action type: frt or visualq frt --api-key …
Jira tracking proof: read ticket → tracking_prove_jira_ticket with confirm: true (semantic intent → generic linked scenario) → paste jiraMarkdown into JIRA.
Jira-driven QA: read ticket → create_scenario / create_frt_scenario with ticket id in name → run tests
Mutating tools require confirm: true in arguments.
For GitHub Actions / Jenkins, use a project-scoped key (vq_live_…) with scope ci — not the org agent key.
{
"mcpServers": {
"visualq": {
"command": "npx",
"args": ["-y", "@visualq/mcp"],
"env": {
"VISUALQ_API_KEY": "vq_live_…",
"VISUALQ_BASE_URL": "https://visualq.ai"
}
}
}
}Create in Project → Settings → API keys.
VisualQ also exposes JSON invoke (not stdio MCP wire protocol):
GET https://visualq.ai/api/mcp — tool catalog (public)
POST https://visualq.ai/api/mcp/v1/invoke — X-API-Key + { "tool", "args" }
The @visualq/mcp npm package is the supported IDE integration path.
npm install
npm run build
VISUALQ_API_KEY=vq_org_live_… VISUALQ_BASE_URL=http://localhost:3000 node dist/index.jsSync tool manifest from the visualq backend (sibling repo):
cd ../visualq && npm run mcp:export-manifest
cd ../visualq-mcp && npm run sync-manifestBefore publishing to npm:
cd ../visualq && npm run mcp:export-manifest
cd ../visualq-mcp && npm test && npm run build && npm run sync-manifest
npm publish --access public
# Optional: publish agent skills if skills/ changed
cd packages/agent-skills && npm publish --access publicPackage: abecms/visualq-mcp
Backend: abecms/visualq (/api/mcp/v1/invoke)
Latest accessibility audit report.
| Name | Required | Description | Default |
|---|---|---|---|
| project | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description should disclose behavioral traits. It states 'Latest accessibility audit report' but does not mention that it is read-only, whether it requires a prior audit run, or if it returns data or a file.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise but at the cost of essential information. It is a single phrase that provides no operational details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one optional parameter, no output schema), the description still fails to clarify the report's content, format, or when it is available.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description must add meaning to parameters. It does not mention the 'project' parameter, its purpose, or how it affects the result.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Latest accessibility audit report' is a noun phrase that implies retrieval but lacks a verb specifying the action (e.g., 'get', 'retrieve'). It does not differentiate from sibling tools that also retrieve reports or data.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like get_quality_score or get_run_failures. There is no mention of prerequisites or context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Approve failed VRT results and promote them to baselines. Requires confirm: true and explicit scenario list.
| Name | Required | Description | Default |
|---|---|---|---|
| project | No | ||
| environment | No | ||
| runId | No | ||
| scenarios | No | ||
| viewports | No | ||
| browsers | No | ||
| confirm | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, description partially reveals behavior: it approves and promotes, requires confirm. However, it does not disclose side effects (e.g., overwriting baselines), permission requirements, or rate limits. Adequate but incomplete.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with purpose and key requirements. Efficient but could benefit from structured listing of parameters.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite high parameter count, no output schema, and no annotations, the description does not explain how to specify scenarios, what confirm does beyond being true, or the workflow context. Incomplete for safe usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 7 parameters with 0% description coverage. Description only adds meaning for 'confirm' and 'scenarios', leaving project, environment, runId, viewports, browsers unexplained. Does not compensate for schema gaps.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the verb 'approve', the resource 'failed VRT results', and the outcome 'promote them to baselines'. It distinguishes from sibling tools like run_vrt or explain_vrt_failure by specifying this is an approval action.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description mentions requirements (confirm: true, explicit scenario list) but does not provide explicit guidance on when to use this tool versus alternatives like compare_runs or get_run_failures. Usage context is implied but not fully clarified.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Project setup blockers and warnings before running audits.
| Name | Required | Description | Default |
|---|---|---|---|
| project | No | ||
| includeEnvironments | No | ||
| includeIntegrations | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; the description only states it returns blockers and warnings. It does not disclose whether the tool is read-only, has side effects, requires authentication, or any rate limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
While concise, the single sentence is too sparse for a tool with 3 optional parameters. Important information is missing, making it under-specified rather than efficiently structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (3 parameters, no output schema, no annotations), the description is completely inadequate. It fails to explain what constitutes a blocker or warning, parameter usage, or output format.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 3 parameters with 0% description coverage. The description does not mention or explain any parameters, leaving their purpose and valid values entirely unspecified.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly indicates the tool checks for setup blockers and warnings, with the context of being a pre-audit check. It uses specific terms ('blockers', 'warnings') and distinguishes from sibling audit/report tools by focusing on setup health rather than results.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. The description implies use before audits but does not specify scenarios, prerequisites, or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Compare metrics between two runs.
| Name | Required | Description | Default |
|---|---|---|---|
| project | No | ||
| runIdA | No | ||
| runIdB | No | ||
| pillar | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It implies a read-only operation but offers no details on what 'compare metrics' entails (e.g., output format, performance implications, or whether it modifies state).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise at one sentence, which is efficient but results in under-specification. Lacks structure like parameter details or usage context.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given four parameters and no output schema, the description provides minimal context. Does not mention return value, required/optional parameters, or how to interpret results.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% - no parameter descriptions in schema. Description does not explain any of the four parameters (project, runIdA, runIdB, pillar) beyond their names, leaving the agent unable to determine correct values.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action (compare) and resource (metrics between two runs). It distinguishes from sibling tools which focus on single runs or other operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives. No mention of prerequisites or scenarios where comparison might fail or be inappropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Add a VRT comparison rule (ignore, dynamic, layout_only) on a scenario. Requires confirm: true.
| Name | Required | Description | Default |
|---|---|---|---|
| project | No | ||
| scenarioLabel | No | ||
| selector | No | ||
| mode | No | ||
| reason | No | ||
| confirm | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden. It only discloses the requirement for `confirm: true` but fails to mention other behavioral aspects such as whether the operation is destructive, idempotent, or any side effects. This is insufficient for a creation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short and to the point, with no redundant information. However, it is too sparse for the complexity of the tool, sacrificing necessary detail for brevity. A better balance is needed.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 6 parameters, no output schema, and the complexity of setting comparison rules (e.g., valid values for mode, how selector works), the description is far from complete. It provides only minimal context, leaving agents likely to misuse the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero descriptions and 6 parameters. The description only adds meaning for 'mode' (by listing rule types) and hints at 'confirm: true'. Other parameters (project, scenarioLabel, selector, reason) remain unexplained, leaving significant gaps.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Add a VRT comparison rule') and the target ('on a scenario'). It also lists the types of rules. However, it does not explicitly distinguish this tool from sibling tools, though no sibling appears to duplicate this function.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions 'Requires confirm: true', which is a usage condition, but provides no guidance on when to use this tool versus alternatives or any prerequisites. No when/not directives are given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Structured VRT failure analysis with hypotheses and next actions.
| Name | Required | Description | Default |
|---|---|---|---|
| project | No | ||
| environment | No | ||
| scenarioLabel | No | ||
| viewport | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
There are no annotations, so the description must fully disclose behavior. It indicates the tool generates hypotheses and next actions, but it does not state if it is read-only, requires authentication, has rate limits, or modifies any state. The description provides minimal behavioral insight beyond the broad output nature.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence that is highly concise and front-loaded with the key action. It avoids fluff and is easy to parse, though it could benefit from a brief breakdown of parameters or output.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of 4 parameters, no annotations, no output schema, and many sibling tools, the description is incomplete. It does not explain the role of each parameter, the expected output format, or how this tool integrates into the VRT workflow. The agent is left with an ambiguous understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 4 parameters with zero descriptions (0% coverage), and the tool description mentions none of them. The agent must infer that 'project', 'environment', 'scenarioLabel', and 'viewport' likely identify a specific VRT failure, but no semantic help is given. This is a significant gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool does 'Structured VRT failure analysis with hypotheses and next actions,' which effectively communicates its purpose and distinguishes it from sibling tools like 'get_run_failures' (retrieval) and 'compare_runs' (comparison). The use of 'structured' and 'hypotheses/next actions' adds valuable specificity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus sibling alternatives. With many related tools (e.g., 'get_run_failures', 'get_diff_stats', 'check_setup_health'), the agent lacks context about prerequisites or typical workflow, such as whether to call this after 'get_run_failures' or 'get_scenario_details'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Detailed diff stats for one scenario from the latest VRT run.
| Name | Required | Description | Default |
|---|---|---|---|
| project | No | ||
| environment | No | ||
| scenarioLabel | No | ||
| viewport | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It states the tool retrieves stats from the 'latest VRT run,' implying a dependency on a recent run, but does not mention whether it is read-only, requires authentication, or any side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, concise but lacking detail. It omits essential information about parameters and output, so it is not optimally structured for comprehensive guidance.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, no parameter descriptions, and no annotations, the tool description fails to provide sufficient context. An agent cannot determine what the returned stats look like, how to construct a request, or any constraints (e.g., valid values for parameters).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has four parameters (project, environment, scenarioLabel, viewport) with 0% schema description coverage. The description does not explain any parameter meaning, format, or usage, leaving the agent with no help beyond the parameter names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool returns 'detailed diff stats for one scenario from the latest VRT run,' identifying the specific resource (diff stats), scope (one scenario), and context (latest VRT run). However, it does not differentiate from siblings like 'get_scenario_details' that might also provide scenario information.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as 'get_scenario_details' or 'get_run_failures.' There is no mention of prerequisites (e.g., a VRT run must exist) or any exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Composite quality score and pillar breakdown for the project.
| Name | Required | Description | Default |
|---|---|---|---|
| project | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must fully disclose behavioral traits. It only states the tool returns a score and breakdown, but omits details such as whether it modifies state, requires authentication, has rate limits, or provides real-time vs cached data.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
While the description is short, it is too vague and lacks structure. The single sentence fails to provide necessary details, making it under-specified rather than concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the absence of output schema, the description should explain the return value structure. It mentions 'composite quality score and pillar breakdown' but no details on format, fields, or nesting. The tool is incomplete for an agent to use effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, and the tool description adds no extra meaning about the 'project' parameter. It does not explain what constitutes a valid project, format, or any constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that it returns a 'composite quality score and pillar breakdown' for a project, which distinguishes it from sibling tools that focus on other aspects like diff stats, run status, or VRT failures. However, it could be more explicit about what 'pillar breakdown' entails.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. There is no mention of prerequisites, context, or situations where this tool is appropriate or inappropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List failed scenarios with mismatch % from a VRT run (batch, no images).
| Name | Required | Description | Default |
|---|---|---|---|
| project | No | ||
| environment | No | ||
| runId | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavior. It notes 'batch, no images' but fails to mention side effects, auth needs, rate limits, or return format. It implies read-only but does not confirm.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise (one sentence, 12 words) and front-loaded with essential info. However, it could include brief parameter hints without sacrificing brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 3 undocumented parameters, no output schema, and no annotations, the description is insufficient. It omits parameter roles, expected output, and usage context beyond the basic purpose.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not mention any parameters or their meanings. The description adds no value beyond the schema's property names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('List'), the resource ('failed scenarios with mismatch %'), and context ('from a VRT run, batch, no images'), distinguishing it from sibling tools like get_scenario_details or get_diff_stats.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use or when not to use this tool. It does not mention alternatives or prerequisites, leaving the agent to infer usage from context alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent VRT and audit runs for the project.
| Name | Required | Description | Default |
|---|---|---|---|
| project | No | ||
| environment | No | ||
| limit | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description must disclose behavioral traits. It says 'recent' but does not define what that means, nor does it indicate the tool is read-only, or mention any side effects or required permissions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short and to the point, but it lacks structure and does not add enough value per sentence. It is concise but under-specified.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has three optional parameters and no output schema, the description is incomplete. It does not explain output format, how parameters affect results, or what 'recent' means.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, and the description provides no information about any of the three parameters (project, environment, limit). The agent cannot infer parameter semantics from the description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves recent VRT and audit runs for the project. The verb 'get' and resource 'runs' are specific, and it distinguishes from siblings that focus on single runs or comparisons.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like get_run_status or compare_runs. The description does not mention exclusions or context for usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get status and summary for a run.
| Name | Required | Description | Default |
|---|---|---|---|
| runId | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description only states the basic read operation without disclosing idempotency, error handling, or state changes.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single, front-loaded sentence with no wasted words, though could be slightly more informative about return structure.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one parameter and no output schema, description provides minimal context but omits return format and error conditions.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and description does not explain the 'runId' parameter beyond its name, failing to compensate for the low coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the verb 'Get' and the resource 'status and summary for a run', distinguishing it from sibling tools like 'get_run_failures' and 'get_run_history'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as 'wait_for_run' or 'get_run_history'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Full configuration for one VRT scenario.
| Name | Required | Description | Default |
|---|---|---|---|
| project | No | ||
| scenarioLabel | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description alone must disclose behavioral traits. It only says 'Full configuration', not specifying whether the operation is read-only, what fields are returned, or if it requires authentication. The term 'configuration' implies a detailed object, but no details on structure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no redundancy. It is front-loaded with the key purpose. However, it could be slightly more structured without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the minimal schema, no output schema, and no annotations, the description is vastly incomplete. It lacks information expected for a configuration retrieval tool: output format, required vs optional parameters, or example usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, yet the description adds no meaning to the two parameters (project, scenarioLabel). It doesn't explain their roles, formats, or how they identify the scenario, forcing the agent to rely solely on parameter names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Full configuration for one VRT scenario' clearly indicates the tool returns configuration details for a specific VRT scenario. It distinguishes from siblings like list_scenarios (which likely lists scenario names) by specifying 'full configuration' for a single scenario.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., list_scenarios, get_run_details). No context on prerequisites or exclusions, leaving the agent to infer when to call this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List projects accessible with the current credentials.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must assume full responsibility for behavioral disclosure. It mentions listing projects with current credentials but does not describe any additional behavior such as sorting, filtering, or pagination. For a parameterless tool, this is minimal but adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence with no unnecessary words. It is front-loaded with the key action and resource.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (no parameters, no output schema, no annotations), the description is fairly complete. It could hint at the return type or format, but for a listing tool, the basic function is clear.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has 0 parameters and schema coverage is 100%, so the description has no need to add parameter meaning. According to guidelines, a baseline of 4 is appropriate for 0 parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'list' and the resource 'projects', making the tool's function unambiguous. It is distinct from all sibling tools, which focus on reports, runs, scenarios, etc.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description indicates usage context ('accessible with the current credentials'), providing clear guidance on when to use the tool. However, it does not explicitly mention exclusions or alternative tools, but given the simplicity, this is acceptable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List VRT scenarios (labels, URLs, viewports) for the project.
| Name | Required | Description | Default |
|---|---|---|---|
| project | No | ||
| environment | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description does not disclose behavioral traits such as read-only nature, pagination, or side effects. The listing operation is implied but not confirmed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One concise sentence that front-loads the action. Could be improved by breaking into bullet points or adding structure, but overall efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite low complexity, the description lacks critical information: no output schema, no parameter details, no mention of required inputs (though none required), and no description of what a 'scenario' entails.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the description does not clarify the purpose or constraints of the 'project' and 'environment' parameters. The phrase 'for the project' only hints at one parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's action ('List') and its resource ('VRT scenarios'), and specifies the returned elements (labels, URLs, viewports). It distinguishes itself from sibling tools like 'get_scenario_details'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. No explicit 'when not to use' or context about prerequisites like project existence.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Latest performance audit report.
| Name | Required | Description | Default |
|---|---|---|---|
| project | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. It only states 'Latest performance audit report' without any information about side effects, read-only nature, or required permissions. This is misleadingly minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely short, which could be considered concise, but it is under-specified. It fails to provide necessary context, making it inadequate for effective tool use.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one optional parameter, no output schema), the description should still explain the purpose and parameter. It does not, making it incomplete for reliable agent invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has one parameter ('project') with 0% schema description coverage. The description does not mention the parameter at all, leaving its meaning and required format completely unknown.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Latest performance audit report' indicates the subject but uses a noun phrase rather than a full verb+resource. It vaguely distinguishes from siblings like a11y_get_report, but it's not explicit about the action (get/retrieve).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No usage guidance provided. The description does not state when to use this tool versus alternatives such as a11y_get_report or seo_get_report, nor are exclusions mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Post or update a VisualQ VRT summary comment on a GitHub PR (requires GitHub integration).
| Name | Required | Description | Default |
|---|---|---|---|
| project | No | ||
| prNumber | No | ||
| runId | No | ||
| environment | No | ||
| body | No | ||
| confirm | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. It states the tool can post or update a comment, but fails to explain key behaviors such as how updates work (e.g., upsert logic), required permissions, rate limits, or side effects. The six parameters remain completely undocumented, leaving the agent blind to their role.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence of 12 words, which is concise and front-loads the core action. However, it sacrifices necessary detail for brevity; while efficient, it leaves out parameter descriptions and usage guidance. Still, no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (6 parameters, no schema descriptions, no annotations, no output schema), the description is far from complete. It only covers the high-level purpose. Critical details about parameters, required inputs, and return behavior are missing, making it inadequate for an agent to use correctly without additional context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, meaning no parameter descriptions exist. The description does not compensate: it provides no information about what each of the six parameters ('project', 'prNumber', 'runId', 'environment', 'body', 'confirm') means or how they should be used. The agent cannot infer correct parameter usage from this minimal description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Post or update a VisualQ VRT summary comment on a GitHub PR'. It specifies the verb (post/update), resource (VisualQ VRT summary comment), and context (GitHub PR). Among sibling tools, no other tool explicitly posts or updates PR comments, so it distinguishes well.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions 'requires GitHub integration' as a prerequisite but does not provide explicit guidance on when to use this tool versus alternatives, or when not to use it. Without sibling comparison or usage context, the agent has limited decision support.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Capture new VRT baselines.
| Name | Required | Description | Default |
|---|---|---|---|
| project | No | ||
| environment | No | ||
| scenarios | No | ||
| browsers | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description bears full responsibility for disclosing behavioral traits. It fails to mention whether capturing baselines is destructive (overwrites existing), requires authentication, or any side effects. The single sentence offers no behavioral insights beyond the basic action.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
While extremely concise (one short sentence), the description is under-specified. Conciseness should not come at the cost of missing critical information. It fails to earn its place by leaving out essential details for correct invocation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (4 parameters, no output schema, no annotations), the description is grossly incomplete. It does not explain return values, side effects, prerequisites, or how to interpret results. The agent cannot safely invoke this tool based on this description alone.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the description does not explain any of the four parameters (project, environment, scenarios, browsers). Without parameter details, the agent cannot correctly fill in values. The description adds zero value beyond the schema's type definitions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Capture new VRT baselines' clearly states the tool's purpose with a specific verb ('capture') and resource ('VRT baselines'). It distinguishes from siblings like 'run_vrt' which likely runs tests. However, it doesn't explain what 'baselines' are, leaving some ambiguity for unfamiliar users.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like 'run_vrt' or 'explain_vrt_failure'. The description lacks context on prerequisites or appropriate scenarios, forcing the agent to infer usage from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Run an FRT feature test suite. Requires confirm: true.
| Name | Required | Description | Default |
|---|---|---|---|
| project | No | ||
| environment | No | ||
| featureId | No | ||
| scenarioFilter | No | ||
| browsers | No | ||
| confirm | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses the behavioral requirement of setting 'confirm' to true, but lacks details on side effects, permissions, or execution consequences, leaving critical behavioral aspects unclear.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short (two sentences) and front-loads the purpose, achieving efficiency. However, it is somewhat under-specified, though not overly verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 6 parameters with zero schema description, no output schema, and no annotations, the description is grossly incomplete. It fails to explain what FRT is, how to use the tool effectively, or what results to expect.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, and the description only adds that 'confirm' must be true. No explanation is given for the other five parameters (project, environment, featureId, etc.), leaving their purpose ambiguous.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Run' and the resource 'FRT feature test suite', making the primary action identifiable. However, it does not distinguish this tool from similar sibling tools like 'run_baseline' or 'run_vrt', which could lead to confusion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions the requirement 'confirm: true' but provides no guidance on when to use this tool versus alternatives, nor any context about prerequisites or typical use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Run a VRT comparison against baselines.
| Name | Required | Description | Default |
|---|---|---|---|
| project | No | ||
| environment | No | ||
| scenarios | No | ||
| browsers | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description must disclose behavioral traits. It only states the action without revealing what happens during execution (e.g., synchronous/asynchronous, required inputs, side effects, return values). Minimal transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, which is concise but too terse. It sacrifices necessary context for brevity. Every sentence should earn its place, but this one omits critical information, making it under-specified rather than appropriately sized.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (4 parameters, no output schema, no annotations) and the presence of many sibling tools, the description is severely incomplete. It does not explain what the tool returns, how to fill parameters, or how it fits into the workflow. Very inadequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 4 parameters with 0% description coverage and no enums. The description adds no information about the parameters' purposes, valid values, or formats. It is entirely uninformative for parameter understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action and resource: 'Run a VRT comparison against baselines.' It specifies that it runs a VRT (Visual Regression Testing) comparison, which differentiates it from siblings like 'run_baseline' that create baselines. However, the acronym VRT may not be universally understood, and it doesn't explicitly distinguish from 'compare_runs', which might have similar functionality.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'run_baseline' or 'compare_runs'. There are no prerequisites mentioned (e.g., baselines must exist), nor any context about ideal scenarios for its use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Latest SEO audit report.
| Name | Required | Description | Default |
|---|---|---|---|
| project | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description bears full responsibility for behavioral transparency. It does not disclose whether the tool is read-only, what happens if no report exists, or any other behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
While extremely concise, the description is under-specified and fails to convey sufficient information. Conciseness should not sacrifice utility.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of annotations, output schema, and parameter descriptions, the description is woefully incomplete. It does not explain what the report contains, how to use the project parameter, or any execution details.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single parameter 'project' has 0% schema description coverage and the description does not explain its purpose or usage. The description adds no meaning beyond the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Latest SEO audit report.' is a tautology of the tool name and fails to specify the action or resource clearly. It does not differentiate from sibling tools like a11y_get_report or perf_get_latest_report.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. There is no mention of context, prerequisites, or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Analytics tracking plan for the project.
| Name | Required | Description | Default |
|---|---|---|---|
| project | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden to disclose behavioral traits. It fails to mention any behavior such as whether the tool is read-only, required permissions, error handling, or what happens when the project parameter is omitted or invalid.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, short sentence which is under-specified for a tool with one parameter and no annotations. It is too brief to be useful.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple structure (1 optional string parameter, no output schema), the description is incomplete. It does not describe the return value or any contextual behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, yet the description does not explain the only parameter 'project'. It neither defines what constitutes a project nor provides any guidance on values or formats.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Analytics tracking plan for the project' states the resource but lacks an explicit verb; the tool name supplies 'get'. It is clear that it retrieves a tracking plan for a project, but it does not distinguish from sibling tools, resulting in moderate clarity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus any of the 17 sibling tools. The description offers no context about usage scenarios, prerequisites, or alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Poll until a run completes or times out.
| Name | Required | Description | Default |
|---|---|---|---|
| runId | No | ||
| maxWaitMs | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose all behavioral traits. It only states the polling loop and timeout condition, omitting details like error handling, return values, polling interval, and whether the operation blocks.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence and immediately states the purpose, but it is too brief, sacrificing important details that would fit concisely.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the absence of annotations and output schema, the description fails to cover essential aspects: parameter semantics, timeout behavior, required fields, and return format. It leaves significant gaps for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not explain the parameters runId and maxWaitMs (e.g., format, units, required status). The agent receives no semantic guidance beyond property names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly conveys the tool's action ('poll') and its termination condition ('run completes or times out'), distinguishing it from sibling tools like get_run_status which return current status without waiting.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as get_run_status or run_baseline. The description does not mention exclusions or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v0.1.1approve_vrt_resultscheck_setup_health2 fields changedInput schema / properties / includeEnvironmentsAdded value: +{
+ "type": "boolean"
+}Input schema / properties / includeIntegrationsAdded value: +{
+ "type": "boolean"
+}create_comparison_rulepost_pr_commentrun_frt_featurev0.1.0a11y_get_reportcheck_setup_healthcompare_runsexplain_vrt_failureget_diff_statsget_quality_scoreget_run_failuresget_run_historyget_run_statusget_scenario_detailslist_projectslist_scenariosperf_get_latest_reportrun_baselinerun_vrtseo_get_reporttracking_get_planwait_for_runScored across 22 tools
Most tools target distinct resources and actions: scenario listing/detail, run execution, failure/diff extraction, audit reports, approval, rules, and PR posting. A few pairs could be confused, such as wait_for_run vs get_run_status and run_baseline vs approve_vrt_results, but descriptions and return shapes generally separate them. Overall, the boundaries are clear enough for an agent to select correctly in most cases.
The dominant pattern is verb_noun, e.g., list_projects, run_vrt, get_run_history, approve_vrt_results, and post_pr_comment. The audit/tracking tools break this slightly with prefix_get style like perf_get_latest_report and tracking_get_plan, and wait_for_run uses a phrasal verb. These are minor deviations rather than a chaotic mix.
22 tools is on the heavy side, especially with some functional overlap such as wait_for_run wrapping get_run_status polling and get_run_failures vs get_diff_stats both reporting VRT diff details. The broad scope across VRT, FRT, audits, tracking, setup, and PR integration explains the count, but trimming convenience/overlapping tools would make the surface feel tighter. It is borderline, not grossly excessive.
The VRT lifecycle is well covered: scenarios, run, status, failures, diffs, analysis, approval, baseline capture, comparison rules, and PR comments. Gaps include no scenario create/update/delete, no run trigger for perf/SEO/a11y audits despite check_setup_health mentioning audits, and no cancel-run or historical audit report retrieval. These are workaroundable if configuration and audit scheduling are managed externally, but they are real gaps.
MCP server for visual regression testing: triage a PR's UI diffs from your coding agent.
Run, debug, and triage tests from your IDE using natural language, no dashboard switching, no manual data transfers. The TestMu AI (formerly LambdaTest) MCP Server is a single remote server exposing four tool suites: HyperExecute — analyze your project, generate YAML configs and test runner commands, then monitor jobs and sessions. Automation — pull a TestID's details plus command, network, and console logs into one chat for instant root-cause analysis. Includes mobile app upload. SmartUI — explain pixel, layout, DOM, and perceptual changes in a visual regression run, with context-aware React/HTML/CSS fixes. Accessibility — audit any public URL or a local React app against WCAG and get ready-to-apply remediation steps. Connects over https://mcp.lambdatest.com/mcp using OAuth 2.1 — no API keys in your config. One-click install in Cursor; works with Claude, GitHub Copilot, Cline, and any MCP client. Tests execute on the TestMu AI cloud: 3,000+ browsers and 10,000+ real devices.
Direct access to Cypress tests results and accessibility reports in your AI workflow.
Capture screenshots, detect visual regressions between page versions, and analyze with AI.
