qa-screens
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@qa-screensrun QA on the homepage and show me what changed"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
qa-screens
An MCP server for visual QA during AI refactoring. It captures pages with Playwright, compares them with reference screenshots using SSIM, and returns the score, the changed regions and a labelled REFERENCE | LIVE preview image, so the agent can see what broke and fix it.
Design conformance: compare the running site with golden or Figma screenshots.
Refactor safety: capture before and after a change and prove it is pixel-neutral.
A/B: compare two running servers, such as
mainand a worktree.Apps behind a login: OAuth (client credentials, password, refresh, authorization code + PKCE), bearer tokens, cookies, sessions, localStorage/sessionStorage and HTTP basic auth.
Self-reporting: significant errors and crashes are filed as GitHub issues automatically, with secrets scrubbed. See Error reporting.
Quick start
No configuration is needed for a first run:
claude mcp add qa-screens -- uvx qa-screensThen ask the agent: "Use qa-screens to make the current homepage the reference, then check it."
The first call downloads Chromium (it starts in the background as soon as the server starts).
update_reference(page="home", confirm=true)screenshotshttp://localhost:8080/home/and saves it asscreenshots/home.png. Append-mobileto the name for the mobile version.run_qacompares the live site with the references from then on.
If your site runs somewhere else, the error message says so. Set base_url in
.qa-screens.json (see Configure) or pass base_url to the tool.
Related MCP server: websight
Install
pip install qa-screens # or: uv tool install qa-screensChromium is downloaded automatically on first use. To do it up front, run
python -m playwright install chromium.
Claude Code
claude mcp add qa-screens -- uvx qa-screensor in a project's .mcp.json:
{
"mcpServers": {
"qa-screens": { "command": "uvx", "args": ["qa-screens"] }
}
}To run the latest main without PyPI, use uvx --from git+https://github.com/astuanax/qa-screens qa-screens.
Copy skills/qa-screenshots into .claude/skills/
so the agent runs QA on its own after every UI change.
Other MCP clients
The server speaks MCP over stdio. The command is qa-screens (or python -m qa_screens),
and it runs in the project directory, or wherever QA_SCREENS_ROOT points.
Configure
Put a .qa-screens.json in the project root. Every key is optional. A mistake in this
file never stops the tools: bad JSON, unknown keys (with a "did you mean" hint) and invalid
values fall back to the defaults, and every tool result lists them under config_warnings
(the CLI prints them as warning: lines).
{
"base_url": "http://localhost:8080",
"references_dir": "qa/screenshots",
"route_template": "/nl-be/{name}/",
"routes": { "home": "/", "nl-be": "/nl-be/" },
"threshold": 0.9,
"mobile_device_scale_factor": 1,
"mask_selectors": [".carousel", "[data-testid=clock]"],
"default_profile": null
}key | default | meaning |
|
| where the site runs (not started by qa-screens) |
|
| reference |
|
| page name → path; |
|
| per-page overrides (a path, or a full URL) |
|
| minimum SSIM to pass |
|
| also fail when more than this share of pixels clearly changed colour ( |
|
| names ending in it are captured in a 390px mobile context |
| 1440×900 / 390×844 | |
|
| set |
|
| elements hidden before capture (dynamic content) |
|
| |
|
| captures, diffs, previews, reports (add it to |
|
| auth profile used when a tool doesn't pass one |
|
| optional Gemini second opinion ( |
The environment variables QA_SCREENS_BASE_URL, QA_SCREENS_REFERENCES_DIR,
QA_SCREENS_ROUTE_TEMPLATE, QA_SCREENS_THRESHOLD, QA_SCREENS_MAX_CHANGED_RATIO and
QA_SCREENS_PROFILE override the file. QA_SCREENS_FIGMA_API changes the Figma API URL.
Tools
tool | purpose |
| effective config, references and their URLs, auth profiles; start here |
| QA all or some pages; returns a JSON summary plus previews of the worst failures |
| QA one page (fast fix loop); |
| screenshot any URL, full page or a single element ( |
| SSIM two image files, for example a Figma export and a capture |
| before/after parity check for refactors |
| compare two running servers page by page |
| make the page's current look its reference: first baseline or accepted redesign (needs |
| download Figma frames as references ( |
| create or update an auth profile: token, OAuth, cookies, headers, storage |
| log in with a real browser (form, SSO or MFA) and save the session |
| OAuth authorization code + PKCE in a browser window |
| import a Playwright |
| check that a profile is logged in (status, redirect, screenshot) |
| list, inspect (never shows secrets) or delete profiles |
| error-reporting status; |
Page results are PASS, FAIL or ERROR. A page fails when its SSIM is below threshold
or more than max_changed_ratio of it clearly changed colour. SSIM is a page-wide average,
so a recoloured header scores about 0.99 and would pass on SSIM alone.
ERROR means the comparison is meaningless:
an HTTP 4xx/5xx, a navigation failure or a missing reference. The agent should fix
the environment, not the CSS.
Authentication for apps
A profile is a named session that is applied to every browser context that uses it:
auth_update_profile(
name="staging",
origins=["https://app.staging.example.com"],
oauth={"grant_type": "client_credentials",
"token_url": "https://idp.example.com/oauth/token",
"client_id": "qa-bot", "client_secret": "env:QA_CLIENT_SECRET",
"audience": "https://api.example.com"},
apply_token_as=["header", "local_storage:access_token"])
run_qa(profile="staging", base_url="https://app.staging.example.com")Secrets: any value can be
"env:VAR". It is resolved from the server's environment at use time, so it never passes through the AI conversation.Scoping: headers and tokens go only to the profile's
origins(default: the base URL's origin), never to CDNs or third parties. Cookies follow normal browser rules.Token placement (
apply_token_as):header(Authorization: Bearer …),header:X-Api-Key,local_storage:<key>,session_storage:<key>,local_storage_json:<key>(the whole token object),cookie:<name>.Refresh: expired tokens are refreshed through
refresh_tokenor by re-running the client-credentials/password grant, including mid-session in the long-lived server.Storage: profiles live in
~/.local/state/qa-screens/profiles/with mode0600, outside the project, so they are never committed.QA_SCREENS_STATE_DIRmoves the location.
Error reporting
qa-screens reports its own bugs, so they reach the maintainers without a manual bug report.
Significant errors (unexpected exceptions in a tool, not user errors like a missing reference or bad credentials) are written to
~/.local/state/qa-screens/reports/pending/and posted in the background as a GitHub issue onastuanax/qa-screens.Crashes: uncaught exceptions are logged to a crash file, and hard crashes (segfault, abort) are captured by
faulthandler. On the next start-up the server scans for these files and posts them.A report stays pending until it has been posted, so reports made offline or without a token are sent later.
De-duplication: each error has a stable fingerprint. A repeat within 24h is not re-posted, and an open issue with the same fingerprint gets a comment instead of a new issue. At most 10 posts per day.
Privacy: tokens, JWTs, cookies, passwords, URL query strings and credentials, and the home directory path are redacted. Tool arguments that hold secrets are dropped entirely.
env var | default | |
|
|
|
|
| where issues go (point it at your fork) |
| falls back to | needs |
|
| API URL, for GitHub Enterprise |
Without a token nothing is posted; reports wait in pending/. qa-screens reports --flush
posts them by hand.
CLI (CI-friendly)
qa-screens # MCP server over stdio (same as `qa-screens serve`)
qa-screens run # QA every reference; exit 1 on FAIL/ERROR
qa-screens run nl-be faq --viewport desktop --base-url http://localhost:8080 --json
qa-screens reports --flush # post pending error reportsA report is written to .qa-screens/runtime/reports/latest.json on every run.
How it works
The reference name maps to a URL (
route_template/routes). Names ending in-mobileuse a mobile context (390px, touch, mobile UA).The page is loaded with the HTTP cache disabled (so the CSS you just edited is what gets measured), with animations, transitions and scrollbars disabled, and after
document.fonts.ready. It is captured full-page.SSIM is computed per colour channel. Grayscale SSIM, as the original tool used, scores a pure hue change at 0.9996. Captures above 12MP are decoded at reduced resolution, so long pages never run out of memory. Different sizes are handled by
align:resize(legacy default),croporpad(height changes count as differences).Separately, pixels whose colour clearly changed are measured on lightly blurred images, so JPEG artefacts and antialiasing don't count. Their share of the page is
changed_ratio.Changed pixels are grouped into region boxes. A red heatmap and a cropped side-by-side preview are written, and the preview is returned to the agent as an image.
Development
uv venv && uv pip install -e ".[dev]"
python -m playwright install chromium
pytestThe tests exercise the server the way an agent does: through an in-process MCP client, against a fake web app (login form, session cookies, OAuth provider with PKCE, basic auth) and fake GitHub and Figma APIs. Assertions are about what those services actually received.
Releases are published to PyPI by GitHub Actions when a v* tag is pushed (trusted publishing).
Links
License
MIT
Available Tools
18 toolsab_compareC
Compare two running servers (e.g. main branch on :8080 vs a worktree on :8091) page by page.
| Name | Required | Description | Default |
|---|---|---|---|
| pages | No | ||
| viewport | No | all | |
| profile_a | No | ||
| profile_b | No | ||
| threshold | No | ||
| base_url_a | Yes | ||
| base_url_b | Yes | ||
| max_images | No | ||
| max_changed_ratio | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure, but it only says 'compare ... page by page.' It does not explain whether comparison uses screenshots, DOM snapshots, or visual diffs; whether it modifies anything; whether auth profiles are required; or what the output format is. Given parameters like threshold, max_images, and max_changed_ratio, the behavior is significantly under-described.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is one sentence with no filler and front-loads the core action. The example is useful and compact. It loses a point because the brevity comes at the expense of necessary operational detail, but as a concise purpose statement it is well structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
This is a 9-parameter tool with no annotations, no output schema, and no parameter-level descriptions, yet the description contains only a single sentence. It omits expected outputs, threshold semantics, image limits, auth profile behavior, and how this differs from sibling comparison tools. The description is too incomplete to support reliable invocation by an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and there are 9 parameters, so the description must compensate for the schema's lack of explanation. It only indirectly references the two base URLs via the example ports. None of pages, viewport, profile_a, profile_b, threshold, max_images, or max_changed_ratio are explained, leaving the agent without enough information to set them correctly.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action ('Compare two running servers') and a clear comparison granularity ('page by page'). The example of main branch on :8080 vs a worktree on :8091 clarifies the intended resource scope. It does not explicitly distinguish itself from sibling tools like compare_images or compare_sets, but the object being compared is clear enough.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The example gives a concrete scenario that implies when the tool should be used: when two live servers need to be compared page by page. However, it does not state when to use this tool instead of alternatives such as compare_images, capture_set, or run_qa, nor does it mention exclusions or prerequisites like server accessibility.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
auth_browser_loginA
Log in with a real browser and save the session (cookies, localStorage and
sessionStorage) into profile name. Works for any login incl. SSO/MFA/OAuth redirects.
headed=true opens a visible window where the user logs in (or finishes MFA); if
username/password are given (use "env:VAR" for secrets) the form is filled first.
Completion is detected by success_url_pattern (regex on the URL), success_selector,
or — if neither is given — by leaving the login page's URL.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | ||
| headed | No | ||
| password | No | ||
| username | No | ||
| login_url | Yes | ||
| timeout_s | No | ||
| submit_selector | No | button[type=submit], input[type=submit] | |
| success_selector | No | ||
| password_selector | No | input[type=password] | |
| username_selector | No | input[type=email], input[name=username], input[name=email], input[autocomplete=username], input[type=text] | |
| success_url_pattern | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full behavioral burden. It discloses the key side effects and behavior: saving cookies/localStorage/sessionStorage, opening a visible window when headed, auto-filling credentials, and the exact completion-detection logic. It does not mention whether an existing profile is overwritten or what happens on failure, but the core behavioral contract is clear.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and front-loaded. The first sentence states the core action and outcome; the following sentences cover interaction mode, credential handling, secrets, and success detection. No redundant words or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For an 11-parameter tool with no annotations and no output schema, the description covers the essential flow, secret handling, and completion criteria. It does not describe the return value or error/edge-case behavior (e.g., profile overwriting, timeout result), but it gives an agent enough to invoke the tool correctly in most cases.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It adds important meaning beyond raw schema: profile `name`, `headed=true` semantics, using 'env:VAR' for secret credentials, and how `success_url_pattern`/`success_selector` drive completion detection. Some parameters like `timeout_s` and the selector parameters are not explicitly explained, though their defaults make them inferable.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb+resource: 'Log in with a real browser and save the session (cookies, localStorage and sessionStorage) into profile `name`.' It also differentiates from sibling auth tools by stating it works for 'any login incl. SSO/MFA/OAuth redirects,' which signals this is the general browser-based login tool rather than a specialized OAuth or import flow.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is clear usage context: it works for any login including SSO/MFA/OAuth, and headed mode is described for user interaction or MFA. It does not explicitly name sibling alternatives or state when to choose auth_oauth_login or auth_import_storage_state instead, so it stops short of full when/when-not guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
auth_checkA
Open url (default: the base URL) with profile name and report whether the session
works: HTTP status, final URL (a redirect to a login page means it expired) and a screenshot.
| Name | Required | Description | Default |
|---|---|---|---|
| url | No | ||
| name | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden and does well by explaining the tool performs a real navigation, reports status, captures a screenshot, and that redirect-to-login means expiration. It does not mention potential side effects like network requests or screenshot storage, but the core behavior is transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One dense sentence covers the action, the default behavior, the parameter meaning, and the success/failure interpretation. No filler or repetition exists.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description adequately explains what the tool returns and how to interpret redirects, which is enough to invoke it. It does not specify output formatting or error behavior for missing profiles, but those are not critical for basic use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description is the only source of parameter meaning. It explains that `url` defaults to the base URL and that `name` selects the auth profile, making both parameters understandable.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific action, opening a URL with a profile and reporting whether the session works, and lists concrete outputs. This clearly distinguishes it from login-related siblings like auth_browser_login or auth_oauth_login.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly conveys that this tool is for validating an existing session by checking HTTP status, final URL, and screenshot. It does not explicitly state when not to use it or mention alternatives, so it stops short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
auth_delete_profileB
Delete an auth profile and its saved session.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full behavioral burden. It clearly discloses the object being destroyed and the side effect of removing the saved session, but it does not add caution about irreversibility, required permissions, or post-deletion consequences.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One short, well-structured sentence that front-loads the destructive verb and includes the key side effect with zero filler. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is low-complexity: one required string parameter and no output schema. The description is sufficient for basic invocation, but a brief note on irreversibility or naming expectations would make it more complete for safe and correct use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only required parameter, `name`, has zero schema description coverage and is never mentioned in the description. The description only vaguely implies that `name` refers to an auth profile, leaving the agent to infer format, existence requirements, or how to discover valid names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a precise action (`Delete`) on a specific resource (`auth profile`) and explicitly names what will be removed (`saved session`). This distinguishes it from sibling auth tools like `auth_update_profile` or `auth_check` by the unique destructive scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives no guidance on when to prefer this tool over alternatives such as `auth_update_profile`, `auth_profiles`, or `auth_import_storage_state`. No prerequisites, exclusions, or conditions for use are mentioned; the intended use is only implied by the verb 'Delete'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
auth_import_storage_stateB
Import a Playwright storage-state JSON (cookies + localStorage), e.g. from a Playwright
test setup's storageState output, into profile name.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | ||
| path | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden, yet it only states that data is imported into a profile. It does not disclose whether the import overwrites/merges existing profile state, whether the profile must already exist, or what side effects occur on the filesystem.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single well-formed sentence that leads with the action and resource, then adds a useful concrete example. No filler or redundant restatement.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple, but no annotations or output schema exist, and the description omits critical operational details like whether existing cookies/localStorage are replaced or merged. It also does not describe expected outcomes or prerequisites, leaving an agent uncertain about side effects.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and no enum/default values exist, so the description must explain both parameters explicitly. It clearly maps `name` to the profile, but `path` is only implied as the source JSON; the description never explicitly states that `path` is a filesystem path to the storage-state JSON.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific verb ('Import'), a specific resource ('Playwright storage-state JSON'), and a clear target ('profile name'). It also clarifies the format as 'cookies + localStorage', which distinguishes it from auth_browser_login and auth_oauth_login without needing to inspect schemas.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The 'e.g. from a Playwright test setup's storageState output' gives a concrete use case, so an agent can infer when this is appropriate. However, it does not explicitly say when to prefer this over sibling auth tools or mention any exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
auth_oauth_loginA
Run the OAuth authorization-code + PKCE flow for profile name (configure oauth with
grant_type "authorization_code", authorize_url, token_url, client_id, redirect_uri first).
The user signs in in the opened window; the redirect is intercepted, the code exchanged,
and the token (with refresh_token if issued) stored and auto-refreshed.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | ||
| headed | No | ||
| timeout_s | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full behavioral burden. It discloses that a window opens, the user signs in, the redirect is intercepted, the code is exchanged, and the token is stored and auto-refreshed. It doesn't mention failure modes or interaction with existing profiles, but the main side effects are clear.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two dense sentences with no filler. The main action and prerequisite are front-loaded, and the flow details are compactly listed. Every clause adds relevant behavioral or setup information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The core OAuth flow is well explained, but there are gaps: no parameter-level guidance for `headed`/`timeout_s`, no comparison with sibling auth tools, and no description of the return value or success/error behavior. It is adequate for a simple call but not fully complete for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It explains the meaning of `name` (the profile) but gives no guidance on `headed` or `timeout_s` beyond the indirect mention of an 'opened window'. The two boolean/integer parameters remain effectively undocumented.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names the specific verb ('Run'), the resource ('OAuth authorization-code + PKCE flow for profile `name`'), and the protocol variant. This clearly separates it from sibling tools like auth_browser_login, which handles browser-based rather than OAuth flow.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives a clear prerequisite (configure oauth with grant_type, authorize_url, token_url, client_id, redirect_uri) and implies this is the OAuth path. However, it never explicitly says when to choose this over sibling auth tools or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
auth_profilesA
List auth profiles, or show one (secrets are never shown).
| Name | Required | Description | Default |
|---|---|---|---|
| name | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description carries the behavioral disclosure burden. The note that 'secrets are never shown' is a valuable behavioral guarantee. However, the description does not disclose the return payload shape, behavior for nonexistent profiles, or explicitly confirm zero side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single compact sentence with the core behavior front-loaded. The parenthetical secrets note earns its place and there is no filler or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a one-parameter read operation, the description is mostly adequate. However, with no output schema, it omits the return shape and any behavior for unknown or missing profile names. The secrets redaction note helps, but some operational uncertainty remains.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. The phrase 'show one' implies that the optional name parameter selects a specific profile, but the description never explicitly states that name is the profile identifier or what happens when it is omitted beyond the implied list behavior.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a clear verb and resource: 'List auth profiles, or show one.' This makes the tool's purpose immediately identifiable and distinguishes it from auth siblings like auth_update_profile, auth_delete_profile, and auth_check, which perform different actions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description says what the tool does but does not specify when to use it over alternatives such as auth_check or auth_update_profile. There are no explicit conditions, exclusions, or guidance about when showing one profile is preferable to other auth operations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
auth_update_profileA
Create or update an auth profile. Only the fields you pass change (dicts are merged).
Any secret may be given as "env:VAR_NAME" so it is read from the server's environment instead of passing through the conversation — prefer that.
origins: origins that receive headers/tokens, e.g. ["https://app.example.com"] (default: the base_url's origin). Credentials are never sent to other origins. access_token/token_type/expires_in/refresh_token: a bearer token you already have. apply_token_as: where the app expects the token — any of "header" (Authorization: Bearer), "header:X-Api-Key", "local_storage:", "session_storage:", "local_storage_json:" (whole token object as JSON), "cookie:". oauth: {grant_type: client_credentials|password|refresh_token|authorization_code, token_url, client_id, client_secret, scope, audience, username, password, authorize_url, redirect_uri, client_auth: post|basic, extra_params}. For client_credentials/password a token is fetched immediately and refreshed automatically. For authorization_code, call auth_oauth_login next. cookies: Playwright cookies [{name, value, url} or {name, value, domain, path}]. local_storage/session_storage: {origin: {key: value}} injected before page scripts run. http_credentials: {username, password} for HTTP basic auth. clear: field names to remove, e.g. ["token", "cookies", "storage_state"].
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | ||
| clear | No | ||
| oauth | No | ||
| cookies | No | ||
| headers | No | ||
| origins | No | ||
| expires_in | No | ||
| token_type | No | ||
| access_token | No | ||
| local_storage | No | ||
| refresh_token | No | ||
| apply_token_as | No | ||
| session_storage | No | ||
| http_credentials | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Since no annotations are provided, the description carries the full behavioral burden. It discloses key behaviors: merging semantics, env: variable handling, security note about credentials not being sent to other origins, and auto-refresh for client_credentials/password. It also explains the destructive 'clear' field. While not exhaustive, it covers the most important behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Despite the length, every sentence contributes necessary detail. The format uses line breaks to separate parameter groups, keeping it scannable. The opening sentence establishes purpose, then each line adds a specific explanation. No filler or redundancy; it is appropriately sized for the tool's complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 14 parameters and no output schema, the description is remarkably complete. It covers all major parameters, provides examples, defaults, security guidance, and next-step instructions for OAuth flows. It explains side effects and merging behavior. Nothing critical is missing for an agent to call it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description must fully compensate. It does so admirably: it explains origins with an example and default, enumerates apply_token_as values, details the oauth object structure and flows, specifies cookie formats, and clarifies injection points for local/session storage. This goes far beyond the bare schema property names and adds practical usage context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Create or update an auth profile,' which is a specific verb+resource statement. It distinguishes itself from siblings like auth_profiles (presumably listing/reading), auth_delete_profile (deletion), and auth_browser_login (interactive login) by clearly indicating its scope is profile creation/update.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives some contextual guidance (e.g., 'For authorization_code, call auth_oauth_login next'), but it never explicitly states when to use this tool versus alternatives like auth_browser_login or auth_oauth_login for login flows. It implies usage by describing what fields do, but lacks explicit 'use when X, not when Y' guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
captureA
Screenshot any URL (desktop or mobile viewport, optionally logged in via profile).
clip_selector captures a single element. Saved under runtime/captures/.png.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | ||
| name | No | ||
| profile | No | ||
| viewport | No | desktop | |
| full_page | No | ||
| return_image | No | ||
| clip_selector | No | ||
| wait_for_selector | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full behavioral disclosure burden. It does usefully disclose that screenshots are saved to runtime/captures/<name>.png and that clip_selector captures a single element, but it does not mention overwrite behavior, authentication requirements beyond 'optionally logged in via profile', or what return_image controls.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with no filler. The primary action is front-loaded, and the key scoping details (viewport, profile, element clipping, output path) are compactly included. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 8 parameters, no annotations, and no output schema, the description is adequate but leaves gaps. It gives enough to make a basic capture call and understand the save location, but an agent would not know the behavioral implications of return_image, wait_for_selector, or full_page without inspecting their titles.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate for 8 parameters. It adds meaning for viewport ('desktop or mobile'), profile ('optionally logged in'), clip_selector ('captures a single element'), and name (via the save path), but leaves full_page, return_image, and wait_for_selector semantically unexplained.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Screenshot any URL'. This clearly identifies the tool's core function and distinguishes it from sibling tools like capture_set or compare_images, which operate on sets or comparisons rather than a single URL capture.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives no explicit guidance on when to use this tool versus alternatives such as capture_set, qa_page, or compare_images. It mentions optional login and element clipping but does not state the conditions that would make this tool preferable over a sibling.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
capture_setA
Capture a named set of pages (e.g. label="before" before a refactor, "after" after it).
Pages default to all reference names; any names work if you pass pages and the route
mapping resolves them. Use compare_sets afterwards. Proves a refactor is pixel-neutral
without depending on golden references.
| Name | Required | Description | Default |
|---|---|---|---|
| label | Yes | ||
| pages | No | ||
| profile | No | ||
| base_url | No | ||
| viewport | No | all |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses that pages default to all reference names, that specific pages work if the route mapping resolves them, and the outcome it proves. This adds significant behavioral context beyond the schema, though it does not mention return format, side effects, or requirements. Given the lack of annotations, this is reasonably transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, three sentences, and front-loads the main purpose. It includes a concrete example and flows logically. It is not overly verbose, though it could have added a bit more structure, but it's effective and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given five parameters with zero schema descriptions, no annotations, and no output schema, the description is incomplete. It explains only label and pages, leaving profile, base_url, and viewport unexplained. It also does not describe what the tool returns or any side effects. An agent would need more information to use it confidently.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It explains the label parameter (via the example) and pages (default behavior and condition), but it does not explain profile, base_url, or viewport at all. With five parameters and no schema descriptions, covering only two is insufficient, leaving the agent guessing about the rest.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a clear verb and resource: 'Capture a named set of pages' with a label. It provides an example use case (label='before'/'after' around a refactor) and distinguishes it from other tools by implying it captures a set under a label rather than a single capture. However, it does not explicitly name sibling tools to differentiate, though it references compare_sets, so it's clear enough.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives clear usage context: 'Use compare_sets afterwards' and the purpose 'Proves a refactor is pixel-neutral without depending on golden references.' This tells the agent when to use it (refactor verification). It does not explicitly state when not to use it or mention alternatives, but the context is clear and actionable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
compare_imagesA
SSIM-compare two image files (e.g. a Figma export vs a capture). Paths may be relative to the project root. Returns score, changed regions, a heatmap path and an a|b preview.
| Name | Required | Description | Default |
|---|---|---|---|
| align | No | pad | |
| image_a | Yes | ||
| image_b | Yes | ||
| threshold | No | ||
| max_changed_ratio | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It explicitly lists the return values: score, changed regions, a heatmap path, and an a/b preview, and clarifies that paths may be relative to the project root. It does not detail side effects like file creation, but this is a reasonable disclosure for a comparison tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with no filler. It front-loads the purpose, gives a concrete example, and then condenses path semantics and output behavior into one efficient second sentence.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description provides a solid starting point for basic use: it names the core purpose, example inputs, and all return values. However, with no output schema and no parameter explanation for three of the five parameters, an agent cannot fully understand how to control the comparison or interpret edge cases.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate, but it only addresses the input image paths. The optional parameters align, threshold, and max_changed_ratio are completely unexplained in both the schema and the description, leaving an agent without enough information to set them meaningfully.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: 'SSIM-compare two image files', with a concrete example ('Figma export vs a capture'). This clearly differentiates it from siblings like compare_sets and ab_compare by focusing on a single pair of image files.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The example of comparing a Figma export to a capture implies a primary use case and gives helpful context. However, it does not explicitly state when to prefer this tool over alternatives such as compare_sets or ab_compare, nor does it mention any exclusion criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
compare_setsB
Diff two capture sets (labels from capture_set, or directories). identical: true
means every page scored >= threshold. Returns previews of the most-changed pages.
| Name | Required | Description | Default |
|---|---|---|---|
| after | No | after | |
| align | No | crop | |
| before | No | before | |
| threshold | No | ||
| max_images | No | ||
| max_changed_ratio | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses the threshold-based meaning of 'identical: true' and states that previews of most-changed pages are returned. With no annotations provided, the description carries the burden for behavioral disclosure, but it does not mention side effects, authentication requirements, or resource implications.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three short, focused sentences with no filler. The core purpose is front-loaded, and each sentence adds distinct, useful information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 6-parameter tool with no output schema and no annotations, the description is not complete enough for reliable invocation. The semantics of several parameters and the exact output format remain undocumented, leaving meaningful gaps for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate by explaining parameters. It provides some meaning for before/after (labels or directories) and threshold (used in the identical condition), but align, max_images, and max_changed_ratio are left unexplained, making it difficult to configure calls beyond defaults.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Diff') and resource ('two capture sets'), and clarifies that the sets can be labels from capture_set or directories. It distinguishes the tool from image-level comparison tools at a general level, though it does not explicitly name sibling alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The intended use case is implied: compare capture sets and identify page-level differences. However, there is no explicit guidance about when to use this tool versus siblings like compare_images or ab_compare, nor any stated exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
error_reportsA
Show automatic error-reporting status (pending/reported crash and error reports, target repo). flush=true posts pending reports to GitHub now.
| Name | Required | Description | Default |
|---|---|---|---|
| flush | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the behavioral disclosure burden. It clearly discloses the key side effect: flush=true posts pending reports to GitHub. It also implies a read-only status mode via 'Show', though it does not detail credentials, failure behavior, or whether reports are removed after posting.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two compact sentences cover both modes and the report scope without wasted words. The primary status-checking purpose is front-loaded, with the side-effect flag described immediately after.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a one-parameter tool with no output schema, the description provides sufficient context: purpose, flush behavior, and what kind of reports are involved. It could be slightly more explicit about what the status response looks like or what happens after a successful flush, but this is a minor gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate for the flush parameter. It does so effectively by explaining exactly what flush=true does, which the bare schema property name does not convey. The default behavior is also inferable from the first sentence.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific action and resource: 'Show automatic error-reporting status' and flush behavior. It also clarifies what that status covers (pending/reported crash/error reports and target repo), which makes the tool's role unambiguous among the unrelated sibling tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly establishes the two usage modes: inspect status by default, or pass flush=true to post pending reports to GitHub. While it does not explicitly name alternatives or exclusions, the intended context is clear and no competing error-reporting sibling exists.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
qa_configA
Show the effective configuration (base URL, reference dir, route mapping), the reference screenshots found and the available auth profiles. Start here.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool 'shows' configuration and profiles, implying a read-only operation, but it does not explicitly confirm no side effects, require authentication, or describe response format. The lack of annotation coverage means a 3 is appropriate; the description provides basic transparency but not rich detail.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with no wasted words. The primary action and content are front-loaded, and the final 'Start here' adds a clear directive without repetition. It is efficient and structured effectively.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter informational tool, the description is largely complete. It lists all the key pieces of information the tool provides. It does not explicitly state the output format or that it is read-only, but these are minor gaps given the tool's simplicity and the absence of an output schema. The description sufficiently equips an agent to call it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the description is not required to explain parameter behavior. The baseline for zero parameters is 4, and the description appropriately focuses on what the tool returns rather than inputs. No additional parameter semantics are needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function with a specific verb 'Show' and a resource: effective configuration, listing specific components (base URL, reference dir, route mapping, screenshots, auth profiles). It also adds the phrase 'Start here,' which establishes its role as an entry point, distinguishing it from action-oriented siblings like run_qa or capture.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description includes the explicit guidance 'Start here,' indicating this tool should be used first to understand the current setup. While it doesn't explicitly name alternatives or exclusions, the 'Start here' instruction effectively signals when to use it, and the sibling list contains no other config-display tool, so usage context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
qa_pageB
QA a single page against its reference (fast iteration while fixing).
url overrides the route mapping for this call. Returns the result and, on failure, a
reference|live side-by-side preview cropped around the changed regions.
| Name | Required | Description | Default |
|---|---|---|---|
| url | No | ||
| page | Yes | ||
| align | No | resize | |
| profile | No | ||
| base_url | No | ||
| threshold | No | ||
| max_changed_ratio | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the burden of explaining behavior. It discloses the return value, failure behavior, and the side-by-side preview cropped around changed regions. It also clarifies the `url` override behavior, which adds meaningful context beyond the bare schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact, front-loaded with the main purpose, and every sentence adds useful information. The failure-preview behavior is placed in the second sentence without unnecessary filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 7 parameters, no output schema, and no annotations, the description is too sparse for reliable invocation. It provides a good high-level purpose and some output behavior, but the missing parameter semantics leave significant gaps for an agent deciding how to call the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate for parameter meaning. It only explains `url`, leaving `page`, `align`, `profile`, `base_url`, `threshold`, and `max_changed_ratio` unexplained. The required `page` parameter is especially critical but receives no clarification.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states the core purpose: 'QA a single page against its reference' with a clear verb, resource, and scope. It distinguishes itself from batch QA tools by emphasizing 'single page' and 'fast iteration while fixing', though it does not explicitly name a sibling alternative.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'fast iteration while fixing' implies the intended usage context, and the note about `url` overriding route mapping gives some operational guidance. However, it does not explicitly state when to use this tool versus alternatives like run_qa or capture.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
run_qaA
Capture pages from the running site and SSIM-compare them with the reference screenshots.
pages: reference names (file names without extension); default all references.
viewport: all | desktop | mobile (names ending in "-mobile" use a 390px mobile context).
profile: auth profile for pages behind a login.
align: how to handle different image sizes — resize (default, legacy), crop, or pad
(pad makes page-height changes count as differences).
A page FAILs when SSIM < threshold OR more than max_changed_ratio of its pixels clearly
changed colour (catches local changes such as a recoloured header that SSIM averages away).
Returns a JSON summary (failures first, with changed regions in px) plus side-by-side
reference|live preview images of up to max_images failing pages.
| Name | Required | Description | Default |
|---|---|---|---|
| align | No | resize | |
| pages | No | ||
| profile | No | ||
| base_url | No | ||
| viewport | No | all | |
| threshold | No | ||
| max_images | No | ||
| max_changed_ratio | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It explains the failure criteria in detail, including the nuanced 'max_changed_ratio' condition that catches local color changes SSIM averages away, and describes the return format (JSON summary with changed regions, preview images). It does not mention side effects, rate limits, or prerequisites like needing reference screenshots to exist, but the core runtime behavior is well disclosed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured: a one-sentence purpose statement, parameter explanations, behavior logic, and output summary. It is information-dense and front-loaded, with every line contributing useful detail. It could be slightly tighter, but the structure is clear and appropriate for the tool's complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (8 parameters, no output schema, no annotations), the description covers the operational behavior, failure semantics, and return values in substantial detail. The main gaps are the missing base_url semantics and no indication of where reference screenshots come from or how they are managed. Still, the description provides enough for an agent to call the tool correctly in most intended scenarios.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate for all 8 parameters. It provides meaningful semantics for pages, viewport, profile, align, threshold, max_changed_ratio, and max_images, but base_url is completely absent from the description, leaving one parameter undocumented. This partial compensation warrants a mid-range score rather than a higher one.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The opening sentence states a specific action ('Capture pages from the running site') and a specific resource ('reference screenshots') with a clear comparison method ('SSIM-compare'). It is clear what the tool does, but it does not explicitly differentiate itself from siblings like capture_set, compare_sets, or compare_images, so it falls short of full sibling distinction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides parameter-level guidance (e.g., when to use profile for login-protected pages, how align changes behavior) but never says when to choose this tool over alternatives such as compare_sets or capture_set. There are no explicit exclusions or alternative recommendations, so the agent is left to infer the appropriate selection context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sync_figmaA
Download Figma frames into the references directory (needs FIGMA_TOKEN). Frame names become reference names; overwrites existing references with the same name.
| Name | Required | Description | Default |
|---|---|---|---|
| frame | No | ||
| file_key | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full responsibility for behavioral disclosure. It explicitly reveals a network/auth prerequisite (FIGMA_TOKEN) and warns that existing references with the same name are overwritten—a key destructive side-effect. It doesn't cover return format or rate limits, but the critical behaviors are disclosed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, front-loaded with the action, and every clause carries meaningful detail: the download, the token requirement, naming behavior, and overwrite side-effect. No filler or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with 2 parameters and no output schema, the description covers the core action, prerequisite, and overwrite behavior. However, it omits what happens when 'frame' is null (presumably all frames) and how 'file_key' maps to the Figma file, which is needed for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description needed to compensate by explaining 'frame' and 'file_key'. It clarifies that frame names become reference names, but it does not explain that 'frame' is an optional filter or what 'file_key' identifies. This leaves a significant gap for an agent trying to set parameters correctly.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action ('Download'), a clear resource ('Figma frames'), and a destination ('references directory'), making the tool's function immediately obvious. It also differentiates from siblings like capture or update_reference by indicating a Figma source, and the overwrite note adds specificity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a clear context for use—syncing Figma frames into the references directory—and notes the FIGMA_TOKEN prerequisite. However, it does not explicitly state when to prefer this tool over siblings like capture or update_reference, nor does it mention exclusions or alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
update_referenceA
Make the current look of a page its reference screenshot: from source (an image),
else the latest qa_page capture, else a fresh capture right now (so this also creates a
first baseline for a page that has no reference yet). The old reference is backed up.
Only do this when the USER has confirmed the look is intended; set confirm=true.
| Name | Required | Description | Default |
|---|---|---|---|
| page | Yes | ||
| source | No | ||
| confirm | No | ||
| profile | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full behavioral disclosure burden. It does well by explaining side effects: the old reference is backed up, a fresh capture may be taken, and a first baseline can be created for pages lacking a reference. It does not mention profile/auth requirements, return behavior, or failure cases, so it is not fully exhaustive, but the core mutation behavior and safety mechanism are transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense but front-loaded with the primary purpose, and every clause adds information: fallback order, baseline creation, backup behavior, and confirmation requirement. It is somewhat run-on and could be split into clearer sentences, but there is no fluff or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutating tool with no annotations and no output schema, the description covers the essential operational context: what the tool does, how the source is resolved, that the old reference is preserved, and when confirmation is required. The main gap is the unexplained profile parameter, which may matter in an auth-aware environment, but the core decision-making information is present.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate for the schema's silence. It adds real meaning for source (explicitly an image) and confirm (must be true after user confirmation), and page is implied. However, the profile parameter is never explained, leaving one of four parameters semantically undocumented.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb and resource: "Make the current look of a page its reference screenshot." It goes beyond a generic statement by specifying the fallback chain (source image → latest qa_page capture → fresh capture), which clearly distinguishes it from sibling tools like capture or qa_page. An agent can immediately understand what this tool changes and when it is invoked.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives a clear usage condition: "Only do this when the USER has confirmed the look is intended; set confirm=true." This tells the agent when it is appropriate to invoke the tool and how to express that precondition through the confirm parameter. It does not explicitly name alternative sibling tools or state when NOT to use it, but the context and fallback logic strongly imply its specialized role.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
18 tool updates
v0.1.0- First observed
ab_compare - First observed
auth_browser_login - First observed
auth_check - First observed
auth_delete_profile - First observed
auth_import_storage_state - First observed
auth_oauth_login - First observed
auth_profiles - First observed
auth_update_profile - First observed
capture - First observed
capture_set - First observed
compare_images - First observed
compare_sets - First observed
error_reports - First observed
qa_config - First observed
qa_page - First observed
run_qa - First observed
sync_figma - First observed
update_reference
TDQS
Scored across 18 tools
Most tools have clearly distinct purposes: run_qa vs qa_page, capture vs capture_set, and compare_images vs compare_sets vs ab_compare are separated by scope. The main ambiguity is between auth_browser_login and auth_oauth_login, though their descriptions clarify the different login flows.
Tool names follow a mostly predictable snake_case style, with useful prefixes like qa_ and auth_. However, the pattern is not strictly verb_noun: capture and ab_compare stand out, and names like qa_config or error_reports are noun-style rather than action-first.
18 tools is on the heavy side, especially considering the 7-tool auth sub-system alongside the core screenshot/compare workflow. It is not excessive in absolute terms, but it crosses into the range where the set starts to feel bloated.
The core visual QA lifecycle is well covered: references, captures, comparisons, set diffs, and running-server diffs are all present, and auth profile management is thorough. Minor gaps exist such as no direct tool for deleting references or listing capture sets, but these can be worked around.
Maintenance
Related MCP Connectors
Capture screenshots, detect visual regressions between page versions, and analyze with AI.
Browser-based QA for AI-built software. Test pages with real browsers via agents.
Pixel-perfect webpage screenshots rendered in a real browser, full-page or viewport, via one POST.
Desktop and mobile website screenshots plus page context for AI agents and automation workflows.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceEnables AI assistants to capture website screenshots, automate browser interactions, and manage recurring screenshot configurations across 150+ global locations. It also supports AI-powered domain research and visual change monitoring for any web page.16 npmMIT
- AlicenseNot gradedqualityDmaintenanceEnables AI agents to see, analyze, and visually verify web page changes through pixel-perfect diffing, theme extraction, layout analysis, and interactive element detection.19 npmMIT
- AlicenseAqualityBmaintenanceScreenshot, visual-diff, and AI page-analysis API for AI agents. Capture any URL as PNG, JPEG, WebP, PDF, or HTML, diff two versions of a page to catch visual regressions, and get an AI summary of what a page contains.344 npm1MIT
- FlicenseAqualityCmaintenanceEnables AI-assisted visual regression testing by capturing baselines, comparing screenshots, and using AI to judge whether visual differences are bugs or acceptable changes.4-