Skip to main content
Glama

qa-screens

An MCP server for visual QA during AI refactoring. It captures pages with Playwright, compares them with reference screenshots using SSIM, and returns the score, the changed regions and a labelled REFERENCE | LIVE preview image, so the agent can see what broke and fix it.

  • Design conformance: compare the running site with golden or Figma screenshots.

  • Refactor safety: capture before and after a change and prove it is pixel-neutral.

  • A/B: compare two running servers, such as main and a worktree.

  • Apps behind a login: OAuth (client credentials, password, refresh, authorization code + PKCE), bearer tokens, cookies, sessions, localStorage/sessionStorage and HTTP basic auth.

  • Self-reporting: significant errors and crashes are filed as GitHub issues automatically, with secrets scrubbed. See Error reporting.

Quick start

No configuration is needed for a first run:

claude mcp add qa-screens -- uvx qa-screens

Then ask the agent: "Use qa-screens to make the current homepage the reference, then check it."

  1. The first call downloads Chromium (it starts in the background as soon as the server starts).

  2. update_reference(page="home", confirm=true) screenshots http://localhost:8080/home/ and saves it as screenshots/home.png. Append -mobile to the name for the mobile version.

  3. run_qa compares the live site with the references from then on.

If your site runs somewhere else, the error message says so. Set base_url in .qa-screens.json (see Configure) or pass base_url to the tool.

Related MCP server: websight

Install

pip install qa-screens          # or: uv tool install qa-screens

Chromium is downloaded automatically on first use. To do it up front, run python -m playwright install chromium.

Claude Code

claude mcp add qa-screens -- uvx qa-screens

or in a project's .mcp.json:

{
  "mcpServers": {
    "qa-screens": { "command": "uvx", "args": ["qa-screens"] }
  }
}

To run the latest main without PyPI, use uvx --from git+https://github.com/astuanax/qa-screens qa-screens.

Copy skills/qa-screenshots into .claude/skills/ so the agent runs QA on its own after every UI change.

Other MCP clients

The server speaks MCP over stdio. The command is qa-screens (or python -m qa_screens), and it runs in the project directory, or wherever QA_SCREENS_ROOT points.

Configure

Put a .qa-screens.json in the project root. Every key is optional. A mistake in this file never stops the tools: bad JSON, unknown keys (with a "did you mean" hint) and invalid values fall back to the defaults, and every tool result lists them under config_warnings (the CLI prints them as warning: lines).

{
  "base_url": "http://localhost:8080",
  "references_dir": "qa/screenshots",
  "route_template": "/nl-be/{name}/",
  "routes": { "home": "/", "nl-be": "/nl-be/" },
  "threshold": 0.9,
  "mobile_device_scale_factor": 1,
  "mask_selectors": [".carousel", "[data-testid=clock]"],
  "default_profile": null
}

key

default

meaning

base_url

http://localhost:8080

where the site runs (not started by qa-screens)

references_dir

screenshots

reference .png/.jpg files; the file name is the page name

route_template

/{name}/

page name → path; {name} excludes the -mobile suffix

routes

{}

per-page overrides (a path, or a full URL)

threshold

0.90

minimum SSIM to pass

max_changed_ratio

0.02

also fail when more than this share of pixels clearly changed colour (1 disables)

mobile_suffix

-mobile

names ending in it are captured in a 390px mobile context

desktop_viewport / mobile_viewport

1440×900 / 390×844

mobile_device_scale_factor

2

set 1 if mobile references are 390px wide

mask_selectors

[]

elements hidden before capture (dynamic content)

wait_until, navigation_timeout_ms

networkidle, 30000

runtime_dir

.qa-screens/runtime

captures, diffs, previews, reports (add it to .gitignore)

default_profile

null

auth profile used when a tool doesn't pass one

ai_critique, gemini_model

false

optional Gemini second opinion (pip install qa-screens[ai], GEMINI_API_KEY)

The environment variables QA_SCREENS_BASE_URL, QA_SCREENS_REFERENCES_DIR, QA_SCREENS_ROUTE_TEMPLATE, QA_SCREENS_THRESHOLD, QA_SCREENS_MAX_CHANGED_RATIO and QA_SCREENS_PROFILE override the file. QA_SCREENS_FIGMA_API changes the Figma API URL.

Tools

tool

purpose

qa_config

effective config, references and their URLs, auth profiles; start here

run_qa

QA all or some pages; returns a JSON summary plus previews of the worst failures

qa_page

QA one page (fast fix loop); url overrides the mapping

capture

screenshot any URL, full page or a single element (clip_selector)

compare_images

SSIM two image files, for example a Figma export and a capture

capture_set / compare_sets

before/after parity check for refactors

ab_compare

compare two running servers page by page

update_reference

make the page's current look its reference: first baseline or accepted redesign (needs confirm=true, keeps a backup, refuses error pages)

sync_figma

download Figma frames as references (FIGMA_TOKEN)

auth_update_profile

create or update an auth profile: token, OAuth, cookies, headers, storage

auth_browser_login

log in with a real browser (form, SSO or MFA) and save the session

auth_oauth_login

OAuth authorization code + PKCE in a browser window

auth_import_storage_state

import a Playwright storageState JSON

auth_check

check that a profile is logged in (status, redirect, screenshot)

auth_profiles / auth_delete_profile

list, inspect (never shows secrets) or delete profiles

error_reports

error-reporting status; flush=true posts pending reports

Page results are PASS, FAIL or ERROR. A page fails when its SSIM is below threshold or more than max_changed_ratio of it clearly changed colour. SSIM is a page-wide average, so a recoloured header scores about 0.99 and would pass on SSIM alone. ERROR means the comparison is meaningless: an HTTP 4xx/5xx, a navigation failure or a missing reference. The agent should fix the environment, not the CSS.

Authentication for apps

A profile is a named session that is applied to every browser context that uses it:

auth_update_profile(
  name="staging",
  origins=["https://app.staging.example.com"],
  oauth={"grant_type": "client_credentials",
         "token_url": "https://idp.example.com/oauth/token",
         "client_id": "qa-bot", "client_secret": "env:QA_CLIENT_SECRET",
         "audience": "https://api.example.com"},
  apply_token_as=["header", "local_storage:access_token"])

run_qa(profile="staging", base_url="https://app.staging.example.com")
  • Secrets: any value can be "env:VAR". It is resolved from the server's environment at use time, so it never passes through the AI conversation.

  • Scoping: headers and tokens go only to the profile's origins (default: the base URL's origin), never to CDNs or third parties. Cookies follow normal browser rules.

  • Token placement (apply_token_as): header (Authorization: Bearer …), header:X-Api-Key, local_storage:<key>, session_storage:<key>, local_storage_json:<key> (the whole token object), cookie:<name>.

  • Refresh: expired tokens are refreshed through refresh_token or by re-running the client-credentials/password grant, including mid-session in the long-lived server.

  • Storage: profiles live in ~/.local/state/qa-screens/profiles/ with mode 0600, outside the project, so they are never committed. QA_SCREENS_STATE_DIR moves the location.

Error reporting

qa-screens reports its own bugs, so they reach the maintainers without a manual bug report.

  • Significant errors (unexpected exceptions in a tool, not user errors like a missing reference or bad credentials) are written to ~/.local/state/qa-screens/reports/pending/ and posted in the background as a GitHub issue on astuanax/qa-screens.

  • Crashes: uncaught exceptions are logged to a crash file, and hard crashes (segfault, abort) are captured by faulthandler. On the next start-up the server scans for these files and posts them.

  • A report stays pending until it has been posted, so reports made offline or without a token are sent later.

  • De-duplication: each error has a stable fingerprint. A repeat within 24h is not re-posted, and an open issue with the same fingerprint gets a comment instead of a new issue. At most 10 posts per day.

  • Privacy: tokens, JWTs, cookies, passwords, URL query strings and credentials, and the home directory path are redacted. Tool arguments that hold secrets are dropped entirely.

env var

default

QA_SCREENS_ERROR_REPORTING

on

on, local (write files, never post) or off

QA_SCREENS_ISSUE_REPO

astuanax/qa-screens

where issues go (point it at your fork)

QA_SCREENS_GITHUB_TOKEN

falls back to GITHUB_TOKEN, then gh auth token

needs issues:write

QA_SCREENS_GITHUB_API

https://api.github.com

API URL, for GitHub Enterprise

Without a token nothing is posted; reports wait in pending/. qa-screens reports --flush posts them by hand.

CLI (CI-friendly)

qa-screens                       # MCP server over stdio (same as `qa-screens serve`)
qa-screens run                   # QA every reference; exit 1 on FAIL/ERROR
qa-screens run nl-be faq --viewport desktop --base-url http://localhost:8080 --json
qa-screens reports --flush       # post pending error reports

A report is written to .qa-screens/runtime/reports/latest.json on every run.

How it works

  1. The reference name maps to a URL (route_template / routes). Names ending in -mobile use a mobile context (390px, touch, mobile UA).

  2. The page is loaded with the HTTP cache disabled (so the CSS you just edited is what gets measured), with animations, transitions and scrollbars disabled, and after document.fonts.ready. It is captured full-page.

  3. SSIM is computed per colour channel. Grayscale SSIM, as the original tool used, scores a pure hue change at 0.9996. Captures above 12MP are decoded at reduced resolution, so long pages never run out of memory. Different sizes are handled by align: resize (legacy default), crop or pad (height changes count as differences).

  4. Separately, pixels whose colour clearly changed are measured on lightly blurred images, so JPEG artefacts and antialiasing don't count. Their share of the page is changed_ratio.

  5. Changed pixels are grouped into region boxes. A red heatmap and a cropped side-by-side preview are written, and the preview is returned to the agent as an image.

Development

uv venv && uv pip install -e ".[dev]"
python -m playwright install chromium
pytest

The tests exercise the server the way an agent does: through an in-process MCP client, against a fake web app (login form, session cookies, OAuth provider with PKCE, basic auth) and fake GitHub and Figma APIs. Assertions are about what those services actually received.

Releases are published to PyPI by GitHub Actions when a v* tag is pushed (trusted publishing).

License

MIT

Available Tools

18 tools
ab_compareC

Compare two running servers (e.g. main branch on :8080 vs a worktree on :8091) page by page.

ParametersJSON Schema
NameRequiredDescriptionDefault
pagesNo
viewportNoall
profile_aNo
profile_bNo
thresholdNo
base_url_aYes
base_url_bYes
max_imagesNo
max_changed_ratioNo

TDQS

C2.8/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure, but it only says 'compare ... page by page.' It does not explain whether comparison uses screenshots, DOM snapshots, or visual diffs; whether it modifies anything; whether auth profiles are required; or what the output format is. Given parameters like threshold, max_images, and max_changed_ratio, the behavior is significantly under-described.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is one sentence with no filler and front-loads the core action. The example is useful and compact. It loses a point because the brevity comes at the expense of necessary operational detail, but as a concise purpose statement it is well structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

This is a 9-parameter tool with no annotations, no output schema, and no parameter-level descriptions, yet the description contains only a single sentence. It omits expected outputs, threshold semantics, image limits, auth profile behavior, and how this differs from sibling comparison tools. The description is too incomplete to support reliable invocation by an agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% and there are 9 parameters, so the description must compensate for the schema's lack of explanation. It only indirectly references the two base URLs via the example ports. None of pages, viewport, profile_a, profile_b, threshold, max_images, or max_changed_ratio are explained, leaving the agent without enough information to set them correctly.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action ('Compare two running servers') and a clear comparison granularity ('page by page'). The example of main branch on :8080 vs a worktree on :8091 clarifies the intended resource scope. It does not explicitly distinguish itself from sibling tools like compare_images or compare_sets, but the object being compared is clear enough.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The example gives a concrete scenario that implies when the tool should be used: when two live servers need to be compared page by page. However, it does not state when to use this tool instead of alternatives such as compare_images, capture_set, or run_qa, nor does it mention exclusions or prerequisites like server accessibility.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

auth_browser_loginA

Log in with a real browser and save the session (cookies, localStorage and sessionStorage) into profile name. Works for any login incl. SSO/MFA/OAuth redirects.

headed=true opens a visible window where the user logs in (or finishes MFA); if username/password are given (use "env:VAR" for secrets) the form is filled first. Completion is detected by success_url_pattern (regex on the URL), success_selector, or — if neither is given — by leaving the login page's URL.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYes
headedNo
passwordNo
usernameNo
login_urlYes
timeout_sNo
submit_selectorNobutton[type=submit], input[type=submit]
success_selectorNo
password_selectorNoinput[type=password]
username_selectorNoinput[type=email], input[name=username], input[name=email], input[autocomplete=username], input[type=text]
success_url_patternNo

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full behavioral burden. It discloses the key side effects and behavior: saving cookies/localStorage/sessionStorage, opening a visible window when headed, auto-filling credentials, and the exact completion-detection logic. It does not mention whether an existing profile is overwritten or what happens on failure, but the core behavioral contract is clear.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured and front-loaded. The first sentence states the core action and outcome; the following sentences cover interaction mode, credential handling, secrets, and success detection. No redundant words or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For an 11-parameter tool with no annotations and no output schema, the description covers the essential flow, secret handling, and completion criteria. It does not describe the return value or error/edge-case behavior (e.g., profile overwriting, timeout result), but it gives an agent enough to invoke the tool correctly in most cases.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It adds important meaning beyond raw schema: profile `name`, `headed=true` semantics, using 'env:VAR' for secret credentials, and how `success_url_pattern`/`success_selector` drive completion detection. Some parameters like `timeout_s` and the selector parameters are not explicitly explained, though their defaults make them inferable.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb+resource: 'Log in with a real browser and save the session (cookies, localStorage and sessionStorage) into profile `name`.' It also differentiates from sibling auth tools by stating it works for 'any login incl. SSO/MFA/OAuth redirects,' which signals this is the general browser-based login tool rather than a specialized OAuth or import flow.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is clear usage context: it works for any login including SSO/MFA/OAuth, and headed mode is described for user interaction or MFA. It does not explicitly name sibling alternatives or state when to choose auth_oauth_login or auth_import_storage_state instead, so it stops short of full when/when-not guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

auth_checkA

Open url (default: the base URL) with profile name and report whether the session works: HTTP status, final URL (a redirect to a login page means it expired) and a screenshot.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlNo
nameYes

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden and does well by explaining the tool performs a real navigation, reports status, captures a screenshot, and that redirect-to-login means expiration. It does not mention potential side effects like network requests or screenshot storage, but the core behavior is transparent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One dense sentence covers the action, the default behavior, the parameter meaning, and the success/failure interpretation. No filler or repetition exists.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description adequately explains what the tool returns and how to interpret redirects, which is enough to invoke it. It does not specify output formatting or error behavior for missing profiles, but those are not critical for basic use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description is the only source of parameter meaning. It explains that `url` defaults to the base URL and that `name` selects the auth profile, making both parameters understandable.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific action, opening a URL with a profile and reporting whether the session works, and lists concrete outputs. This clearly distinguishes it from login-related siblings like auth_browser_login or auth_oauth_login.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly conveys that this tool is for validating an existing session by checking HTTP status, final URL, and screenshot. It does not explicitly state when not to use it or mention alternatives, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

auth_delete_profileB

Delete an auth profile and its saved session.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYes

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full behavioral burden. It clearly discloses the object being destroyed and the side effect of removing the saved session, but it does not add caution about irreversibility, required permissions, or post-deletion consequences.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One short, well-structured sentence that front-loads the destructive verb and includes the key side effect with zero filler. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is low-complexity: one required string parameter and no output schema. The description is sufficient for basic invocation, but a brief note on irreversibility or naming expectations would make it more complete for safe and correct use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The only required parameter, `name`, has zero schema description coverage and is never mentioned in the description. The description only vaguely implies that `name` refers to an auth profile, leaving the agent to infer format, existence requirements, or how to discover valid names.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a precise action (`Delete`) on a specific resource (`auth profile`) and explicitly names what will be removed (`saved session`). This distinguishes it from sibling auth tools like `auth_update_profile` or `auth_check` by the unique destructive scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives no guidance on when to prefer this tool over alternatives such as `auth_update_profile`, `auth_profiles`, or `auth_import_storage_state`. No prerequisites, exclusions, or conditions for use are mentioned; the intended use is only implied by the verb 'Delete'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

auth_import_storage_stateB

Import a Playwright storage-state JSON (cookies + localStorage), e.g. from a Playwright test setup's storageState output, into profile name.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYes
pathYes

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden, yet it only states that data is imported into a profile. It does not disclose whether the import overwrites/merges existing profile state, whether the profile must already exist, or what side effects occur on the filesystem.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single well-formed sentence that leads with the action and resource, then adds a useful concrete example. No filler or redundant restatement.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple, but no annotations or output schema exist, and the description omits critical operational details like whether existing cookies/localStorage are replaced or merged. It also does not describe expected outcomes or prerequisites, leaving an agent uncertain about side effects.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% and no enum/default values exist, so the description must explain both parameters explicitly. It clearly maps `name` to the profile, but `path` is only implied as the source JSON; the description never explicitly states that `path` is a filesystem path to the storage-state JSON.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb ('Import'), a specific resource ('Playwright storage-state JSON'), and a clear target ('profile name'). It also clarifies the format as 'cookies + localStorage', which distinguishes it from auth_browser_login and auth_oauth_login without needing to inspect schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'e.g. from a Playwright test setup's storageState output' gives a concrete use case, so an agent can infer when this is appropriate. However, it does not explicitly say when to prefer this over sibling auth tools or mention any exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

auth_oauth_loginA

Run the OAuth authorization-code + PKCE flow for profile name (configure oauth with grant_type "authorization_code", authorize_url, token_url, client_id, redirect_uri first). The user signs in in the opened window; the redirect is intercepted, the code exchanged, and the token (with refresh_token if issued) stored and auto-refreshed.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYes
headedNo
timeout_sNo

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full behavioral burden. It discloses that a window opens, the user signs in, the redirect is intercepted, the code is exchanged, and the token is stored and auto-refreshed. It doesn't mention failure modes or interaction with existing profiles, but the main side effects are clear.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two dense sentences with no filler. The main action and prerequisite are front-loaded, and the flow details are compactly listed. Every clause adds relevant behavioral or setup information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The core OAuth flow is well explained, but there are gaps: no parameter-level guidance for `headed`/`timeout_s`, no comparison with sibling auth tools, and no description of the return value or success/error behavior. It is adequate for a simple call but not fully complete for an agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It explains the meaning of `name` (the profile) but gives no guidance on `headed` or `timeout_s` beyond the indirect mention of an 'opened window'. The two boolean/integer parameters remain effectively undocumented.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names the specific verb ('Run'), the resource ('OAuth authorization-code + PKCE flow for profile `name`'), and the protocol variant. This clearly separates it from sibling tools like auth_browser_login, which handles browser-based rather than OAuth flow.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives a clear prerequisite (configure oauth with grant_type, authorize_url, token_url, client_id, redirect_uri) and implies this is the OAuth path. However, it never explicitly says when to choose this over sibling auth tools or when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

auth_profilesA

List auth profiles, or show one (secrets are never shown).

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNo

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are present, so the description carries the behavioral disclosure burden. The note that 'secrets are never shown' is a valuable behavioral guarantee. However, the description does not disclose the return payload shape, behavior for nonexistent profiles, or explicitly confirm zero side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single compact sentence with the core behavior front-loaded. The parenthetical secrets note earns its place and there is no filler or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter read operation, the description is mostly adequate. However, with no output schema, it omits the return shape and any behavior for unknown or missing profile names. The secrets redaction note helps, but some operational uncertainty remains.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. The phrase 'show one' implies that the optional name parameter selects a specific profile, but the description never explicitly states that name is the profile identifier or what happens when it is omitted beyond the implied list behavior.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a clear verb and resource: 'List auth profiles, or show one.' This makes the tool's purpose immediately identifiable and distinguishes it from auth siblings like auth_update_profile, auth_delete_profile, and auth_check, which perform different actions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description says what the tool does but does not specify when to use it over alternatives such as auth_check or auth_update_profile. There are no explicit conditions, exclusions, or guidance about when showing one profile is preferable to other auth operations.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

auth_update_profileA

Create or update an auth profile. Only the fields you pass change (dicts are merged).

Any secret may be given as "env:VAR_NAME" so it is read from the server's environment instead of passing through the conversation — prefer that.

origins: origins that receive headers/tokens, e.g. ["https://app.example.com"] (default: the base_url's origin). Credentials are never sent to other origins. access_token/token_type/expires_in/refresh_token: a bearer token you already have. apply_token_as: where the app expects the token — any of "header" (Authorization: Bearer), "header:X-Api-Key", "local_storage:", "session_storage:", "local_storage_json:" (whole token object as JSON), "cookie:". oauth: {grant_type: client_credentials|password|refresh_token|authorization_code, token_url, client_id, client_secret, scope, audience, username, password, authorize_url, redirect_uri, client_auth: post|basic, extra_params}. For client_credentials/password a token is fetched immediately and refreshed automatically. For authorization_code, call auth_oauth_login next. cookies: Playwright cookies [{name, value, url} or {name, value, domain, path}]. local_storage/session_storage: {origin: {key: value}} injected before page scripts run. http_credentials: {username, password} for HTTP basic auth. clear: field names to remove, e.g. ["token", "cookies", "storage_state"].

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYes
clearNo
oauthNo
cookiesNo
headersNo
originsNo
expires_inNo
token_typeNo
access_tokenNo
local_storageNo
refresh_tokenNo
apply_token_asNo
session_storageNo
http_credentialsNo

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Since no annotations are provided, the description carries the full behavioral burden. It discloses key behaviors: merging semantics, env: variable handling, security note about credentials not being sent to other origins, and auto-refresh for client_credentials/password. It also explains the destructive 'clear' field. While not exhaustive, it covers the most important behavioral traits.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Despite the length, every sentence contributes necessary detail. The format uses line breaks to separate parameter groups, keeping it scannable. The opening sentence establishes purpose, then each line adds a specific explanation. No filler or redundancy; it is appropriately sized for the tool's complexity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 14 parameters and no output schema, the description is remarkably complete. It covers all major parameters, provides examples, defaults, security guidance, and next-step instructions for OAuth flows. It explains side effects and merging behavior. Nothing critical is missing for an agent to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0% schema description coverage, the description must fully compensate. It does so admirably: it explains origins with an example and default, enumerates apply_token_as values, details the oauth object structure and flows, specifies cookie formats, and clarifies injection points for local/session storage. This goes far beyond the bare schema property names and adds practical usage context.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with 'Create or update an auth profile,' which is a specific verb+resource statement. It distinguishes itself from siblings like auth_profiles (presumably listing/reading), auth_delete_profile (deletion), and auth_browser_login (interactive login) by clearly indicating its scope is profile creation/update.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives some contextual guidance (e.g., 'For authorization_code, call auth_oauth_login next'), but it never explicitly states when to use this tool versus alternatives like auth_browser_login or auth_oauth_login for login flows. It implies usage by describing what fields do, but lacks explicit 'use when X, not when Y' guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

captureA

Screenshot any URL (desktop or mobile viewport, optionally logged in via profile). clip_selector captures a single element. Saved under runtime/captures/.png.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYes
nameNo
profileNo
viewportNodesktop
full_pageNo
return_imageNo
clip_selectorNo
wait_for_selectorNo

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full behavioral disclosure burden. It does usefully disclose that screenshots are saved to runtime/captures/<name>.png and that clip_selector captures a single element, but it does not mention overwrite behavior, authentication requirements beyond 'optionally logged in via profile', or what return_image controls.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with no filler. The primary action is front-loaded, and the key scoping details (viewport, profile, element clipping, output path) are compactly included. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 8 parameters, no annotations, and no output schema, the description is adequate but leaves gaps. It gives enough to make a basic capture call and understand the save location, but an agent would not know the behavioral implications of return_image, wait_for_selector, or full_page without inspecting their titles.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate for 8 parameters. It adds meaning for viewport ('desktop or mobile'), profile ('optionally logged in'), clip_selector ('captures a single element'), and name (via the save path), but leaves full_page, return_image, and wait_for_selector semantically unexplained.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Screenshot any URL'. This clearly identifies the tool's core function and distinguishes it from sibling tools like capture_set or compare_images, which operate on sets or comparisons rather than a single URL capture.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives no explicit guidance on when to use this tool versus alternatives such as capture_set, qa_page, or compare_images. It mentions optional login and element clipping but does not state the conditions that would make this tool preferable over a sibling.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

capture_setA

Capture a named set of pages (e.g. label="before" before a refactor, "after" after it). Pages default to all reference names; any names work if you pass pages and the route mapping resolves them. Use compare_sets afterwards. Proves a refactor is pixel-neutral without depending on golden references.

ParametersJSON Schema
NameRequiredDescriptionDefault
labelYes
pagesNo
profileNo
base_urlNo
viewportNoall

TDQS

A3.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It discloses that pages default to all reference names, that specific pages work if the route mapping resolves them, and the outcome it proves. This adds significant behavioral context beyond the schema, though it does not mention return format, side effects, or requirements. Given the lack of annotations, this is reasonably transparent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise, three sentences, and front-loads the main purpose. It includes a concrete example and flows logically. It is not overly verbose, though it could have added a bit more structure, but it's effective and efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given five parameters with zero schema descriptions, no annotations, and no output schema, the description is incomplete. It explains only label and pages, leaving profile, base_url, and viewport unexplained. It also does not describe what the tool returns or any side effects. An agent would need more information to use it confidently.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It explains the label parameter (via the example) and pages (default behavior and condition), but it does not explain profile, base_url, or viewport at all. With five parameters and no schema descriptions, covering only two is insufficient, leaving the agent guessing about the rest.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a clear verb and resource: 'Capture a named set of pages' with a label. It provides an example use case (label='before'/'after' around a refactor) and distinguishes it from other tools by implying it captures a set under a label rather than a single capture. However, it does not explicitly name sibling tools to differentiate, though it references compare_sets, so it's clear enough.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives clear usage context: 'Use compare_sets afterwards' and the purpose 'Proves a refactor is pixel-neutral without depending on golden references.' This tells the agent when to use it (refactor verification). It does not explicitly state when not to use it or mention alternatives, but the context is clear and actionable.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

compare_imagesA

SSIM-compare two image files (e.g. a Figma export vs a capture). Paths may be relative to the project root. Returns score, changed regions, a heatmap path and an a|b preview.

ParametersJSON Schema
NameRequiredDescriptionDefault
alignNopad
image_aYes
image_bYes
thresholdNo
max_changed_ratioNo

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It explicitly lists the return values: score, changed regions, a heatmap path, and an a/b preview, and clarifies that paths may be relative to the project root. It does not detail side effects like file creation, but this is a reasonable disclosure for a comparison tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with no filler. It front-loads the purpose, gives a concrete example, and then condenses path semantics and output behavior into one efficient second sentence.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description provides a solid starting point for basic use: it names the core purpose, example inputs, and all return values. However, with no output schema and no parameter explanation for three of the five parameters, an agent cannot fully understand how to control the comparison or interpret edge cases.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate, but it only addresses the input image paths. The optional parameters align, threshold, and max_changed_ratio are completely unexplained in both the schema and the description, leaving an agent without enough information to set them meaningfully.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'SSIM-compare two image files', with a concrete example ('Figma export vs a capture'). This clearly differentiates it from siblings like compare_sets and ab_compare by focusing on a single pair of image files.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The example of comparing a Figma export to a capture implies a primary use case and gives helpful context. However, it does not explicitly state when to prefer this tool over alternatives such as compare_sets or ab_compare, nor does it mention any exclusion criteria.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

compare_setsB

Diff two capture sets (labels from capture_set, or directories). identical: true means every page scored >= threshold. Returns previews of the most-changed pages.

ParametersJSON Schema
NameRequiredDescriptionDefault
afterNoafter
alignNocrop
beforeNobefore
thresholdNo
max_imagesNo
max_changed_ratioNo

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses the threshold-based meaning of 'identical: true' and states that previews of most-changed pages are returned. With no annotations provided, the description carries the burden for behavioral disclosure, but it does not mention side effects, authentication requirements, or resource implications.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three short, focused sentences with no filler. The core purpose is front-loaded, and each sentence adds distinct, useful information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 6-parameter tool with no output schema and no annotations, the description is not complete enough for reliable invocation. The semantics of several parameters and the exact output format remain undocumented, leaving meaningful gaps for an agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate by explaining parameters. It provides some meaning for before/after (labels or directories) and threshold (used in the identical condition), but align, max_images, and max_changed_ratio are left unexplained, making it difficult to configure calls beyond defaults.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Diff') and resource ('two capture sets'), and clarifies that the sets can be labels from capture_set or directories. It distinguishes the tool from image-level comparison tools at a general level, though it does not explicitly name sibling alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The intended use case is implied: compare capture sets and identify page-level differences. However, there is no explicit guidance about when to use this tool versus siblings like compare_images or ab_compare, nor any stated exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

error_reportsA

Show automatic error-reporting status (pending/reported crash and error reports, target repo). flush=true posts pending reports to GitHub now.

ParametersJSON Schema
NameRequiredDescriptionDefault
flushNo

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the behavioral disclosure burden. It clearly discloses the key side effect: flush=true posts pending reports to GitHub. It also implies a read-only status mode via 'Show', though it does not detail credentials, failure behavior, or whether reports are removed after posting.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two compact sentences cover both modes and the report scope without wasted words. The primary status-checking purpose is front-loaded, with the side-effect flag described immediately after.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter tool with no output schema, the description provides sufficient context: purpose, flush behavior, and what kind of reports are involved. It could be slightly more explicit about what the status response looks like or what happens after a successful flush, but this is a minor gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate for the flush parameter. It does so effectively by explaining exactly what flush=true does, which the bare schema property name does not convey. The default behavior is also inferable from the first sentence.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific action and resource: 'Show automatic error-reporting status' and flush behavior. It also clarifies what that status covers (pending/reported crash/error reports and target repo), which makes the tool's role unambiguous among the unrelated sibling tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly establishes the two usage modes: inspect status by default, or pass flush=true to post pending reports to GitHub. While it does not explicitly name alternatives or exclusions, the intended context is clear and no competing error-reporting sibling exists.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qa_configA

Show the effective configuration (base URL, reference dir, route mapping), the reference screenshots found and the available auth profiles. Start here.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool 'shows' configuration and profiles, implying a read-only operation, but it does not explicitly confirm no side effects, require authentication, or describe response format. The lack of annotation coverage means a 3 is appropriate; the description provides basic transparency but not rich detail.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with no wasted words. The primary action and content are front-loaded, and the final 'Start here' adds a clear directive without repetition. It is efficient and structured effectively.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter informational tool, the description is largely complete. It lists all the key pieces of information the tool provides. It does not explicitly state the output format or that it is read-only, but these are minor gaps given the tool's simplicity and the absence of an output schema. The description sufficiently equips an agent to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so the description is not required to explain parameter behavior. The baseline for zero parameters is 4, and the description appropriately focuses on what the tool returns rather than inputs. No additional parameter semantics are needed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function with a specific verb 'Show' and a resource: effective configuration, listing specific components (base URL, reference dir, route mapping, screenshots, auth profiles). It also adds the phrase 'Start here,' which establishes its role as an entry point, distinguishing it from action-oriented siblings like run_qa or capture.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description includes the explicit guidance 'Start here,' indicating this tool should be used first to understand the current setup. While it doesn't explicitly name alternatives or exclusions, the 'Start here' instruction effectively signals when to use it, and the sibling list contains no other config-display tool, so usage context is clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qa_pageB

QA a single page against its reference (fast iteration while fixing). url overrides the route mapping for this call. Returns the result and, on failure, a reference|live side-by-side preview cropped around the changed regions.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlNo
pageYes
alignNoresize
profileNo
base_urlNo
thresholdNo
max_changed_ratioNo

TDQS

B3.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the burden of explaining behavior. It discloses the return value, failure behavior, and the side-by-side preview cropped around changed regions. It also clarifies the `url` override behavior, which adds meaningful context beyond the bare schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact, front-loaded with the main purpose, and every sentence adds useful information. The failure-preview behavior is placed in the second sentence without unnecessary filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 7 parameters, no output schema, and no annotations, the description is too sparse for reliable invocation. It provides a good high-level purpose and some output behavior, but the missing parameter semantics leave significant gaps for an agent deciding how to call the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate for parameter meaning. It only explains `url`, leaving `page`, `align`, `profile`, `base_url`, `threshold`, and `max_changed_ratio` unexplained. The required `page` parameter is especially critical but receives no clarification.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states the core purpose: 'QA a single page against its reference' with a clear verb, resource, and scope. It distinguishes itself from batch QA tools by emphasizing 'single page' and 'fast iteration while fixing', though it does not explicitly name a sibling alternative.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'fast iteration while fixing' implies the intended usage context, and the note about `url` overriding route mapping gives some operational guidance. However, it does not explicitly state when to use this tool versus alternatives like run_qa or capture.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

run_qaA

Capture pages from the running site and SSIM-compare them with the reference screenshots.

pages: reference names (file names without extension); default all references. viewport: all | desktop | mobile (names ending in "-mobile" use a 390px mobile context). profile: auth profile for pages behind a login. align: how to handle different image sizes — resize (default, legacy), crop, or pad (pad makes page-height changes count as differences). A page FAILs when SSIM < threshold OR more than max_changed_ratio of its pixels clearly changed colour (catches local changes such as a recoloured header that SSIM averages away). Returns a JSON summary (failures first, with changed regions in px) plus side-by-side reference|live preview images of up to max_images failing pages.

ParametersJSON Schema
NameRequiredDescriptionDefault
alignNoresize
pagesNo
profileNo
base_urlNo
viewportNoall
thresholdNo
max_imagesNo
max_changed_ratioNo

TDQS

A3.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It explains the failure criteria in detail, including the nuanced 'max_changed_ratio' condition that catches local color changes SSIM averages away, and describes the return format (JSON summary with changed regions, preview images). It does not mention side effects, rate limits, or prerequisites like needing reference screenshots to exist, but the core runtime behavior is well disclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured: a one-sentence purpose statement, parameter explanations, behavior logic, and output summary. It is information-dense and front-loaded, with every line contributing useful detail. It could be slightly tighter, but the structure is clear and appropriate for the tool's complexity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (8 parameters, no output schema, no annotations), the description covers the operational behavior, failure semantics, and return values in substantial detail. The main gaps are the missing base_url semantics and no indication of where reference screenshots come from or how they are managed. Still, the description provides enough for an agent to call the tool correctly in most intended scenarios.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate for all 8 parameters. It provides meaningful semantics for pages, viewport, profile, align, threshold, max_changed_ratio, and max_images, but base_url is completely absent from the description, leaving one parameter undocumented. This partial compensation warrants a mid-range score rather than a higher one.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence states a specific action ('Capture pages from the running site') and a specific resource ('reference screenshots') with a clear comparison method ('SSIM-compare'). It is clear what the tool does, but it does not explicitly differentiate itself from siblings like capture_set, compare_sets, or compare_images, so it falls short of full sibling distinction.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides parameter-level guidance (e.g., when to use profile for login-protected pages, how align changes behavior) but never says when to choose this tool over alternatives such as compare_sets or capture_set. There are no explicit exclusions or alternative recommendations, so the agent is left to infer the appropriate selection context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

sync_figmaA

Download Figma frames into the references directory (needs FIGMA_TOKEN). Frame names become reference names; overwrites existing references with the same name.

ParametersJSON Schema
NameRequiredDescriptionDefault
frameNo
file_keyYes

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full responsibility for behavioral disclosure. It explicitly reveals a network/auth prerequisite (FIGMA_TOKEN) and warns that existing references with the same name are overwritten—a key destructive side-effect. It doesn't cover return format or rate limits, but the critical behaviors are disclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, front-loaded with the action, and every clause carries meaningful detail: the download, the token requirement, naming behavior, and overwrite side-effect. No filler or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with 2 parameters and no output schema, the description covers the core action, prerequisite, and overwrite behavior. However, it omits what happens when 'frame' is null (presumably all frames) and how 'file_key' maps to the Figma file, which is needed for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description needed to compensate by explaining 'frame' and 'file_key'. It clarifies that frame names become reference names, but it does not explain that 'frame' is an optional filter or what 'file_key' identifies. This leaves a significant gap for an agent trying to set parameters correctly.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action ('Download'), a clear resource ('Figma frames'), and a destination ('references directory'), making the tool's function immediately obvious. It also differentiates from siblings like capture or update_reference by indicating a Figma source, and the overwrite note adds specificity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides a clear context for use—syncing Figma frames into the references directory—and notes the FIGMA_TOKEN prerequisite. However, it does not explicitly state when to prefer this tool over siblings like capture or update_reference, nor does it mention exclusions or alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_referenceA

Make the current look of a page its reference screenshot: from source (an image), else the latest qa_page capture, else a fresh capture right now (so this also creates a first baseline for a page that has no reference yet). The old reference is backed up. Only do this when the USER has confirmed the look is intended; set confirm=true.

ParametersJSON Schema
NameRequiredDescriptionDefault
pageYes
sourceNo
confirmNo
profileNo

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full behavioral disclosure burden. It does well by explaining side effects: the old reference is backed up, a fresh capture may be taken, and a first baseline can be created for pages lacking a reference. It does not mention profile/auth requirements, return behavior, or failure cases, so it is not fully exhaustive, but the core mutation behavior and safety mechanism are transparent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but front-loaded with the primary purpose, and every clause adds information: fallback order, baseline creation, backup behavior, and confirmation requirement. It is somewhat run-on and could be split into clearer sentences, but there is no fluff or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutating tool with no annotations and no output schema, the description covers the essential operational context: what the tool does, how the source is resolved, that the old reference is preserved, and when confirmation is required. The main gap is the unexplained profile parameter, which may matter in an auth-aware environment, but the core decision-making information is present.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate for the schema's silence. It adds real meaning for source (explicitly an image) and confirm (must be true after user confirmation), and page is implied. However, the profile parameter is never explained, leaving one of four parameters semantically undocumented.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource: "Make the current look of a page its reference screenshot." It goes beyond a generic statement by specifying the fallback chain (source image → latest qa_page capture → fresh capture), which clearly distinguishes it from sibling tools like capture or qa_page. An agent can immediately understand what this tool changes and when it is invoked.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives a clear usage condition: "Only do this when the USER has confirmed the look is intended; set confirm=true." This tells the agent when it is appropriate to invoke the tool and how to express that precondition through the confirm parameter. It does not explicitly name alternative sibling tools or state when NOT to use it, but the context and fallback logic strongly imply its specialized role.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 18 tool updatesv0.1.0
    • First observedab_compare
    • First observedauth_browser_login
    • First observedauth_check
    • First observedauth_delete_profile
    • First observedauth_import_storage_state
    • First observedauth_oauth_login
    • First observedauth_profiles
    • First observedauth_update_profile
    • First observedcapture
    • First observedcapture_set
    • First observedcompare_images
    • First observedcompare_sets
    • First observederror_reports
    • First observedqa_config
    • First observedqa_page
    • First observedrun_qa
    • First observedsync_figma
    • First observedupdate_reference

TDQS

A3.5/5.0

Scored across 18 tools

Disambiguation4/5

Most tools have clearly distinct purposes: run_qa vs qa_page, capture vs capture_set, and compare_images vs compare_sets vs ab_compare are separated by scope. The main ambiguity is between auth_browser_login and auth_oauth_login, though their descriptions clarify the different login flows.

Naming Consistency4/5

Tool names follow a mostly predictable snake_case style, with useful prefixes like qa_ and auth_. However, the pattern is not strictly verb_noun: capture and ab_compare stand out, and names like qa_config or error_reports are noun-style rather than action-first.

Tool Count3/5

18 tools is on the heavy side, especially considering the 7-tool auth sub-system alongside the core screenshot/compare workflow. It is not excessive in absolute terms, but it crosses into the range where the set starts to feel bloated.

Completeness4/5

The core visual QA lifecycle is well covered: references, captures, comparisons, set diffs, and running-server diffs are all present, and auth profile management is thorough. Minor gaps exist such as no direct tool for deleting references or listing capture sets, but these can be worked around.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    D
    maintenance
    Enables AI assistants to capture website screenshots, automate browser interactions, and manage recurring screenshot configurations across 150+ global locations. It also supports AI-powered domain research and visual change monitoring for any web page.
    16 npm
    MIT
  • A
    license
    Not graded
    quality
    D
    maintenance
    Enables AI agents to see, analyze, and visually verify web page changes through pixel-perfect diffing, theme extraction, layout analysis, and interactive element detection.
    19 npm
    MIT
  • A
    license
    A
    quality
    B
    maintenance
    Screenshot, visual-diff, and AI page-analysis API for AI agents. Capture any URL as PNG, JPEG, WebP, PDF, or HTML, diff two versions of a page to catch visual regressions, and get an AI summary of what a page contains.
    3
    44 npm
    1
    MIT