Skip to main content
Glama
Coding-Crashkurse

e2e-verifier

e2e-verifier

MCP server that reproduces and verifies website bug tickets in a real browser and hands back evidence: an annotated video, a Playwright trace, screenshots, logs and a verdict per acceptance criterion.

  • Input: a structured BugTicket (steps, expected behaviour, acceptance criteria) or a plain Scenario. The MCP client (for example Claude Code) turns free-text bug reports into tickets; the server itself never calls an LLM.

  • Execution: Playwright, Chromium headless by default, one isolated browser context per run.

  • Evidence: video.webm with a HUD overlay (step banner, red frame around the failing element), trace.zip for the Playwright trace viewer, per-step screenshots, console.jsonl, network.jsonl, chapters.vtt, report.md and result.json.

  • Modes: reproduce before a fix (a failing assertion means reproduced) and verify after it (all criteria met means fixed).

The full design is in spec.md.

Requirements

  • Python 3.12+ and uv

  • Chromium for Playwright (uv run playwright install chromium)

  • Optional: ffmpeg on the path for an additional mp4 export (E2E_FFMPEG_PATH)

Related MCP server: Limetest MCP Server

Setup

uv sync
uv run playwright install chromium

Run the server

uv run e2e-verifier                 # stdio transport (default)
E2E_TRANSPORT=http uv run e2e-verifier   # Streamable HTTP on 127.0.0.1:8765

Claude Code configuration (.mcp.json in the project that contains the website under test):

{
  "mcpServers": {
    "e2e-verifier": {
      "command": "uv",
      "args": ["run", "e2e-verifier"],
      "env": {
        "E2E_ARTIFACT_DIR": "artifacts",
        "E2E_ALLOWED_HOSTS": "localhost,127.0.0.1"
      }
    }
  }
}

The paths are relative to the project root, which is the working directory Claude Code uses for a project-scoped .mcp.json. When the server is started from another directory, add "--directory", "<absolute path to end_to_end_testmcp>" after "run" and use an absolute E2E_ARTIFACT_DIR. The HTTP alternative is E2E_TRANSPORT=http uv run e2e-verifier and a client entry {"type": "http", "url": "http://127.0.0.1:8765/mcp"}.

Typical session

  1. probe_target("http://127.0.0.1:5173/") – is the site up?

  2. discover_elements(url, query="Save") – which targets exist? Returns suggested Targets.

  3. Build a BugTicket (prompt ticket_from_report), then validate_ticket.

  4. run_ticket(ticket, options={"mode": "reproduce"}) – attach video.webm to the ticket.

  5. After the fix: run_ticket(ticket, options={"mode": "verify"}).

Minimal ticket:

{
  "id": "WEB-142",
  "title": "Save button does nothing",
  "url": "http://127.0.0.1:5173/settings",
  "steps": [
    {"id": "s1", "action": "fill", "target": {"label": "Name"}, "value": "Anna"},
    {"id": "s2", "action": "click", "target": {"role": {"role": "button", "name": "Save"}}},
    {"id": "s3", "action": "expect_visible", "target": {"role": {"role": "status", "name": "Saved"}}},
    {"id": "s4", "action": "expect_request", "method": "POST", "url_pattern": "**/api/settings", "status": 200}
  ],
  "expected": "A toast 'Saved' appears and the settings are posted.",
  "actual": "Nothing happens.",
  "acceptance_criteria": [
    {"id": "AC1", "description": "Toast appears", "step_ids": ["s3"]},
    {"id": "AC2", "description": "Settings are posted", "step_ids": ["s4"]}
  ]
}

Interactive sessions: letting an agent operate the site

An agent that does not know the site yet can drive it step by step. Every session is recorded (video with HUD banners, trace, screenshots, logs), and every tool result comes with a screenshot the agent can look at.

session_open(url)                         -> session id, first navigation result, screenshot
session_inspect(session_id, query="Save") -> ARIA snapshot + candidate targets with suggested Target JSON
session_act(session_id, step)             -> step result + screenshot; step = any action/assertion,
                                             e.g. {"id":"s1","action":"click","target":{"test_id":"save"}}
session_annotate(session_id, message, target?, tone?) -> callout/frame drawn into the video + screenshot
session_screenshot(session_id, full_page?) -> screenshot
session_close(session_id, verdict?, summary?) -> RunResult with video.webm, trace.zip, report.md ...

Failed assertions do not end a session; they are marked with a red frame in the video and the agent can keep going. get_schema(kind="step") returns the schema of all step types. Sessions are closed automatically after E2E_SESSION_IDLE_TIMEOUT_S (default 600 s); at most E2E_MAX_SESSIONS (default 3) are open at once. The session id doubles as run id for get_artifact and the runs:// resources.

Tools

Tool

Purpose

session_open / session_act / session_inspect / session_annotate / session_screenshot / session_close / session_list

Interactive, recorded browser sessions (see above)

get_schema

JSON schema for ticket, scenario, run_options, run_result

validate_ticket

Semantic validation without a browser (host allowlist, criteria coverage, brittle selectors)

run_ticket / run_scenario

Execute in the browser, record artifacts, return RunResult with verdict

get_run / list_runs / delete_run

Manage stored runs

get_artifact

Fetch video, trace, report, screenshots, logs (inline up to E2E_MAX_INLINE_ARTIFACT_MB)

probe_target

Reachability, status, title, load time, console errors of a URL

discover_elements

ARIA snapshot plus candidate targets for a query, optional steps_before

server_info

Versions, configuration, run counts

Resources: schema://ticket, schema://scenario, schema://run-options, schema://run-result, runs://index, runs://{run_id}/{result,ticket,report,video,trace,console,network,chapters,har}, runs://{run_id}/screenshots/{index}.

Prompts: ticket_from_report, analyze_run, refine_target.

Configuration

All settings are environment variables with the prefix E2E_ (a .env file is read too).

Variable

Default

Meaning

E2E_ARTIFACT_DIR

artifacts

Root directory for runs

E2E_ALLOWED_HOSTS

localhost,127.0.0.1,[::1]

Hosts the browser may navigate to (*.example.test wildcards allowed)

E2E_ALLOW_ANY_HOST

false

Disable the allowlist

E2E_BROWSER

chromium

chromium, firefox, webkit

E2E_HEADLESS

true

Headed mode for local debugging

E2E_DEFAULT_STEP_TIMEOUT_MS

10000

Timeout per action/assertion

E2E_DEFAULT_RUN_TIMEOUT_S

120

Hard limit per run (capped by E2E_MAX_RUN_TIMEOUT_S, default 600)

E2E_MAX_PARALLEL_RUNS

1

Concurrent runs

E2E_MAX_RUNS_RETAINED

50

Retention by count

E2E_MAX_ARTIFACT_AGE_H

168

Retention by age

E2E_MAX_INLINE_ARTIFACT_MB

8

Larger artifacts are returned as a path

E2E_FIXTURE_DIR

unset

Root for upload_file paths and storage_state_path

E2E_FFMPEG_PATH

unset

Optional mp4 export

E2E_TRANSPORT

stdio

stdio or http

E2E_HTTP_HOST / E2E_HTTP_PORT

127.0.0.1 / 8765

HTTP transport bind address

E2E_HUD_LANGUAGE

en

en or de for fixed HUD texts

Artifacts

artifacts/runs/<run_id>/
  result.json  report.md  ticket.json  options.json
  video.webm   chapters.vtt   trace.zip
  console.jsonl  network.jsonl  [network.har]  [video.mp4]
  screenshots/001_passed.png ... 003_failed.png 003_failure_fullpage.png

Open a trace with:

uv run playwright show-trace artifacts/runs/<run_id>/trace.zip

Development

uv run ruff format .
uv run ruff check .
uv run mypy src tests
uv run pytest tests/unit
uv run pytest tests/integration/test_run_ticket_broken_button.py -x --timeout=120

Integration tests start a real Chromium against a fixture site under tests/fixtures/site; run them per file. The code base follows a no-comments, no-docstrings rule that is enforced by a unit test; tool descriptions live in decorator arguments and Field(description=...).

Available Tools

18 tools
delete_runA

Delete a finished run and all of its artifacts.

ParametersJSON Schema
NameRequiredDescriptionDefault
run_idYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
run_idYes
deletedYes

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are present, so the description bears the full burden. It does disclose the destructive scope (run and all artifacts) and a prerequisite condition ('finished'), but it does not state whether deletion is irreversible or whether special permissions are required.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single front-loaded sentence with no filler. It states the action, target, and scope directly, making it an efficient and well-structured definition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter destructive operation, the description captures the key side effect (artifact deletion) and applicability constraint ('finished'). An output schema exists so return-value details are not the description's job, but irreversibility and error/status behavior are left unspecified, and no annotations fill that gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, and the description adds little for run_id. It does not explain where the ID comes from, its format, ownership constraints, or how it relates to the broader run lifecycle, leaving the agent to infer the parameter meaning.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Delete') and names the resource ('a finished run') plus the cascade ('all of its artifacts'). It clearly distinguishes this from read/list siblings like get_run, list_runs, and get_artifact.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies a usage condition: only finished runs are eligible for deletion. However, it does not name alternatives or explicitly state when to choose this over get_run or run_ticket, leaving tool-selection guidance mostly implicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

discover_elementsB

Inspect a page: returns the ARIA snapshot and candidate targets matching a query, each with a suggested Target for tickets. steps_before can log in or navigate first.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYes
queryNo
max_resultsNo
steps_beforeNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
urlYes
queryYes
final_urlYes
candidatesYes
aria_snapshotYes
aria_snapshot_truncatedYes

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description bears the full transparency burden, but it only states that the tool returns an ARIA snapshot/candidates and that steps_before may log in or navigate. It does not disclose whether the tool mutates session state, whether navigation to the URL is an implicit side effect, or any error/limit behaviors. This is meaningful but partial transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two front-loaded sentences, each earning its place: first states the action and output, second explains the auxiliary steps parameter. No redundancy or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite an output schema, the tool has a large, intricate steps_before schema and no annotations; the description does not explain query syntax, max_results behavior, or whether the tool itself navigates before inspecting. An agent would need to infer several call conventions from the schema or examples. This under-specifies a complex tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema descriptions cover 0% of parameters, so the description must clarify semantics. It adds meaning for steps_before ('can log in or navigate first') and implicitly for query ('matching a query'), but it does not explain url, max_results, or how a query string is matched. The schema supplies types and defaults, but not semantics, leaving significant gaps.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a clear action ('Inspect a page') and resource, and specifies the return value: an ARIA snapshot plus candidate targets matching a query, each with a suggested Target. This distinguishes it from sibling inspection tools like probe_target or session_inspect by emphasizing ticket-target generation. The verb and outcome are unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the tool is for inspecting pages and obtaining targets for tickets ('suggested Target for tickets'), and notes steps_before can handle login/navigation. However, it gives no explicit guidance on when to prefer this tool over siblings such as session_inspect or probe_target, and no 'when not to use' boundary. Usage context is present only by implication.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_artifactB

Fetch one artifact of a run (video, trace, report, screenshot, logs). Small files are returned inline, large ones as a path.

ParametersJSON Schema
NameRequiredDescriptionDefault
kindYes
indexNo
run_idYes

TDQS

B3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the behavioral disclosure burden. It adds one useful behavioral trait: small files are returned inline and large files as a path. However, it does not disclose auth requirements, error conditions, or what 'path' means.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences with no fluff. The core purpose is front-loaded, and the size-dependent return behavior earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with no annotations, no output schema, zero parameter descriptions, and an enum of 11 kinds, this description is too thin. An agent cannot reliably determine when to provide 'index', what each kind returns, or how to interpret the returned path.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It loosely explains the 'kind' parameter with examples, but it does not explain 'run_id' semantics or the purpose of the optional 'index' parameter, which is a significant gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb and resource: 'Fetch one artifact of a run', and gives concrete examples of artifact kinds. It is clear and distinct from sibling tools like get_run or list_runs, though it does not explicitly name those alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives such as get_run or list_runs. There are no exclusions, prerequisites, or condition-based routing instructions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_runB

Return the stored RunResult of a finished run.

ParametersJSON Schema
NameRequiredDescriptionDefault
run_idYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
kindYes
modeYes
labelNo
stepsYes
titleYes
run_idYes
browserYes
outcomeYes
summaryYes
verdictYes
warningsNo
artifactsYes
run_errorNo
ticket_idYes
started_atYes
target_urlYes
duration_msYes
finished_atYes
observationsYes
acceptance_criteriaYes
video_offset_uncertainty_msYes

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden of behavioral disclosure. 'Stored' and 'finished' imply a read-only retrieval of an already-computed result, but auth requirements, failure behavior, and any side effects are not addressed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single, focused sentence with no filler. The core behavior is stated up front, and every word contributes meaning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given that an output schema exists and the tool has only one straightforward parameter, the description is mostly sufficient. It could add error behavior or timing expectations, but for a simple getter it is reasonably complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, and the description does not explain run_id or its format. The only relevant hint is that it refers to a finished run; this is minimal compensation for the missing schema detail.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the verb ('Return') and resource ('stored RunResult'), with a scope qualifier ('of a finished run'). It doesn't explicitly compare against siblings like get_artifact or list_runs, so it lacks explicit sibling differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no direct guidance on when to use this tool versus alternatives. The phrase 'finished run' implies it should be called after a run completes, but no exclusions or alternatives like list_runs or get_artifact are mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_schemaB

Return the JSON schema for a BugTicket, Scenario, RunOptions or RunResult.

ParametersJSON Schema
NameRequiredDescriptionDefault
kindYes

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description is the only behavioral signal; it implies a read-only lookup but does not state side effects, output format, or any constraints. It does not contradict annotations, but it adds no behavioral context beyond the bare operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One short sentence, front-loaded with the core action and resource. No filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter getter with an output schema, the description is nearly sufficient, but the enum mismatch (step missing) creates a real gap in completeness. It also fails to clarify what 'RunOptions' corresponds to in the enum, though 'run_options' is a close mapping.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single 'kind' parameter is undocumented in the schema (0% coverage) and the description only loosely maps to four of the five enum values without naming the parameter. It does not mention 'step', so an agent could not infer the full set of valid inputs from the description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly identifies a specific verb and resource: returning JSON schemas for four named domain types, which distinguishes it from sibling action tools. However, the input schema's enum also includes 'step', which the description omits, so the stated scope is not fully aligned.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The intended use is implied—call this when you need the JSON schema for one of the listed types—but there is no explicit 'use when' guidance or comparison to alternatives. Since get_schema is unique among siblings, no exclusion is stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_runsA

List finished runs, newest first, optionally filtered by ticket or outcome.

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNo
outcomeNo
ticket_idNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral disclosure burden. It reveals that only finished runs are returned and that results are ordered newest first. It does not mention pagination, the default limit, or read-only nature explicitly. Adequate but not comprehensive.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that front-loads the core purpose and ordering with no redundancies or extraneous detail. It earns its length.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so the return structure is covered. The description covers the primary use case and filters. Missing guidance on the limit parameter's behavior, pagination, and any prerequisites such as an active session (implied by sibling tools). Reasonably complete for a list operation, but with notable gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0% schema description coverage, the description must compensate for parameter meaning. It mentions filtering by ticket and outcome, covering ticket_id and outcome, but omits any explanation of the limit parameter. The outcome enum values are in the schema, but ticket_id format is undefined. Incomplete compensation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb (list), resource (finished runs), ordering (newest first), and optional filters (ticket or outcome). It clearly distinguishes from siblings like get_run (single run) and delete_run (mutation) by defining a list operation on completed runs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for enumerating completed runs and mentions optional filters, which suggests when filtering is appropriate. However, it does not explicitly direct the agent to use get_run for single-run lookups or exclude active runs, so the guidance is clear but not fully explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

probe_targetA

Open a URL once and report status, title, load time and console errors.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
urlYes
errorYes
titleYes
statusYes
final_urlYes
reachableYes
page_errorsYes
load_time_msYes
console_errorsYes

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It discloses that the tool opens a URL once and reports specific data, but it does not mention side effects, authentication needs, rate limits, or what happens on failure. This is minimal but adequate for a simple probe, though it lacks depth about edge cases.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single, well-structured sentence that front-loads the action and lists the key outputs. No fluff or redundancy; every word contributes to the meaning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (one parameter) and the presence of an output schema, the description covers the essential purpose and reported data. It doesn't mention error handling or session independence, but for a one-off probe, these are minor gaps. The description is largely complete for an agent to use it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. The description clearly states 'Open a URL once,' defining the url parameter as the target URL. It adds meaning by indicating the action and the one-time nature, though it does not specify URL format or required protocols.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Open'), a clear resource ('a URL'), and explicitly lists the reported outputs (status, title, load time, console errors). This clearly distinguishes it from sibling tools like session_open or session_inspect, which imply persistent sessions or element discovery.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies a one-time probe scenario but does not explicitly state when to use this tool over siblings like session_open or server_info. No exclusions or alternative routing is provided, leaving the agent to infer the appropriate context from the one-time nature.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

run_scenarioC

Execute a Scenario (steps without bug semantics) and record artifacts.

ParametersJSON Schema
NameRequiredDescriptionDefault
optionsNo
scenarioYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
kindYes
modeYes
labelNo
stepsYes
titleYes
run_idYes
browserYes
outcomeYes
summaryYes
verdictYes
warningsNo
artifactsYes
run_errorNo
ticket_idYes
started_atYes
target_urlYes
duration_msYes
finished_atYes
observationsYes
acceptance_criteriaYes
video_offset_uncertainty_msYes

TDQS

C2.6/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of disclosure. It mentions recording artifacts, which is a side effect, but it doesn't clarify whether the tool mutates state, requires an open session, how long it runs, what happens on failure, or what the artifacts imply for the agent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is one short sentence with no fluff, and the primary verb and object are front-loaded. The vague parenthetical about 'bug semantics' is the only detractor.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity of the input schema (a full Scenario definition with RunOptions and many step types), this description does not helped the agent understand when or how to invoke it. It doesn't explain the relationship with run_ticket, whether options affect behavior, or any prerequisites such as sessions, output schema presence. It is not complete enough.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, and the description adds no meaning to either parameter. The 'scenario' parameter is merely restated and 'options' is not mentioned at all, leaving the agent with zero guidance on how to construct a Scenario or configure runs.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states both the action and resource clearly: 'Execute a Scenario' and 'record artifacts.' The parenthetical 'steps without bug semantics' hints at a distinction from ticket-style workflows, but it does not explicitly name a sibling alternative and relies on 'bug semantics' being understood.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no direct guidance on when to use this tool versus run_ticket, validate_ticket, or the session_* tools. The parenthetical 'without bug semantics' implies that run_ticket is for bugs, but it is not explicit and leaves the agent to infer the appropriate context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

run_ticketA

Execute a BugTicket in a real browser. Records an annotated video, a Playwright trace, screenshots and logs, then returns the verdict per acceptance criterion. Use mode=reproduce before a fix and mode=verify after it.

ParametersJSON Schema
NameRequiredDescriptionDefault
ticketYes
optionsNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
kindYes
modeYes
labelNo
stepsYes
titleYes
run_idYes
browserYes
outcomeYes
summaryYes
verdictYes
warningsNo
artifactsYes
run_errorNo
ticket_idYes
started_atYes
target_urlYes
duration_msYes
finished_atYes
observationsYes
acceptance_criteriaYes
video_offset_uncertainty_msYes

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden. It discloses non-obvious behavior: execution happens in a real browser and produces an annotated video, Playwright trace, screenshots, and logs before returning per-criterion verdicts. It doesn't cover side effects on the target site or execution duration, but the core behavioral profile is clearly stated.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, no filler. The first sentence states the action, the second states the outputs, and the third gives mode guidance. Information is front-loaded and every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the high complexity of the schema, the description covers the essential invocation context: what gets executed, what artifacts are produced, and how mode selection should differ before vs after a fix. The presence of an output schema relieves it from explaining return values. It could mention retrieval of artifacts or asynchronous behavior, but those are inferable from sibling tools.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It does add meaning for the options.mode parameter via the reproduce/verify guidance. For the ticket parameter it mostly repeats the type name, and it doesn't explain the structure or required fields of BugTicket beyond what the schema already defines. This is a partial but not complete compensation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb ('Execute'), a specific resource ('BugTicket'), and the concrete environment ('real browser'). It also states the unique output ('verdict per acceptance criterion'), which distinguishes it from sibling tools like run_scenario or get_run.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear operational guidance: 'Use mode=reproduce before a fix and mode=verify after it.' This is actionable context for invoking the tool correctly. It doesn't explicitly contrast with sibling tools or state when not to use it, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

server_infoB

Versions, configuration and run counts of this server.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
headlessYes
runs_storedYes
artifact_dirYes
allowed_hostsYes
allow_any_hostYes
server_versionYes
default_browserYes
fastmcp_versionYes
runs_in_progressYes
max_parallel_runsYes
playwright_versionYes

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure, but it only names output categories. It never confirms the tool is read-only, has no side effects, or requires specific permissions, which matters for an agent deciding whether invocation is safe.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single tight, front-loaded sentence with no filler or repetition. Every word contributes meaning by specifying the resource and the categories of information returned.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter informational tool with an output schema, this description provides adequate basic context: the caller knows what data to expect. However, it omits usage guidance and explicit safety/read-only framing, and it relies heavily on the tool name and output schema to fill gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parametershare, so schema description coverage is complete and there is no parameter documentation burden. The description compensates by clarifying what information the call returns, which aligns with the empty input schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names the resource ('this server') and enumerates the specific data categories returned: versions, configuration, and run counts. It is clearly distinct from the session-, ticket-, and run-oriented siblings, though it lacks an explicit verb like 'retrieves' or 'returns'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided about when to call this tool versus any sibling. It does not state whether this is the right tool for environment diagnostics, health checks, or setup verification, and it offers no exclusions or alternative suggestions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

session_actA

Execute one step in an open session and return the step result plus a screenshot. A step is any action or assertion from the ticket schema, for example {"id": "s1", "action": "click", "target": {"role": {"role": "button", "name": "Save"}}} or {"id": "s2", "action": "expect_visible", "target": {"test_id": "toast"}}. Failed assertions do not end the session; they are marked in the video with a red frame. Call get_schema(kind="step") for all step types.

ParametersJSON Schema
NameRequiredDescriptionDefault
stepYes
session_idYes
include_screenshotNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It explicitly reveals that failed assertions do not end the session and are marked with a red frame in the video, a non-obvious behavior. It also discloses the return content (step result plus screenshot), which is useful for the agent to know what to expect.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact—two sentences plus two examples—and front-loaded with the primary purpose. Every sentence earns its place: the first states what and returns, the second defines step via examples, the third conveys failure behavior, and the fourth points to get_schema for more detail. No filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the large, complex step union schema, the description cannot enumerate all step types, but it wisely references get_schema(kind="step") to fill the gap. It also notes the screenshot return, which is important for an agent that may need visual verification. It does not mention potential error conditions beyond assertions, but the schema and the get_schema pointer cover most of the missing context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema coverage is 0% at the top level, but the description partially compensates by explaining that 'A step is any action or assertion from the ticket schema' and giving two concrete step examples. However, it does not describe session_id beyond what the name implies, and the rich per-field descriptions inside the $defs of the schema carry most of the parameter semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Execute'), a specific resource ('one step in an open session'), and a clear output ('step result plus a screenshot'). It provides two concrete examples that make the resource unambiguous aid distinguishes it from sibling tools like session_open (which creates a session) and session_screenshot (which only captures a screenshot).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states the precondition 'in an open session' and instructs the agent to call get_schema(kind="step") for all step types, which is a clear usage pointer for constructing the step parameter. It does not explicitly name alternatives or exclusions, but the session-related siblings make the context clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

session_annotateA

Draw an annotation into the recorded video: a callout with your message, optionally with a frame around a target element (tone info, success or failure). The annotation stays visible for hold_ms and is captured as a screenshot.

ParametersJSON Schema
NameRequiredDescriptionDefault
toneNoinfo
targetNo
hold_msNo
messageYes
session_idYes

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description reveals important behavioral traits not visible from the name or annotations: the annotation persists for hold_ms and a screenshot is captured. It also mentions the option of a frame and tone. Since no annotations are provided, this carries the full burden, and it does well in disclosing side effects like screenshot capture.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no fluff, with the primary action front-loaded and supporting details following. Every clause adds useful information about function or behavior.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers the tool's core behavior and parameters, but there is no output schema and no mention of what the tool returns. The phrase 'captured as a screenshot' hints at an artifact but doesn't clearly state whether the result is a screenshot reference or something else. For an agent deciding whether to call this tool announced, the return value remains ambiguous.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The description explicitly explains message, target, tone, and hold_ms: 'callout with your message', 'frame around a target element', 'tone info, success or failure', and directly references hold_ms. It does not mention session_id, but that is self-evident from the context. It compensates well for the 0% schema description coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb 'Draw' and resource 'recorded video', then elaborates with callout, message, optional frame, tone, hold_ms, and screenshot capture. This clearly distinguishes it from sibling tools like session_screenshot or session_act, which do different things. The scope is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when to use this tool versus alternatives. It does not say 'use this for annotating video' or compare with session_screenshot for plain captures. The context of 'recorded video' implies its domain, but there is no direct statement about when to choose it over other session tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

session_closeA

Close an open session, stop the recording and write video.webm, trace.zip, report.md, chapters.vtt and result.json. Optionally pass your own verdict (reproduced, not_reproduced, fixed, still_broken, passed, failed, inconclusive) and summary; otherwise they are derived from the executed assertions. Fetch artifacts with get_artifact using the session id as run_id.

ParametersJSON Schema
NameRequiredDescriptionDefault
summaryNo
verdictNo
session_idYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
kindYes
modeYes
labelNo
stepsYes
titleYes
run_idYes
browserYes
outcomeYes
summaryYes
verdictYes
warningsNo
artifactsYes
run_errorNo
ticket_idYes
started_atYes
target_urlYes
duration_msYes
finished_atYes
observationsYes
acceptance_criteriaYes
video_offset_uncertainty_msYes

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does well: it discloses that recording stops, five artifacts are written, and verdict/summary are either user-supplied or derived from assertions. It stops short of stating irreversibility or failure behavior, but the main side effects are clearly surfaced.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, no filler, and the most important information is front-loaded: what the tool does and what it produces. The optional-parameter behavior and artifact-retrieval follow-up are stated efficiently.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers the tool's lifecycle role, preconditions ('open session'), side effects, artifact outputs, optional inputs, and a follow-up retrieval path. An output schema exists for return values, so the only notable gap is error behavior and whether closing is irreversible.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must explain the parameters. It explains that summary and verdict are optional and default to being derived from executed assertions, and it clarifies that the session id doubles as the run_id for get_artifact. It could go further on session_id format, but the key semantics are covered.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action: closing an open session, stopping recording, and writing named artifacts. It distinguishes this tool from siblings like session_act and session_screenshot by focusing on termination and artifact finalization.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'Close an open session' implies it should be used at the end of a session, and the note about get_artifact is useful follow-up guidance. However, there is no explicit statement about when not to use it or how it compares to alternatives like session_annotate or session_inspect.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

session_inspectA

Inspect the current page of an open session without changing it: ARIA snapshot plus candidate targets matching a query, each with a suggested Target.

ParametersJSON Schema
NameRequiredDescriptionDefault
queryNo
session_idYes
max_resultsNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
urlYes
queryYes
final_urlYes
candidatesYes
aria_snapshotYes
aria_snapshot_truncatedYes

TDQS

A3.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It clearly discloses the non-mutating behavior and describes the return payload in useful terms (ARIA snapshot, candidate targets, suggested Target). It does not cover error cases, permissions, or invalid-session behavior, so it is not a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One tightly structured sentence front-loads the action and the non-mutation guarantee, then packs the output contents into an appositive. There is no filler and every phrase contributes meaning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, some return-value detail is already covered, and the tool is fairly simple. However, the lack of explicit guidance on max_results behavior and the absence of preconditions/error hints leave the definition slightly incomplete for an agent deciding whether and how to invoke it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It only indirectly explains 'query' via 'matching a query'; session_id and max_results are never described, and the effect of max_results on the candidate target list is absent.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Inspect') and names the exact resource ('current page of an open session'), plus the key output ('ARIA snapshot plus candidate targets'). It is clear and easy to distinguish from act/screenshot tools by its read-only framing, though it does not explicitly name or contrast sibling tools like discover_elements or probe_target.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The context is implied: use it on an open session when you want to inspect rather than modify. The phrase 'without changing it' hints at when not to use it, but there is no explicit guidance about when to choose this tool over nearby alternatives such as discover_elements, probe_target, or session_screenshot.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

session_listA

List open interactive sessions.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the behavior disclosure burden, and it does state a read-only list operation scoped to 'open interactive sessions'. For a parameterless list tool, this is reasonably transparent, though it does not mention edge cases like empty results or whether closed sessions are excluded.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single short sentence with no filler: 'List open interactive sessions.' Every word contributes to the meaning, and the verb is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a no-parameter list tool with an output schema, the description is nearly complete. It could be slightly more explicit about what counts as an 'open interactive session' or how this relates to session_open, but these are minor gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has zero parameters, so there is no parameter semantics to document. The description adds no parameter detail, but the baseline of 4 applies because the schema fully covers the empty parameter set.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description uses a specific verb ('List') and a specific resource ('open interactive sessions'), making the tool's purpose clear. It differentiates from siblings implicitly by scope, but does not explicitly name any alternative or contrast with them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is given about when to use this tool versus session_open, session_act, or other session-related siblings. The intended context is implied but no alternatives or exclusions are mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

session_openA

Open an interactive, recorded browser session on a URL and return its id, the result of the first navigation and a screenshot. Everything that happens in the session is recorded as video, trace, screenshots and logs until session_close. Use session_act to drive the page step by step.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYes
optionsNo

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full behavioral disclosure burden; it successfully does so by stating that the session is recorded as video, trace, screenshots, and logs until session_close. It also discloses that the session is interactive and that this call returns both navigation results and a screenshot. It doesn't mention auth requirements or cleanup costs directly, but the lifecycle is adequately surfaced.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences with no filler: it front-loads the action and return values, then gives the essential lifecycle fact and the sibling-tool pointer. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description adequately covers the core flow: open, get id/navigation/screenshot, record until close, and use session_act for interactions. However, given the complexity of the options object and the absence of an output schema, an agent may still lack enough context about optional behaviors like headless mode, browser choice, and timeout semantics. It is usable for basic calls but not fully complete for advanced usage.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has near-zero description coverage for the parameters, and the description compensates only minimally by mentioning 'a URL.' The large 'options' object with many settings (headless, browser, mode, timeouts, etc.) is left entirely unexplained, so an agent has to infer or remember what those mean. This is below the minimum viable level given the schema's lack of help.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Open'), names the resource ('browser session on a URL'), and states the concrete return values: id, first navigation result, and screenshot. It also distinguishes itself from siblings like session_act and session_close, so an agent can tell this tool apart without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly implies when to use this tool: to start an interactive recorded session. It provides guidance for the next step by saying 'Use session_act to drive the page step by step,' which is a clear context cue. However, it does not explicitly state when not to use this tool versus alternatives, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

session_screenshotB

Take a screenshot of the current page of an open session (viewport or full page).

ParametersJSON Schema
NameRequiredDescriptionDefault
full_pageNo
session_idYes

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It discloses the basic behavior (screenshot) but does not mention whether the session must be visible/active, whether it waits for page load, what format the screenshot is returned in, or any side effects. For a tool with no annotation coverage, this is a significant gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single, front-loaded sentence that conveys the core action and the key option. No filler or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no annotations, no output schema, and 0% schema description coverage, the description is too thin. It does not explain the return value (e.g., image path/URL), prerequisites (open session), or failure conditions. An agent would need to guess at important invocation details.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It explains the full_page parameter's effect ('viewport or full page') but does not explain session_id semantics beyond the obvious. The description adds some value but leaves the agent to infer that session_id identifies the open session.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Take a screenshot') and resource ('current page of an open session'), and adds a scope qualifier ('viewport or full page'). It is clear and distinguishable from sibling tools like session_inspect or discover_elements, though it does not explicitly name a sibling alternative.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage context: it applies to an open session and can capture viewport or full page. However, it does not state when to prefer this over session_inspect or other inspection tools, nor does it mention prerequisites like needing an active session.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

validate_ticketA

Validate a ticket without opening a browser; returns errors and warnings.

ParametersJSON Schema
NameRequiredDescriptionDefault
ticketYes
optionsNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYes
errorsYes
warningsYes

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It discloses the most important behavior — no browser is opened — and that it reports errors and warnings. It does not clarify side effects, permissions, or whether options affect validation, but for a non-mutating validation tool the core guarantee is present.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One tight, front-loaded sentence with no filler. Every clause adds information: the operation, the no-browser guarantee, and the return type.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The purpose and no-browser behavior are enough to select the tool, and the output schema covers return details. However, the missing usage guidance and unexplained options make it incomplete for an agent deciding whether to call this before run_ticket or how to set options.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description should compensate, but it says nothing about the two parameters beyond the word 'ticket'. The optional 'options' parameter is entirely unexplained, leaving the agent to guess whether RunOptions affect validation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action ('validate a ticket'), the key constraint ('without opening a browser'), and the result type ('returns errors and warnings'). This clearly separates it from execution-oriented siblings like run_ticket, even though no sibling is named.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'without opening a browser' implies this is the safe, static validation step to use before running a ticket, but the description never states when to use it versus run_ticket or run_scenario, nor does it name alternatives or exclusions. The usage context is only implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 18 tool updatesv0.1.0
    • First observeddelete_run
    • First observeddiscover_elements
    • First observedget_artifact
    • First observedget_run
    • First observedget_schema
    • First observedlist_runs
    • First observedprobe_target
    • First observedrun_scenario
    • First observedrun_ticket
    • First observedserver_info
    • First observedsession_act
    • First observedsession_annotate
    • First observedsession_close
    • First observedsession_inspect
    • First observedsession_list
    • First observedsession_open
    • First observedsession_screenshot
    • First observedvalidate_ticket

TDQS

A3.5/5.0

Scored across 18 tools

Disambiguation4/5

Tools are grouped into clear clusters: target probing, session control, schema/validation, ticket/scenario runs, and run artifacts. The only real ambiguity is discover_elements vs session_inspect, which perform nearly the same ARIA snapshot/target query and differ only by whether an interactive session is already open; probe_target and session_open also both open a URL but have different intent.

Naming Consistency4/5

Most tools follow clear snake_case verb_noun names (validate_ticket, run_ticket, list_runs, delete_run), and the session_* prefix makes that cluster immediately recognizable. It is not completely uniform because session tools invert the order (session_open instead of open_session) and server_info is a noun_noun outlier, but the pattern is still predictable.

Tool Count4/5

18 tools is on the heavier side, but each one maps to a distinct stage of the workflow: one-off probing, session lifecycle, ticket/scenario execution, and artifact retrieval. The count is slightly above the ideal range but feels justified by the domain rather than bloated.

Completeness5/5

The tool surface covers the full lifecycle: probe/open, inspect/discover targets, act/assert, annotate/screenshot, close session, run tickets/scenarios, fetch runs/artifacts, and delete. There are no obvious dead ends or missing operations for a bug-verification server.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers

  • F
    license
    Not graded
    quality
    D
    maintenance
    Enables HTML page analysis, verification, and automated correction using Playwright for rendering and Mistral AI for visual inspection. Captures screenshots, analyzes renders against specifications, and generates fixes for HTML issues.
    2
    -
  • A
    license
    Not graded
    quality
    C
    maintenance
    Enables automated end-to-end testing powered by Playwright where test cases are defined in natural language and executed by AI. Uses lightweight snapshot analysis with vision mode fallback for sophisticated testing scenarios.
    3
    Apache 2.0
  • A
    license
    Not graded
    quality
    C
    maintenance
    Enables AI to perform real-browser QA testing on websites via Playwright, finding bugs, accessibility, SEO, and performance issues through natural language conversations.
    1
    MIT