Skip to main content
Glama

yatt-ts

Your own highly configurable YATT MCP server, as an npm library.

yatt-ts spins up a complete Model Context Protocol server that gives AI clients (Claude, Cursor, your own agents) full access to YATT — a UI-testing toolbox with 35 tools: a test library (create/edit/validate), a headless Playwright runner with HTML reports, a live Chromium browser with screenshots the AI can actually see, session management, visual baselines, spec export, and read-only SQL against your app's database.

Everything is configurable from a single typed config object: transport (stdio/HTTP), bearer auth, per-tool permissions, session persistence, artifact paths, browser defaults, retention. Strict validation fails fast and names the offending key — never a silent misconfiguration.

  • Runtime: Node ≥ 22.5 or Bun 1.4+ (dual-runtime, including the SQLite layer)

  • Engine: vendored Playwright Chromium, headless-first (screenshots always available), auto-downloaded on first use

  • License: MIT

Install

npm install yatt-ts
# or
bun add yatt-ts

Node 22.5+ or Bun 1.4+. On first browser use, Chromium (and chromium-headless-shell) is downloaded automatically — no extra setup.

Related MCP server: MCP Playwright Server

Quickstart (30 seconds)

Stdio server, zero config — the data root is your current working directory:

import { createYattServer } from 'yatt-ts';

const server = await createYattServer(); // all defaults (C28: zero-config works)
await server.start(); // speaks MCP over stdio

HTTP + bearer token — connect remote clients over the network:

import { createYattServer, generateToken } from 'yatt-ts';

const server = await createYattServer({
  http: { enabled: true, port: 3191, host: '127.0.0.1' },
  auth: { token: generateToken() }, // or your own string (min 16 chars) / tokenHash
});
await server.start();
// Clients send: Authorization: Bearer <token>  (401 otherwise; token never logged)

Or from the terminal (no code at all):

npx yatt-ts --version
npx yatt-ts --http --port 3191 --token my-secret-token-16ch
npx yatt-ts --read-only --root /path/to/yatt-data --locale es

--http without a token generates a crypto-secure ephemeral one and prints it once to stderr.

Point your MCP client at it (Claude Desktop / Cursor style):

{
  "mcpServers": {
    "yatt": { "command": "npx", "args": ["yatt-ts", "--root", "/path/to/yatt-data"] }
  }
}

What you get

  • 35 MCP toolsping, schema, test CRUD + validate/rename/duplicate/export (test_*), headless runner (test_run, test_run_dataset), reports (report_*), live browser (browser_*, tab_*, session_*), baselines (baseline_*), and read-only SQL (db_query).

  • Resources: yatt://schema, yatt://tests/{name}, yatt://reports/{name}.

  • Prompts: 5 ready-made work prompts, in English or Spanish.

  • Real browser evidence: browser_preview returns a PNG the AI can see; failed steps attach an evidence screenshot.

Configuration

Everything is optional; omitted keys take defaults. Unknown keys are rejected with an error that names the key.

Domain

Keys (default)

What it controls

paths

root (cwd), tests (tests), reports (reports), exports (exports), baselines (baselines), sessions (sessions), db (yatt.db)

Where every artifact lives. Relative paths resolve against root; ~ expands. Point root at an existing YATT desktop data folder — fully compatible.

http

enabled (false), port (3191), host (127.0.0.1), cors.origins (['*'])

Streamable HTTP transport instead of stdio.

auth

token | tokenHash

Bearer auth for HTTP. Plain token (≥16 chars) or its SHA-256 hex digest — never both. stdio ignores auth.

permissions

readOnly (false), allowTools, denyTools, denyBehavior ('error' | 'hide')

Mutating tools denied with an announced reason; denied tools stay visible ('error') or vanish from listings ('hide'); allowTools default-denies the rest.

sessions

persist (true)

false = browser sessions live in memory only: usable live, zero disk/DB writes, wiped on close.

storage

retention.maxAgeDays, retention.maxReports

Opt-in report cleanup. Absent = never deletes anything.

engine

enabled (true), runtime ('auto' | 'bun' | 'node' | binary path), autoInstallBrowser (true), timeouts

The vendored Playwright engine. enabled: false runs engine-free (browser/run/db tools fail clearly; ping reports 'deferred').

browser

defaultHeadless (true), toolbarInjection (false), defaultViewport (1280×800), engine ('chromium'), timeouts, cdpSync

Headless-first with full screenshot support; floating toolbar OFF unless you turn it on.

runner

defaultBrowser ('chromium'), stepTimeoutMs (40000), saveReport (true)

Headless run defaults.

appDb

{ type: 'sqlite', file } | { type: 'postgres', host, port, user, password | passwordProvider, database, ssl }

The app-under-test database for db_query (read-only). Credentials travel via env, never argv; passwordProvider resolves secrets at call time.

logging

level ('info', or 'silent')

Server diagnostics go to stderr (never the protocol channel).

locale

'en' (default) | 'es'

Language for prompts, tool descriptions, and messages.

Invalid values (wrong type, impossible path, both token forms, …) throw ConfigError at boot naming the exact key.

Security notes

  • CORS is permissive by default. http.cors.origins defaults to ['*'], mirroring the base YATT tool. For untrusted environments, restrict it to an explicit origin list in your config.

  • test_run variable overrides travel as CLI arguments to the one-shot engine process (e.g. --override token=…), matching the base-tool behavior. They are visible in the host process list (e.g. ps) for the duration of the run — avoid secret values in overrides, or contribute env-based override passing later.

Recipes

Permission, session, retention, and connection examples live in examples/ — each one is a runnable, self-contained file:

01-zero-config-stdio · 02-http-with-token · 03-read-only-server · 04-deny-list · 05-ephemeral-sessions · 06-custom-paths · 07-report-retention · 08-appdb-postgres-object · 09-nestjs-mcp-handler

Embed in your HTTP framework (NestJS)

Instead of the built-in HTTP server (http.enabled), you can expose the MCP endpoint at any route of your own HTTP framework — NestJS, Express, Fastify, plain node:http — with createMcpHttpHandler:

import { createYattServer, createMcpHttpHandler } from 'yatt-ts';

const yatt = await createYattServer({ engine: { enabled: false } });
const mcp = createMcpHttpHandler(yatt);

// Any framework route (Nest controller, Express router, …):
app.use('/api/mcp', (req, res) => mcp.handle(req, res, req.body));
  • When to use which: http.enabled for a standalone MCP endpoint you own end-to-end; the handler when an existing backend should host it (shared middleware, TLS, deployment).

  • Lifecycle is yours: do NOT call yatt.start() in handler mode (the handler connects the MCP server per session itself); on teardown call mcp.close() first, then yatt.shutdown(). In a Nest app call app.enableShutdownHooks() — without it Nest never fires onModuleDestroy on SIGINT/SIGTERM, so graceful teardown never runs.

  • Auth via host guards: no auth by default — your framework's guards/middleware rule. Or pass authenticate: (req) => boolean (false → 401 JSON, throw → controlled 500). Exception: if you configured auth.token/auth.tokenHash on the server itself, the handler wires it as the default bearer check (and logs a one-line notice) unless you pass your own authenticate.

  • Body size: the raw-request path rejects bodies over 2 MB with 413 (maxBodyBytes option to change it). Bodies pre-parsed by your framework bypass the cap — your parser owns that limit.

  • One client at a time per server (single browser engine): when a second client initializes, the stale session is evicted (new-wins).

  • Full NestJS recipe (controller + service + bearer guard): examples/09-nestjs-mcp-handler.ts.

Public API

import {
  createYattServer,     // (config?) → { server, ctx, start(transport?), shutdown() }
  createMcpHttpHandler, // mount the MCP endpoint inside your own HTTP framework
  resolveConfig,        // validate + apply defaults yourself
  ConfigError,          // named-key config failures
  generateToken,        // crypto-secure 64-hex token
  verifyToken, hashToken, redact,
  evaluateToolAccess, isToolListed,
  applyReportRetention,
  Store, openDatabase,
  VERSION,
} from 'yatt-ts';

start() with no argument picks the transport from config (HTTP when http.enabled, stdio otherwise) and handles SIGINT/SIGTERM; pass an MCP Transport (e.g. InMemoryTransport) to embed the server in your own process and own the lifecycle yourself. shutdown() is idempotent and always safe to call.

CLI

yatt-ts [serve] [--http] [--port N] [--host H] [--root PATH]
        [--token T | --token-hash HEX64] [--read-only] [--locale en|es]
        [--no-engine] [--allow-tool NAME] [--deny-tool NAME]
        [--deny-behavior error|hide]
yatt-ts --version | --help

--http without --token/--token-hash generates an ephemeral token and prints it once to stderr. App-database credentials are never accepted on the command line — configure them programmatically (D22).

Engine notes

  • Chromium only, headless-first. browser_open runs headless by default (D10); screenshots/preview work exactly the same. Pass headless: false when a human needs to watch.

  • Auto-download. First browser use installs chromium + chromium-headless-shell via the bundled Playwright. Disable with engine.autoInstallBrowser: false.

  • Floating toolbar OFF by default (D11). Enable with browser.toolbarInjection: true.

  • Runtime selection. The engine child process resolves bun → node automatically (override: engine.runtime). One browser at a time — engine calls are serialized by design (single concurrent MCP client).

  • No network needed for the server itself; the only download is the browser on first use.

Requirements

  • Node ≥ 22.5 (uses the built-in node:sqlite) or Bun ≥ 1.4 (bun:sqlite)

  • ~300 MB disk for the auto-downloaded Chromium

Development

npm run build            # tsc → dist/ (+ executable bin)
npm test                 # vitest unit suite
npm run typecheck        # src + test + examples
YATT_TS_E2E=1 npx vitest run test/e2e   # real-Chromium e2e smokes
node test/e2e/install.mjs               # npm pack + fresh-consumer install proof

License

MIT

Available Tools

35 tools
baseline_getC

Returns a reference image as PNG (for visual comparison).

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYes

TDQS

C2.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It states the output format (PNG) but doesn't disclose what happens if the named baseline doesn't exist, whether the image is returned inline or as a reference/URL, or any error behavior. For a retrieval tool, this is a meaningful gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that front-loads the action and output format. It earns its place with no filler, though it could add a brief usage hint without becoming bloated.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter retrieval tool, the description is nearly adequate, but it lacks error semantics and parameter clarification. With no annotations and no output schema, the agent has no way to know what a failure looks like or how the PNG is delivered. Sibling tools like baseline_list suggest a discovery workflow that isn't mentioned.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate for the undocumented 'name' parameter. The description says 'reference image' but doesn't explicitly state that 'name' is the baseline identifier, nor does it clarify the expected format (e.g., exact name vs. partial match). The agent can guess from the tool name, but the description adds no parameter-level meaning.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Returns') and resource ('a reference image as PNG'), and the purpose is clear: to fetch a baseline image for visual comparison. It doesn't explicitly distinguish from sibling tools like baseline_list, but the action is distinct enough that an agent can infer the difference.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is given about when to use this tool versus alternatives. The description doesn't mention that baseline_list should be used to discover available baselines first, nor does it explain when a reference image is needed. The agent must infer usage from the tool name and sibling context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

baseline_listA

Lists the saved reference images (visual asserts).

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It states a read operation ('Lists'), implying no side effects, but does not mention return format, pagination, or any access constraints. For a simple listing tool, this is adequate but not rich.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with no wasted words. It immediately states the action and the subject, making it easy to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simplicity of the tool (no parameters, no output schema), the description covers the essential purpose. It could mention the shape of the returned data (e.g., an array of image references), but the verb 'Lists' implies a list, so this is a minor gap. Overall, it is sufficient for a basic listing operation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so the description cannot add meaning beyond the schema. The baseline for 0-parameter tools is 4, and the description correctly omits any parameter details since none exist.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Lists') and a specific resource ('saved reference images'), clearly conveying that this tool returns a collection of visual assets. It distinguishes from the sibling 'baseline_get' by implying a listing action rather than retrieving a single item, though it does not explicitly name the alternative.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus siblings like 'baseline_get' or how it fits into a workflow. The description only states what it does without indicating context, prerequisites, or exclusions, leaving the agent to infer when listing is appropriate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

browser_click_atA

Click at viewport coordinates (x, y in CSS pixels); returns the resolved selector of the clicked element (data-testid → id → CSS path) plus the updated preview. Prefer browser_run_step with a selector for reproducible steps.

ParametersJSON Schema
NameRequiredDescriptionDefault
xYesX coordinate (CSS pixels)
yYesY coordinate (CSS pixels)

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure. It explains coordinate semantics (viewport, CSS pixels), the resolved selector fallback chain (data-testid → id → CSS path), and the updated preview. It could additionally disclose caveats like out-of-viewport coordinates or non-clickable elements, but it covers the core behavior well.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences: the first states the action and return value, the second provides usage guidance. No filler or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter coordinate click tool, the description covers the essential behavioral contract: what it clicks, how coordinates are interpreted, what it returns, and when to use the alternative. No output schema exists, so the return value description is especially valuable and sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description reinforces that x and y are viewport coordinates in CSS pixels but does not add meaningful new constraints or clarifications beyond the schema property descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Click'), a concrete resource ('viewport coordinates (x, y in CSS pixels)'), and the return value, which distinguishes it from sibling browser tools. It also explicitly differentiates itself from browser_run_step by indicating coordinate-based versus selector-based interaction.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear guidance on when to prefer an alternative: 'Prefer browser_run_step with a selector for reproducible steps.' This explicitly tells the agent the situational trade-off and names the sibling alternative.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

browser_closeA

Closes the controlled browser (clears the browser session).

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the burden. It discloses the main side effect (clears the browser session), which is helpful. However, it does not mention destructive nature (closing all open tabs) or whether the action is reversible, but for a close operation the intent is clear.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single sentence with no filler. The primary action ('closes the controlled browser') is front-loaded, and the parenthetical adds a key side effect without bloating the description.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a parameterless tool with low complexity, the description is adequate. It explains the core behavior and session-clearing effect. Though it doesn't mention return values or error conditions, these are unlikely to be critical for a close operation, and no output schema exists to demand more.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Tool has zero parameters, so schema coverage is trivially 100% and there is nothing to document. Baseline for 0 params is 4, and the description does not need to add parameter detail.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states a specific verb and resource: 'Closes the controlled browser.' It adds '(clears the browser session)' which distinguishes its effect from simply closing a tab, though it does not explicitly name sibling tools like tab_close. Still, the action is unambiguous and unique among the browser_* family.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives such as tab_close or session_delete. The description only states what it does without explaining when to choose it over other close/cleanup operations.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

browser_conditionA

Checks whether an element exists in the page (or whether a variable condition holds); waits for the condition with optional polling (timeoutMs > 0 repeats the check until true or the timeout expires). Returns {value, elapsedMs}.

ParametersJSON Schema
NameRequiredDescriptionDefault
valueNoAlternative: variable condition (e.g. {{status}} == ok)
selectorNoSelector of the element to check
timeoutMsNoIf > 0, polls until the condition is true or the timeout expires (default 0 = a single check)
intervalMsNoInterval between checks in ms (default 300)

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations present, the description carries the full transparency burden and does a solid job: it explains polling semantics, the meaning of timeoutMs > 0, and the return shape {value, elapsedMs}. It does not fully specify the return type of 'value' or behavior when both selector and value are provided, but the core runtime behavior is disclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with no filler: it states the purpose, the polling behavior, and the return shape in a compact, front-loaded way. Every clause earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with no annotations and no output schema, the description covers the essential return object and timeout behavior. It is missing guidance on the case where neither selector nor value is supplied, but for a condition-checking tool this is a minor gap given the otherwise thorough parameter schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all four parameters. The description adds only a marginal restatement of timeout behavior and does not provide extra meaning such as selector/value precedence or variable expression syntax, so the baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the tool as a condition checker for elements or variable conditions, using a specific verb ('Checks') and named resources ('element', 'variable condition'). It is clear in isolation, but it does not explicitly differentiate itself from nearby siblings like browser_eval, so it stops short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use the tool: when a condition needs to be checked or waited on, especially with polling via timeoutMs. However, it does not name alternatives or state when not to use it, leaving the comparison to siblings like browser_eval implicit rather than explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

browser_evalA

Executes arbitrary JavaScript in the current page and returns the value (useful to inspect the DOM, read texts, count elements, try selectors).

ParametersJSON Schema
NameRequiredDescriptionDefault
expressionYesJS expression (evaluated with the returned result)

TDQS

A3.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure, but it only states that arbitrary JS is executed and a value is returned. It does not warn that arbitrary code can mutate the page, navigate, trigger network requests, or leave persistent state, which is a significant transparency gap for an eval tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, well-structured sentence with the core behavior front-loaded and useful use cases in parentheses. There is no filler or redundant content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple in shape (one required string, no output schema), but arbitrary JS execution has open-ended side effects and evaluation semantics. The description covers why to use it and what it returns, yet omits prerequisites like an open page and behavior around asynchronous or non-serializable results, so it is only partially complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The only parameter is already fully described by the input schema ('JS expression (evaluated with the returned result)'), so the baseline is 3. The top-level description adds 'arbitrary JavaScript' and 'returns the value' but provides little additional semantic detail beyond what the schema already states.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb ('executes') and resource ('arbitrary JavaScript in the current page') and states that it returns the value. The concrete use cases ('inspect the DOM, read texts, count elements, try selectors') clearly distinguish it from the click/scroll/navigation sibling tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear, concrete use cases that tell an agent when to choose this tool, such as inspecting the DOM or trying selectors. It does not explicitly mention alternatives or when not to use it, so it falls just short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

browser_openB

Open (or reopen) the controlled browser. headless=true by default; visible only when you need to point by hand.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlNoInitial URL (default about:blank)
browserNoEngine (default chromium)
sessionNoName of a saved session (cookies/localStorage). With sessions.persist:false the state is restored inline from memory
headlessNoHeadless mode (default true)
viewportNoViewport size (default 1280×800)
timezoneIdNo
geolocationNo

TDQS

B3.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description bears the full disclosure burden, but it mostly restates the schema's headless default and adds only the 'point by hand' rationale. It does not say what happens if a browser is already open, whether reopening restores state, what the call returns, or how it can fail.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is one tight, front-loaded sentence with no filler. The semicolon cleanly separates the open action from the headless/visible guidance, even if the headless default also appears in the schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with seven parameters, nested objects, no output schema, and no annotations, this description is too thin. It omits tool-level behavior around existing browser instances, session restore, return values, and failure modes, so an agent cannot fully predict the tool's effect beyond what the schema implies.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already covers most parameter meanings and defaults, and the description adds a useful behavioral gloss for headless via 'visible only when you need to point by hand.' However, it does not help with the schema's undocumented parameters like timezoneId or geolocation, leaving the semantics only partially enriched.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb and resource: 'Open (or reopen) the controlled browser,' so the core action is unmistakable. It does not explicitly contrast itself with sibling tools like browser_status or tab_open, so differentiation comes from the clear verb rather than an explicit comparison.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The headless/visible guidance is a concrete decision rule: use headless by default, switch to visible only when manual pointing is needed. It does not mention alternatives or exclusion conditions, such as when to use browser_status or tab_open, so it stops short of full routing guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

browser_previewA

Captures the current viewport as a PNG image (the AI sees the page) + url, title, scroll and dimensions. It is the main visual inspection tool.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the behavioral burden. It transparently indicates a non-mutating capture action ('Captures') and discloses the output: PNG, URL, title, scroll, and dimensions. It does not explicitly state that it does not alter the page or what happens if no browser is open, but the action is clearly read-only in intent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with no real filler. The parenthetical '(the AI sees the page)' is slightly redundant with 'visual inspection tool,' but it reinforces the purpose and the description remains compact and front-loaded with the core capture behavior.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a parameterless screenshot tool with no output schema, the description adequately covers what is returned (PNG plus metadata) and when to use it. It is slightly incomplete in not mentioning prerequisites like an already-open browser, but that is a minor gap given the simplicity of the tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so the baseline is 4. The description correctly adds no unnecessary parameter detail and instead focuses on what output is produced, which is exactly what matters for a no-argument tool.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Uses a specific verb ('Captures') with a clear resource ('current viewport as a PNG image') and enumerates the extra metadata returned. It also labels itself as 'the main visual inspection tool,' which clearly positions it relative to sibling inspection tools like browser_eval and browser_status.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'main visual inspection tool' gives clear context for when to use it: whenever the agent needs to visually inspect the rendered page. However, it does not explicitly name alternatives or exclusion criteria, such as 'use browser_eval for DOM-level inspection instead.'

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

browser_run_stepA

Runs a YATT (leaf) step in the current page: click, type, hover, assert_*, goto, wait_visible, etc. Returns ok/error, duration and, on failure, an evidence screenshot. With vars it interpolates {{name}} into the step before running it. Structure steps (if/repeat/for_each/run_flow) run with test_run, not here.

ParametersJSON Schema
NameRequiredDescriptionDefault
stepYesYATT step: {action, selector?, value?, attribute?, disabled?}
varsNoVariables to interpolate {{name}} into the step (e.g. { "email": "a@b.com" })
timeoutMsNoTimeout in ms (default 40000)

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the behavioral burden and does a solid job: it discloses the return payload (ok/error, duration, failure evidence screenshot), variable interpolation behavior, and the leaf-only step restriction. It does not mention side effects on the browser page or prerequisites like an already-open page, but core behavioral traits are covered.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tightly packed sentences with no filler: purpose and examples first, then return behavior, then interpolation and exclusion. Every sentence earns its place and the most important routing information is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with nested objects, three parameters, and no output schema, the description covers the essential call semantics: what steps are accepted, what happens on success/failure, and how vars are applied. The only minor gap is the unspecified format/location of the evidence screenshot, but overall the agent has enough to invoke it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds value by explaining the step object with action examples and clarifying how vars interpolate {{name}} placeholders before execution. This goes beyond the schema's field descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: it runs a YATT leaf step in the current page, with concrete examples of supported actions (click, type, hover, assert_*, goto, wait_visible). It also explicitly distinguishes itself from structure-step execution handled by test_run.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear usage context by defining what kind of steps belong here and explicitly saying structure steps (if/repeat/for_each/run_flow) should go to test_run instead. It does not enumerate all alternative tools, but the primary routing decision is handled.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

browser_scrollA

Scrolls the page vertically (dy in pixels, positive moves down) and returns the updated preview.

ParametersJSON Schema
NameRequiredDescriptionDefault
dyYesPixels to scroll

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the behavioral burden. It discloses that scrolling is vertical, that positive dy moves down, and that the tool returns an updated preview. However, it does not mention behavior at scroll limits, whether a browser/page must already be open, or any failure conditions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One sentence that is fully front-loaded with the action, parameter semantics, and return value. Every phrase earns its place and there is no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter scrolling tool with no output schema, the description provides enough to call it correctly: what changes, how direction works, and what is returned. The main gap is the lack of explicit preconditions or failure behavior, which is minor given the simplicity of the tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema covers dy 100% with 'Pixels to scroll', and the description adds the directional meaning (positive moves down), which is genuinely useful beyond the schema. No other parameter constraints or formats need clarification.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names the resource (page), the action (scrolls vertically), and the core parameter semantics (dy in pixels, positive moves down). It is clear and matches the tool name, but it does not contrast itself against sibling browser tools such as browser_eval or browser_click_at.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no statement about when to choose browser_scroll over its sibling tools, nor any exclusions or preconditions such as requiring an open page. The only guidance is how scrolling works, not when to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

browser_statusB

State of the controlled browser: open/closed, engine, URL and interaction.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. 'State of the controlled browser' implies a read-only query, but the description never explicitly states that it does not modify the browser, nor does it disclose side effects, permissions, or limitations. The term 'interaction' is also ambiguous and does not clarify actual behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single front-loaded sentence with a clear subject and a colon-delimited list of contents. Every word earns its place, and there is no filler or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a no-parameter status getter, the description covers the main elements of the returned state: open/closed, engine, URL, and interaction. Since there is no output schema, it would benefit from clarifying what 'interaction' means and whether the response includes more fields, but overall it is reasonably complete for a simple status tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so there is no parameter documentation burden on the description. The baseline of 4 applies here, and the description appropriately focuses on what the state reports rather than on parameter details.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the resource as the state of the controlled browser and enumerates its contents: open/closed, engine, URL, and interaction. It lacks an explicit verb like 'returns' or 'gets,' but the noun-phrase form is unambiguous and easy to distinguish from action-oriented siblings like browser_open and browser_eval.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives, and it mentions no exclusions or prerequisites. There is an implicit sense that one would use it to check browser state, but the description does not state that or contrast it with related tools such as browser_open, browser_close, or tab_list.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

db_queryA

Executes a read-only query (SELECT/WITH/EXPLAIN/PRAGMA) against the app-under-test database. Pass the connection in db (SQLite path or "file:" / postgres:// URL); otherwise the engine-side YATT_APP_DB/--app-db configuration applies. Returns {columns, rows, totalRows}; rows are capped at 200.

ParametersJSON Schema
NameRequiredDescriptionDefault
dbNoConnection: SQLite path or "file:" / postgres:// URL (overrides the global configuration)
sqlYesRead-only SQL query (SELECT, WITH, EXPLAIN or PRAGMA)

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and delivers: it explicitly states the operation is read-only, caps rows at 200, and defines the return shape {columns, rows, totalRows}. It also discloses the connection fallback behavior. This goes beyond a simple action statement.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with zero redundancy. The first sentence states the action and scope; the second covers connection override, return shape, and row cap. All information is front-loaded and essential.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter read-only query tool, the description covers everything needed: supported SQL verbs, connection handling, return format, and row limit. No gaps remain for an agent to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema documents both parameters. The description adds meaningful context: it clarifies that 'db' is a connection string (SQLite path or URL) that overrides the global config, and that 'sql' must be a read-only query. This adds value beyond the schema's brief descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Executes') with a clear resource ('read-only query... against the app-under-test database'), and explicitly enumerates the supported SQL verbs (SELECT/WITH/EXPLAIN/PRAGMA). This clearly distinguishes it from sibling tools, none of which handle raw database access.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context: it explains the db parameter overrides the global configuration, and notes the fallback to YATT_APP_DB/--app-db. It also limits usage to read-only queries. While it doesn't name an alternative (none exists among siblings), the guidance is sufficient for correct usage.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

pingA

Checks that the MCP server and the engine (sidecar) respond.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden. It discloses that the tool performs a connectivity check on both components, which is the core behavior. However, it does not mention whether it is read-only, what happens on failure, or any side effects—though for a ping these are largely implicit. It is not misleading, but it is minimal.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single sentence that is entirely on-topic and front-loaded. There is no wasted text, and the purpose is communicated immediately.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simplicity (no parameters, no output schema, no annotations), the description is adequate for an agent to know what the tool does and when to call it. It could mention that it returns a status, but that is not critical for a ping tool. Overall, nothing essential is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, and the schema description coverage is 100% (empty properties). With no parameters, the description cannot add parameter meaning, and the baseline of 4 is appropriate. The description does not need to elaborate further.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('checks') and a clear resource scope ('the MCP server and the engine (sidecar) respond'). It is unambiguous and distinct from all sibling tools, which are clearly about other operations. For a health-check tool, this is exactly what an agent needs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage (when you need to verify connectivity) but provides no explicit when-to-use or when-not-to-use guidance. Since there are no sibling tools with similar purposes, alternatives are not needed, but the lack of any contextual advice keeps this at a mid score.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

report_deleteA

Deletes a saved report (DB + file under reports/).

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYes

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

There are no annotations, so the description carries the full burden of behavioral disclosure. It does a good job by revealing that deletion affects both the database record and the file under reports/. It stops short of stating irreversibility or error behavior, but the destructive scope is clearly communicated.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The entire description is one compact sentence with no filler. The verb is front-loaded, and the parenthetical detail about DB + file is substantive rather than redundant.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description is minimally viable for a simple destructive tool: it says what is deleted and the single required parameter is inferable. However, it lacks guidance on finding valid report names, error behavior when the name is not found, and confirmation of permanence, which leaves some context gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate for the undocumented 'name' parameter. It only implies that 'name' identifies the saved report and does not clarify how names are resolved, expected formats, or how to obtain valid names. This is a meaningful gap for a single-parameter tool.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Deletes') and resource ('saved report'), and includes the scope of deletion ('DB + file under reports/'). This clearly distinguishes it from read-only siblings like report_get and report_list and from other delete tools like test_delete.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the tool is for removing a saved report, which separates it from retrieval tools. However, it does not explicitly state when to use it versus alternatives, nor mention any prerequisites such as first using report_list to find valid names.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

report_getA

Returns the content of a report (parseable JSON). To diagnose failures, the report has steps[] with status ok/fail, error and ms.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesReport name (see report_list, e.g. "my-test-YATT-20260902-101500.json")

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It discloses that the tool returns parseable JSON and describes the internal structure of a report (steps[] with status ok/fail, error and ms), which is valuable behavioral context for an agent diagnosing failures. It doesn't mention error behavior or permissions, but for a read-only retrieval tool, the disclosed structure is substantial.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with no waste. The first sentence states the core function and format; the second adds diagnostic context about the report structure. Information is front-loaded and every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter read tool with no output schema, the description is quite complete. It explains what the tool returns and the structure of the content. It could mention what happens if the report doesn't exist or is malformed, but given the simplicity and the schema's example, this is a minor gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents the 'name' parameter well, including an example format. The description adds the hint that the name comes from report_list, which is already in the schema's description. Baseline 3 is correct because the schema does the heavy lifting and the description adds minimal extra meaning.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Returns') and resource ('content of a report'), and clarifies the format ('parseable JSON'). It distinguishes itself from report_list and report_delete by focusing on content retrieval. The mention of steps[] with status ok/fail, error and ms further clarifies what the report content looks like.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use this tool: when you need the content of a report, especially to diagnose failures. It doesn't explicitly state when not to use it or name alternatives, but the sibling context (report_list, report_delete) and the schema's reference to report_list provide clear context. A 4 is appropriate because the usage context is clear but exclusions are not explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

report_listA

Lists the saved run reports (names .json and .html).

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the behavioral burden. It signals a non-mutating operation through 'Lists' and discloses the output composition: names ending in .json and .html. It does not detail sorting, paths, or empty-case behavior, but for a simple read-only listing this is reasonable.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One sentence, front-loaded with the action, and the parenthetical adds useful detail without any filler. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a parameterless tool with no output schema, the description is nearly complete: it states the resource, the scope ('saved'), and the returned names. It could be more explicit about the exact return structure, but the naming detail is sufficient for an agent to consume the result.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There are zero parameters, so the baseline is 4. The description correctly spends no space on parameters, and the naming convention in the description provides the only relevant context an agent needs.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Lists') with a concrete resource ('saved run reports') and defines the exact naming pattern (<slug>.json and <slug>.html). This clearly distinguishes it from siblings like report_get and report_delete, which operate on individual reports rather than enumerating them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear context: use this tool when you need the set of saved run report names. It does not explicitly name alternatives or exclusions, but for a parameterless enumeration tool the usage context is sufficiently clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

schemaA

Complete documentation of the YATT test format (schema v1): fields, actions, variables, dataset and usage guide. Read it before creating tests.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description bears the full burden of behavioral disclosure. 'Complete documentation' strongly implies a read-only informational call and lists the topics covered, but it does not explicitly state that the tool has no side effects or describe the output format or size. For a zero-parameter schema reference, this is an acceptable but not fully transparent description.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, information-dense sentence with a clear imperative. Every phrase contributes value: schema version, covered topics, and the call-to-action before test creation. There is no redundancy or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a parameterless documentation tool with no output schema, the description provides the necessary information: what the documentation covers, which version it documents, and when the agent should call it. An agent can correctly decide to invoke this tool before creating tests without needing additional detail.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool accepts zero parameters and the input schema is an empty object, so there are no parameter semantics to clarify. The description's list of documentation contents adds useful context beyond the empty schema, even though parameter-specific explanation is not needed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the tool as documentation for the YATT test format and enumerates its contents: fields, actions, variables, dataset, and usage guide. It distinguishes itself from sibling test_* tools by positioning itself as the format reference to read before authoring tests. It lacks an explicit action verb like 'returns' or 'gets', but 'Complete documentation' effectively communicates the tool's role.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states when to use the tool: 'Read it before creating tests.' This ties usage to the test-creation workflow and makes the tool's purpose actionable. It does not explicitly name alternatives or state when not to use it, but no sibling tool serves the same documentation role, so the guidance is sufficient.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

session_deleteA

Deletes a saved session.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesSession name

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does state the destructive action explicitly. However, it does not mention irreversibility, behavior when the session does not exist, or any side effects on related data.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, direct sentence with no filler or redundant information. It is appropriately front-loaded and easy to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter delete tool with no output schema, this is minimally adequate: it names the operation and target. It lacks edge-case behavior and confirmation details, but nothing structurally important is missing for a basic call.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the description adds no additional meaning or constraints beyond what the schema already provides for the 'name' parameter. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description uses the specific verb 'Deletes' and identifies the target as 'a saved session', making the operation unambiguous. It also distinguishes clearly from sibling tools like session_save and session_list.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is given on when to use this tool versus alternatives, nor any exclusions or prerequisites. The decision to use it is only implied by the tool name and the one-line description.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

session_listB

Lists the saved sessions.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations present, the description must carry the behavioral disclosure burden. It indicates a read-only listing operation but does not disclose whether it returns all sessions, the result format, ordering, or any side effects. It is minimal and leaves important behavior unstated.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single short sentence that directly states the operation. It is appropriately sized for such a simple tool with no parameters and no complex behavior.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a parameterless listing tool with no output schema, the description is mostly sufficient to invoke it. However, it lacks any detail about what a 'session' is or what the returned list contains, which would help an agent confirm the result is what it expects.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has zero parameters)Skip, so the baseline is 4. There are no parameter semantics for the description to clarify, and the description does not mislead about parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a clear verb and resource: "Lists the saved sessions." It is obvious what the tool does.among the sibling tools, though it doesn't explicitly distinguish itself from other list tools, the tool name and scope make its purpose clear.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives such as session_save or session_delete. There is no mention of use cases, exclusions, or conditions, leaving the agent to infer when listing sessions is appropriate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

session_saveA

Saves the current session state (cookies/localStorage) under a name, for tests with prior authentication.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesSession name

TDQS

A3.6/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full burden. It discloses what is saved (cookies/localStorage) but does not address key behaviors: what happens if the name already exists (overwrite or error), whether an active browser session is required, or failure conditions. This is a mutation-like operation with significant gaps in side-effect disclosure.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that front-loads the action and purpose. Every word contributes meaning without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter save operation with no output schema, the description covers the essential purpose and parameter adequately. It explains why the tool exists and what it stores. Minor gaps like overwrite behavior and session requirement prevent a 5, but the low complexity makes it reasonably complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% with a basic 'Session name' description. The tool description adds only the phrase 'under a name', which clarifies the parameter's role but not constraints or uniqueness. Baseline 3 applies since the schema already documents the parameter.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (saves), a concrete resource (session state, explicitly cookies/localStorage), and the context (for tests with prior authentication). This distinguishes it from sibling tools session_list and session_delete, making the intent unmistakable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives a purpose context ('for tests with prior authentication') which implies when to use it, but it does not explicitly state when not to use it or compare against alternatives like session_list or session_delete. No exclusions or prerequisites are mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

tab_closeA

Closes a tab (by index; without an index closes the active one).

ParametersJSON Schema
NameRequiredDescriptionDefault
indexNoTab index (see tab_list)

TDQS

A3.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the operation (close) but does not mention that closing a tab is irreversible, what happens on invalid index, or whether it returns a confirmation. This is a destructive action, and the description offers no warning or detail about side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single concise sentence that front-loads the primary action and includes the key qualifier about index behavior. There is no filler or redundant information; every word contributes to understanding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple single-parameter tool, the description covers the core behavior but omits important operational context such as error handling (out-of-range index), side effects (data loss), and return value. Given no annotations and no output schema, the description should at least hint at the destructive nature and possible failure modes. It is adequate for a basic use case but not fully complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema provides a description for the index parameter, but the tool description adds crucial semantics by explaining that omitting the index closes the active tab. This goes beyond the schema's 'Tab index (see tab_list)' by clarifying the default behavior, which is essential for correct invocation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action (closes) and resource (tab), and specifies the index-based behavior. It distinguishes itself from siblings like tab_open and tab_switch by its verb and target. The mention of 'active one' adds specificity that disambiguates from other tab operations.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explains how to specify which tab to close (by index, or default to active), which serves as usage guidance. However, it does not explicitly state when to use this tool versus alternatives like browser_close or tab_switch, nor does it mention any exclusions or prerequisites (e.g., must have tabs open).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

tab_listA

Lists the open tabs: index, active, title and URL.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It clearly signals a read-only listing operation and specifies the output fields, which is substantial transparency for a simple zero-parameter tool. It doesn't elaborate on edge cases like empty tab list or ordering, but nothing suggests hidden side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single, front-loaded sentence with no filler. Every word adds information, and the list of returned fields is presented compactly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter, no-output-schema tool, the description fully covers what an agent needs: what it does and what information it returns. The tool is trivial to invoke, and the listed fields (index, active, title, URL) are sufficient context for interpreting the result.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, and the rubric gives a baseline of 4 for such cases. The description doesn't need to add parameter meaning; it correctly avoids mentioning parameters that don't exist.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Lists') and a clear resource ('open tabs') and enumerates the returned fields (index, active, title, URL). This distinguishes it from the modifying sibling tools like tab_open, tab_switch, and tab_close without needing to inspect them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the tool is for inspecting current open tabs, but it provides no explicit when-to-use guidance or alternatives. With siblings like tab_open and tab_switch, the intended usage is inferable, but there is no stated exclusion or comparison.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

tab_openA

Opens a new tab (optionally with a URL) and returns the tab list.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlNoInitial URL of the new tab

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden. It discloses the core action and return value, but does not mention whether the new tab becomes active, how duplicates are handled, or what happens if the URL is invalid.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One efficient sentence that front-loads the action, then states the optional parameter and the return value with no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with one optional parameter and no output schema, the description provides the essential info: what it does and what it returns. It could be more complete by noting tab focus or failure behavior, but those are minor for this simple operation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the parameter is already documented as 'Initial URL of the new tab'. The description only adds 'optionally', which matches the schema's zero required parameters but does not add deeper meaning.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: 'Opens a new tab'. Also notes the callback behavior—'returns the tab list'—which distinguishes it from sibling tab tools like tab_switch and tab_close.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied rather than explicit. The description tells the agent what the tool does but gives no guidance on when to choose it over alternatives such as browser_open or tab_list, and provides no exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

tab_switchA

Switches to the tab with the given index (0-based).

ParametersJSON Schema
NameRequiredDescriptionDefault
indexYesTab index (see tab_list)

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description conveys the core state-changing behavior (switching the active tab) and the 0-based indexing convention, which is non-obvious. However, with no annotations and no output schema, it does not disclose return behavior or what happens on an out-of-range index, so transparency is only minimally adequate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single, front-loaded sentence states action, target, and indexing convention with no filler. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter action with no output schema, the definition supplies the necessary invocation details: what to switch, how to identify the tab, and how the index is counted. It is complete enough for an agent that can obtain the index from tab_list, and the sibling context covers the rest.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema documents the only parameter, index, and the description adds the crucial detail that it is 0-based, which is not present in the schema property description. Since schema coverage is 100%, the description goes beyond the schema by clarifying indexing semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Switches') with a clear resource ('tab') and the selection criterion (index), making the operation unambiguous. It is easily distinguished from siblings like tab_open, tab_close, and tab_list, which have different verbs and effects.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the tool is for activating an existing tab by index, but it does not explicitly state when to prefer it over tab_open/tab_close. The schema's 'see tab_list' hint provides some guidance, but the description itself lacks explicit when-to-use or alternative routing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

test_createA

Creates a new test. Accepts the content as a JSON string or as an object. Validates the schema (schemaVersion, steps, actions). Fails if the name already exists unless overwrite: true.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoName of the saved test
contentYesTest content: JSON string or object
overwriteNoOverwrite if it already exists (default false)

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the behavioral burden. It discloses meaningful behaviors beyond the schema: validation against schemaVersion/steps/actions, and failure on name collision unless overwrite is true. It does not mention auth, rate limits, or return values, but the key behavioral traits are present.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with no filler. The core verb ('creates') is front-loaded, followed by input form, validation, and conflict behavior. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple 3-parameter create tool with no output schema and no annotations, this description is largely complete: it covers input format, validation, and overwrite handling. Minor gaps remain around the optional name parameter's behavior and the return value, but complexity is low and schema coverage is complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds value beyond the schema. It clarifies that content may be provided as a JSON string or object and must conform to a schema containing schemaVersion, steps, and actions, and it explains overwrite's conflict semantics, which the schema only hints at.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with 'Creates a new test,' which is a specific verb and resource that clearly distinguishes it from sibling tools like test_list, test_update, test_delete, test_rename, and test_duplicate. It also states the main input behavior and validation scope, leaving no doubt about the tool's intended role.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context for when to use the tool: creating a new test, accepting content as a JSON string or object, validating a schema, and handling name conflicts with overwrite. It does not explicitly name alternative tools or exclusion conditions, so it is not a 5, but the context is sufficient for an agent to route correctly.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

test_deleteA

Deletes a test from the library (database and tests/.yatt.json file).

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesName of the saved test

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden and does disclose the destructive side effect, explicitly naming both the database record and the file as affected. It stops short of stating irreversibility or error behavior, but the core behavioral surprise is covered.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence conveys action, resource, and side effects with no filler. The parenthetical adds the important file detail without bloating the description.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter destructive tool with no output schema, the description is complete enough to call safely: it identifies the required input and what will be affected. It could mention permanence or return value, but those are minor for this simple operation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% for the single `name` parameter, giving a baseline of 3. The description adds value by showing that the parameter maps to both the database entry and the `tests/<name>.yatt.json` file path, which is not in the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Deletes') and resource ('a test from the library'), and further specifies the two storage locations (database and the .yatt.json file). This clearly separates it from sibling list/update/duplicate/validate tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The usage context is clear: use this tool to remove a saved test by name. It does not explicitly enumerate when-not-to-use or alternatives, but no sibling tool performs deletion of tests, so the exclusions are implicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

test_duplicateA

Duplicates an existing test under a new name (default " (copy)").

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesName of the saved test
newNameNoNew name

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure. It adds a useful detail: the default generated name when newName is omitted. However, it does not mention collision behavior, permissions, or whether the original remains untouched, which is important for a mutation-like operation with no annotation coverage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, polished sentence that front-loads the core operation and includes the essential default behavior in a parenthetical. There is no filler or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple two-parameter duplication tool, the description plus schema is mostly sufficient for selecting and invoking it correctly. However, with no annotations and no output schema, it leaves return behavior and error/name-collision semantics unspecified. It is adequate but not fully complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so both parameters are already documented. The description adds meaningful value by clarifying that newName is optional and defaults to '<name> (copy)', which is not expressed in the schema. This goes beyond the schema's terse 'New name'.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description explicitly states 'Duplicates an existing test under a new name', naming both the operation and the resource with a specific verb. The default naming parenthetical adds further precision. This clearly distinguishes the tool from sibling operations like test_create and test_rename.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The verb 'Duplicates' implies the intended use case, and 'existing test' signals that it applies to already-saved tests. However, it does not explicitly mention alternatives such as test_create or test_rename, nor does it state when not to use this tool. Usage guidance is implied rather than explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

test_export_playwrightA

Generates the code of a saved test as a TypeScript spec (.spec.ts), in Playwright (@playwright/test) or Jest (jest-environment-playwright) format, with sub-flows embedded as functions. Returns the content; with write: true it saves the file into exports/.spec.ts like the app.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesName of the saved test
writeNoWrite the file into exports/ (default false)
formatNoFormat of the generated spec (default "playwright")

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden. It transparently discloses that the tool returns content by default and writes a file when write is true, including the file path pattern. This covers the key side-effect behavior, though it does not detail error handling or permissions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with zero fluff. The primary action is front-loaded, and the side-effect condition is stated succinctly. Every clause earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given three parameters, no output schema, and no annotations, the description covers the essential operational aspects: what is generated, the return value, and the write behavior. It does not elaborate on error cases or the internal structure of the generated code, but these are not critical for invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with each parameter described, so the baseline is 3. The description adds extra value by specifying the exact file name pattern (exports/<name>.spec.ts) and noting 'like the app', which enriches the meaning of the write parameter beyond the schema's simple 'Write the file into exports/'.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Generates' with a specific resource 'saved test' and the output type '.spec.ts'. It distinguishes from sibling tools like test_get by focusing on code generation rather than data retrieval, and specifies both Playwright and Jest formats.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the use case (generating exportable code) but does not explicitly state when to use this tool over alternatives. It does not mention any exclusions or alternative tools, leaving the selection to inference from the purpose.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

test_getB

Returns a saved test as a full JSON object (steps, variables, environments, dataset).

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesName of the saved test

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden of behavioral disclosure. It does communicate a read-only retrieval and specifies the returned payload, but it does not mention error/not-found behavior, permissions, or any side-effect guarantees.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single efficient sentence that front-loads the verb and resource and then states the output payload. There is no filler or redundant information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter read-only tool with no output schema, the description is largely complete: the schema covers the input and the description explains the returned JSON structure. It omits not-found behavior and alternative tool routing, but those are minor for this simple getter.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% for the single parameter, which already documents 'name' as the name of the saved test. The description adds no additional parameter semantics such as format, uniqueness, or case sensitivity, so the baseline score applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Returns') and identifies the resource ('a saved test'), with concrete payload details (steps, variables, environments, dataset). It is clear this fetches a single saved test, though it does not explicitly contrast itself with test_list or other siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is given about when to use this tool versus alternatives like test_list or test_export_playwright. The agent must infer the usage boundary from the name and the phrase 'full JSON object', so the description leaves this implicit rather than explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

test_listA

Lists the tests saved in the YATT library (names).

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

There are no annotations, so the description carries the full burden. It clearly states this is a read-only listing action and specifies that it returns names rather than full test details. It does not mention ordering or edge cases, but those are minor for a simple list tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence that contains precisely the necessary information: what is listed and what is returned. There is no filler or redundant repetition of the tool name.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter tool with no output schema, this description is nearly complete. It tells the agent the tool returns test names from the YATT library. The only slight gap is the lack of any mention of response format, but that is not essential for invoking the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters)Skip, so there is no parameter meaning to document. The baseline of 4 applies: the schema is complete and the description adds no misleading information.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb ('Lists'), a specific resource ('tests'), and the scope ('YATT library') with an explicit output ('names'). This clearly distinguishes it from sibling tools like test_get or test_create, which cover different actions on tests.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the tool should be used when you need an overview or list of saved test names, but it does not explicitly compare against alternatives or state when not to use it. For a zero-parameter listing tool this is acceptable but not fully explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

test_renameA

Renames a test in the library (updates the DB and the mirror file).

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesName of the saved test
newNameYesNew name

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It explicitly states that the operation 'updates the DB and the mirror file,' which is a meaningful side-effect disclosure beyond the simple rename semantics. However, it does not mention potential failures, uniqueness constraints, or reversibility.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single compact sentence that front-loads the action and then adds the essential side-effect detail. There is no filler, redundancy, or unnecessary complexity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter rename operation with no output schema, the description is largely sufficient: it identifies the resource, the operation, and the persistence effects. It does not state constraints like 'newName must be unique' or 'test must exist,' but those are reasonably inferable from standard rename semantics.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and both parameters already have clear descriptions ('Name of the saved test' and 'New name'). The tool description adds no additional parameter-level meaning, so the baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Renames') and a clear resource ('a test in the library'), and even notes the underlying effect on the DB and mirror file. This distinguishes it from sibling tools like test_update or test_duplicate because the action is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description offers no guidance on when to use this tool versus alternatives such as test_update, test_duplicate, or test_delete. There are no exclusions, prerequisites, or preferred scenarios given; the only usage signal is the verb 'Renames,' which is implied rather than explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

test_runB

Runs a saved test headless (Chromium engine by default) and saves a report in the library. Parameters: variable environment, per-run overrides, per-step timeout. Returns the summary with the steps and the report name.

ParametersJSON Schema
NameRequiredDescriptionDefault
envNoVariable environment (default "default")
urlNoOverrides the initial URL
nameYesName of the saved test
browserNoEngine (default chromium)
overridesNoVariable values that win over the environment
saveReportNoSave report in the library (default true)
stepTimeoutMsNoTimeout per step in ms (default 40000)

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the behavioral burden. It discloses that execution is headless, defaults to Chromium, saves a report, and returns a summary with steps and report name. It does not mention that saveReport can disable report saving, nor does it describe failure behavior or other side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences with front-loaded core behavior followed by key parameters and return value. The parameter recap is somewhat redundant with the schema, but the description remains efficient and scannable.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers the main behavior and return format, which partially compensates for the missing output schema. However, with 7 parameters and no annotations, it leaves out important nuances such as the optional report-saving behavior and the distinction from dataset-driven runs. Adequate but not comprehensive.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already documents every parameter. The description adds brief grouping ('variable environment, per-run overrides, per-step timeout') but no real semantic value beyond what the schema provides; baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific action ('runs a saved test') and the context (headless, default Chromium engine, saves a report). It is easy to distinguish from listing/deleting tools, but it does not explicitly differentiate from the sibling test_run_dataset, so it falls short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The intended use is implied by 'Runs a saved test' plus the required 'name' parameter. However, there is no explicit when-to-use guidance, no exclusion criteria, and no mention of alternatives such as test_run_dataset or browser_run_step.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

test_run_datasetA

Data-driven: runs a test once per row of overrides and returns each row result plus totals. No report is saved.

ParametersJSON Schema
NameRequiredDescriptionDefault
envNoVariable environment (default "default")
nameYesName of the saved test
rowsYesList of rows; one run per row (values = variable overrides)
browserNoEngine (default chromium)
stepTimeoutMsNoTimeout per step in ms (default 40000)

TDQS

A3.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It does mention that no report is saved and that results are returned, which is useful. However, it omits critical behavioral traits such as whether the operation is read-only or mutating, whether it can be safely repeated, any side effects on the test or environment, or permission requirements. The lack of such disclosure for an execution tool is a significant gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, information-dense sentence that front-loads the key concept ('Data-driven') and immediately states the core behavior, return type, and a notable exclusion (no report saved). There is no redundancy, fluff, or unnecessary detail. Every word contributes to understanding the tool's purpose and key differentiator.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with no output schema and no annotations, the description provides the core functional behavior but leaves out important contextual details. It does not describe the structure of the returned results, error handling, limits on rows, or any prerequisites like requiring a saved test. It also does not disclose safety or side effects, which is especially relevant for an execution tool. The description is adequate for a basic understanding but not fully complete for safe and correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all parameters have descriptive text in the schema. The tool description adds a small amount of semantic value by clarifying that 'rows' represent overrides and that each row corresponds to one run, which aids understanding of the rows parameter. However, it does not elaborate on other parameters like env, browser, or stepTimeoutMs, so the added value over the schema is marginal. This aligns with the baseline of 3 for high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a clear, specific behavior: runs a test once per row of overrides and returns per-row results plus totals. It differentiates from siblings like test_run by emphasizing the data-driven, multi-run nature and explicitly notes that no report is saved, which distinguishes it from report-related tools. This is a precise and non-tautological statement of purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for data-driven scenarios ('Data-driven') and clarifies the behavior, but it does not explicitly state when to choose this tool over alternatives like test_run or test_validate. It lacks direct exclusions or comparative guidance, leaving the selection to inference from the name and description. There is no mention of prerequisites (e.g., a saved test must exist) or when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

test_updateA

Replaces the full content of an existing test (same validation as test_create).

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesName of the test to replace
contentYesTest content: JSON string or object

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It states 'Replaces' which implies destructive mutation, and the validation reference adds some constraint info. But it does not disclose error behavior for nonexistent names, return values, or whether old content is irreversibly lost beyond the obvious implication. This is adequate but not rich.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence with the action verb front-loaded. It conveys the core purpose and a key constraint (validation parity) with zero wasted words. The parenthetical reference is efficient and informative.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple two-parameter mutation tool with full schema coverage and no output schema, the description combined with the schema is mostly sufficient for calling the tool. However, the absence of annotations leaves gaps around error handling and side effects; the description mentions replacement but not what happens if the test does not exist. The validation reference partially mitigates this, but overall it is only minimally complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with both parameters already described in the schema, so the baseline is 3. The description adds marginal semantic value by emphasizing 'full content' and referencing validation rules, but it does not significantly deepen understanding of the name or content parameters beyond what the schema already provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Replaces') and resource ('full content of an existing test') that clearly distinguishes this from test_create, test_rename, and test_duplicate. The phrase 'full content' unambiguously signals overwrite semantics rather than partial updates.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage context: use when you need to replace an existing test's entire content. The reference to 'same validation as test_create' provides implicit consistency guidance. However, it does not explicitly list alternatives or state when not to use the tool, though the action itself makes the main use case clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

test_validateA

Validates test content (JSON string or object) without saving it. Returns {ok: true, doc: summary} or {ok: false, error}. Useful before test_create/test_update.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoSuggested name (optional)
contentYesTest content: JSON string or object

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden. It discloses that the tool does not save content and specifies the exact return structure ({ok: true, doc: summary} or {ok: false, error}), which are critical behavioral traits. It does not mention any side effects beyond no-save, but that is the primary concern. The description could add more context about validation rules, but it covers the essential behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with no filler. The main purpose and no-save behavior are front-loaded, followed by the return format and usage guidance. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 2 parameters, no output schema, and no annotations, the description is quite complete. It covers the purpose, the no-save behavior, the return format, and usage guidance. It does not explain what 'summary' contains or any validation specifics, but these are not critical for an agent to invoke it correctly. The description is sufficient for safe and correct use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema description coverage is 100%, so the schema already documents both parameters (name and content). The description repeats that content is a JSON string or object, which adds little beyond the schema. It does not provide additional context about how the 'name' parameter is used in validation, so the description adds marginal value here. Baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's action ('Validates test content'), the resource (test content), and a key distinguishing trait ('without saving it'). It also specifies the return format, which helps an agent understand what to expect. This is specific and not a tautology.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly says 'Useful before test_create/test_update', which gives direct guidance on when to use this tool relative to its siblings. This is an explicit usage cue that eliminates ambiguity.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 35 tool updatesv0.1.0
    • First observedbaseline_get
    • First observedbaseline_list
    • First observedbrowser_click_at
    • First observedbrowser_close
    • First observedbrowser_condition
    • First observedbrowser_eval
    • First observedbrowser_open
    • First observedbrowser_preview
    • First observedbrowser_run_step
    • First observedbrowser_scroll
    • First observedbrowser_status
    • First observeddb_query
    • First observedping
    • First observedreport_delete
    • First observedreport_get
    • First observedreport_list
    • First observedschema
    • First observedsession_delete
    • First observedsession_list
    • First observedsession_save
    • First observedtab_close
    • First observedtab_list
    • First observedtab_open
    • First observedtab_switch
    • First observedtest_create
    • First observedtest_delete
    • First observedtest_duplicate
    • First observedtest_export_playwright
    • First observedtest_get
    • First observedtest_list
    • First observedtest_rename
    • First observedtest_run
    • First observedtest_run_dataset
    • First observedtest_update
    • First observedtest_validate

TDQS

B3.4/5.0

Scored across 35 tools

Disambiguation4/5

The tool set is organized by clear resource prefixes (test_, report_, browser_, tab_, session_), and each operation targets a distinct resource or action. A few browser-related tools such as browser_run_step, browser_condition, and browser_click_at have overlapping interaction/inspection responsibilities, but the descriptions clarify their boundaries.

Naming Consistency4/5

The vast majority of tools follow a consistent snake_case verb_noun pattern, e.g. test_create, browser_open, and session_delete. The exceptions are ping and schema, which are single-word names that do not follow the pattern but are still clear and unlikely to confuse.

Tool Count2/5

35 tools is well over the 25+ threshold and creates a heavy selection surface for an agent. The tools are logically grouped, but the overall count feels excessive for a single MCP server even though each tool has a legitimate role in the testing workflow.

Completeness4/5

The test lifecycle is well covered: create, read, update, delete, rename, duplicate, validate, run, and export are all present. Minor gaps exist in baseline management, which only offers list and get with no create/delete, and reports lack comparison or export operations, but these are workable gaps.

Maintenance

ActivityNo data
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers

  • A
    license
    B
    quality
    D
    maintenance
    A Model Context Protocol server that provides browser automation capabilities using Playwright, enabling LLMs to interact with web pages, take screenshots, generate test code, scrape web content, and execute JavaScript in a real browser environment.
    32
    15,344 npm
    MIT
  • A
    license
    Not graded
    quality
    D
    maintenance
    Enables AI assistants to control web browsers through Playwright automation, providing 50+ tools for navigation, interaction, testing, accessibility audits, and visual testing across Chromium, Firefox, and WebKit.
    6 npm
    MIT
  • A
    license
    Not graded
    quality
    F
    maintenance
    Enables Model Context Protocol tools for automating web browsers and Electron desktop applications using Playwright's accessibility tree snapshots. It supports window management and JavaScript execution in the main process, providing deterministic automation through structured data.
    241 npm
    17
    Apache 2.0
  • F
    license
    A
    quality
    D
    maintenance
    Enables AI-powered browser automation controlled through natural language, integrating Playwright with the Model Context Protocol to perform web interactions like navigation, form filling, and screenshots.
    10
    -