Vibe Testing
This server is an MCP testing tool that lets AI agents scan, explore, and test a live web app against its actual codebase.
Scan a codebase to discover routes, forms, components, API endpoints, and coverage gaps
Retrieve source context for features so tests use real selectors and field names
Log in with credentials and keep an authenticated browser session
Scan and explore pages to find interactive elements, click buttons, fill inputs, and observe API calls/errors
Execute structured test scenarios (navigate, fill, click, select, wait, assert, upload) with step-by-step logs and screenshots
Take screenshots of any URL for quick visual verification
Get coverage maps, gap analysis, and prioritized ready-to-run test suggestions
Run full end-to-end test suites in one call, or iteratively converge until pass-rate and gap thresholds are met
Generate self-contained HTML reports with embedded screenshots
Detect regressions and fixes by diffing results against previous runs
Close browsers and reset session state when done
Integrates with Windsurf (powered by Codeium) to provide browser testing capabilities: codebase scanning, scenario execution, and automated report generation.
Enables GitHub Copilot (VS Code) to perform browser-based testing of web applications, including login, element exploration, and full test run.
vibe-testing
Code-aware browser testing for AI coding agents.
Product site · npm · MCP Registry · Launch status
vibe-testing reads your codebase so tests use your real routes and field names, runs them in a real Playwright browser, remembers what broke, and tells you what your last change fixed or regressed. It works as an MCP server that gives your editor (Claude Code, Cursor, Windsurf, VS Code Copilot, Roo Code) 14 testing tools, or as a standalone CLI.
The repository is also an Agent Plugin with a release-qa skill. Compatible agents get both the testing tools and a senior-QA workflow for coverage, evidence, regression checks, and a clear release verdict.
cd /path/to/your/project
npx vibe-testing@latest initThen open your editor and say:
"Scan this codebase and test it against http://localhost:3000"
What code-aware testing adds
Playwright MCP gives your agent hands. vibe-testing gives it a testing workflow: code-derived scenarios, memory across runs, and a report. Two things a stateless browser tool cannot do:
1. Run it twice and it tells you what you broke. Every run writes .vibe/run-snapshot.json and diffs it against the previous run. Every scan writes .vibe/route-manifest.json and diffs your routes. The second run prints regressions and fixes instead of a wall of results:
Changes since last run
Fixed: /login
Regression: /checkout
New: /admin/usersThe same diff reaches your editor as snapshot_diff on run_full_test and run_converge, and as route_changes on scan_codebase, so the agent can flag "checkout broke after that commit" without anyone scrolling a report. Flaky routes, working selectors, and measured timeouts are also remembered between runs.
2. Zero LLM calls inside the tool. Pass/fail verification is heuristic: URL changes, toast detection, API errors. Your editor's model decides what to test; vibe-testing does the browsing and checking. No API key, no per-run cost beyond the editor subscription you already pay for.
Related MCP server: Vigilis
Contents
How it works
npx vibe-testing@latest init
|
Registers 14 MCP tools in your editor
|
You ask: "Test the checkout flow"
|
AI calls: scan_codebase -> get_context("checkout") -> login -> explore_page -> execute_scenario -> generate_report
|
HTML report opens in browser with screenshots of every stepNo test cases to write. The AI reads your source code to understand real field names and routes, opens a browser, tests everything, and shows you what's broken.
init also:
Detects which AI editors you have installed
Registers vibe-test in global editor configs (
~/.claude/settings.json,~/.cursor/mcp.json, and so on) so the tools are available in every project, every sessionCreates project-level MCP configs and AI instruction files
Auto-detects your app's URL (reads
.env,vite.config, framework defaults)Creates
VIBE.md(edit with your test credentials) andvibe.config.jsonInstalls the matching Playwright Chromium build
MCP setup
Agent Plugin
Install this repository as a plugin when your agent supports the Agent Plugins standard. It includes the portable release-qa skill and starts the published npm MCP server with no API key.
Cursor: submit or install
https://github.com/AishwaryShrivastav/vibe-testingas an Agent Plugin.Claude Code:
claude plugin marketplace add AishwaryShrivastav/vibe-testing claude plugin install vibe-testing@vibe-testingOther compatible agents: load the repository root containing
plugin.json,skills/, andmcp.json.
The direct MCP and CLI setup below remains available for editors without plugin support.
MCPB bundle
Claude Desktop and other MCPB hosts can install one local bundle. Build and validate it from this repository:
npm ci
npm run build:mcpbThe artifact is written to artifacts/vibe-testing-<version>.mcpb. See the MCPB distribution guide for the bundle contents, validation steps, Smithery handoff, and the one-time Playwright Chromium prerequisite.
Option 1: automatic (recommended)
npx vibe-testing@latest initDetects and configures all installed editors. Done.
Option 2: manual per editor
Claude Code
Add to ~/.claude/settings.json (global, works in every project):
{
"mcpServers": {
"vibe-test": {
"command": "npx",
"args": ["-y", "vibe-testing@latest", "--mcp"]
}
}
}Or add to .mcp.json in your project root (project-level only):
{
"mcpServers": {
"vibe-test": {
"command": "npx",
"args": ["-y", "vibe-testing@latest", "--mcp"]
}
}
}Cursor
Add to ~/.cursor/mcp.json (global) or .cursor/mcp.json (project):
{
"mcpServers": {
"vibe-test": {
"command": "npx",
"args": ["-y", "vibe-testing@latest", "--mcp"]
}
}
}Windsurf
Add to ~/.codeium/windsurf/mcp_config.json:
{
"mcpServers": {
"vibe-test": {
"command": "npx",
"args": ["-y", "vibe-testing@latest", "--mcp"]
}
}
}VS Code (GitHub Copilot)
Add to .vscode/mcp.json in your project:
{
"servers": {
"vibe-test": {
"command": "npx",
"args": ["-y", "vibe-testing@latest", "--mcp"]
}
}
}Roo Code / Cline
Add to .roo/mcp.json:
{
"mcpServers": {
"vibe-test": {
"command": "npx",
"args": ["-y", "vibe-testing@latest", "--mcp"]
}
}
}From local build (development)
{
"mcpServers": {
"vibe-test": {
"command": "node",
"args": ["/path/to/vibe-testing/dist/mcp-server.js"]
}
}
}MCP tools reference
14 tools available to your AI editor after setup:
Tool | When to call | Returns |
| Start here on a new project. Detects framework, server, and authentication | Structured setup result and idempotent project files |
| Always first. Reads source code, finds routes/forms/tests/gaps | Routes, forms, coverage map, generated scenarios, |
| Before writing test steps. Returns source files for a feature | Actual source code with real field names and selectors |
| When app requires authentication | Post-login screenshot, token state, API calls observed |
| To see all interactive elements on a page | Element list with selectors plus page screenshot |
| Broad "does everything work?" testing | Interaction results, API calls, errors, screenshot |
| Run specific test steps | Step-by-step logs plus screenshots |
| View coverage map and untested routes | Coverage entries, gaps, available scenarios |
| Find coverage gaps after exploration | Prioritized, ready-to-run scenarios with steps |
| Quick visual verification | Screenshot of any URL |
| Build HTML report (auto-opens) | Report path plus summary |
| One-shot: scan, execute, explore, report | Full results plus |
| Iterative testing until thresholds | Summary across all rounds plus |
| Close browsers, free resources | - |
Tool inputs
scan_codebase
{
"codebase_path": "/path/to/project",
"url": "http://localhost:3000",
"mode": "deep"
}get_context
{ "feature": "login" }
{ "feature": "/checkout" }
{ "feature": "user profile form" }login
{
"email": "test@example.com",
"password": "TestPass123!",
"login_url": "/login"
}scan_page_elements / explore_page
{
"route": "/dashboard",
"authenticated": true
}execute_scenario
{
"scenario": {
"id": "create-item",
"name": "Create a new item",
"route": "/items",
"steps": [
{ "action": "navigate", "url": "/items", "description": "Open items page" },
{ "action": "click", "selector": "text=Add Item", "description": "Open create form" },
{ "action": "fill", "selector": "[name='title']", "value": "Test Item", "description": "Fill title" },
{ "action": "fill", "selector": "[name='description']", "value": "Test description", "description": "Fill description" },
{ "action": "click", "selector": "button[type='submit']", "description": "Submit form" }
],
"expected_outcome": "New item appears in the list",
"requires_auth": true
}
}Step actions: navigate, fill, click, select, wait, assert, upload
CLI and MCP use the same scenario runner. An assert step evaluates every
supplied condition:
selector: the target must be visible. CSS,text=,label=, andplaceholder=are supported; the named locators match exactly.value: visible text must contain this string, on the selected element or on the page body when no selector is supplied.url: the current URL must exactly match this absolute URL or path resolved against the configured base URL, including query and fragment.
For example:
{ "action": "assert", "selector": "#confirmation", "value": "Saved", "url": "/settings", "timeout": 5000, "description": "Check saved settings" }These checks wait up to the step timeout (default 15000 ms per check). A failed
assertion stops the scenario, records a failed step and reason, and returns
status: "fail"; MCP also sets isError: true. Explicit conditions are used as
the verdict after the existing authentication and form API checks. The
description and expected_outcome prose are not executable assertions.
Without selector, value, or url, assert only checks page health: at least
10 characters of visible body text and no nonempty visible error indicators.
This compatibility check does not prove a described business outcome.
upload requires a file-input selector and a nonempty file path in value.
Relative paths resolve against the codebase root. Hidden file inputs are
supported. A missing file, directory, missing target, or target that is not a
file input returns status: "error". The action selects one file using
Playwright; add an assertion of the resulting UI to verify application-side
processing. It does not automate native file-picker dialogs.
Scenarios must contain at least one step. Unknown actions are errors.
take_screenshot
{ "url": "/settings", "authenticated": true, "full_page": false }run_full_test
{ "url": "http://localhost:3000", "codebase_path": "/path/to/project", "mode": "deep" }run_converge
{
"url": "http://localhost:3000",
"max_followup_rounds": 4,
"target_pass_rate": 0.92,
"max_high_severity_gaps": 2
}Recommended workflow
Full test session
Tell your AI editor:
Scan this codebase and test it against http://localhost:3000.
Log in with test@example.com / pass123. Explore the dashboard and
settings pages, run the suggested tests, and generate a report.The AI will:
scan_codebase, to understand routes, forms, existing testsget_context("login"), to read the actual login form source codelogin, to authenticate in a real browserexplore_page("/dashboard"), clicking everything and observing what breaksexplore_page("/settings"), samesuggest_tests, to find coverage gapsexecute_scenariox N, running targeted test flowsgenerate_report, HTML report opens automaticallycleanup, closing browsers
Test a specific feature
Test the checkout flow using vibe-test. Get context for checkout,
then run the full purchase flow with card number 4242424242424242.The AI will:
scan_codebase(if not already done)get_context("checkout"), readingCheckoutForm.tsx,api/orders/route.tsand so onloginexecute_scenario, filling the real form fields from source codegenerate_report
Verify a bug fix
I fixed the login redirect bug. Use vibe-test to confirm it's working.The AI will:
login, testing the login flowtake_screenshot, visual confirmation of the post-login stateReport back what it sees
Find what's broken
Explore every page and tell me what's broken.The AI will run explore_page on every route, collecting API errors, broken elements, and failed interactions, then suggest_tests with the broken items marked as high priority.
init command
npx vibe-testing@latest init [options]What it creates:
File | Where | Purpose |
| Project root | Claude Code MCP config (project-level) |
| Global | Claude Code MCP config (all projects) |
| Project root | Cursor MCP config |
| Global | Cursor MCP config (all projects) |
| Project | Cursor rules, |
| Project | Windsurf instructions |
| Global | Windsurf MCP config (all projects) |
| Project | VS Code Copilot MCP config |
| Project | GitHub Copilot instructions |
| Project | Roo Code MCP config |
| Project | Claude Code session instructions |
| Project | Universal agent instructions (Codex, Devin, Zed) |
| Project | Test guidance, edit with your credentials |
| Project | Config, URL auto-detected from your project |
Options:
npx vibe-testing@latest init # auto-detect editors, register globally + project
npx vibe-testing@latest init --no-global # project-level only, skip global registration
npx vibe-testing@latest init --skip-browser-install
npx vibe-testing@latest init --editor cursor # only configure Cursor
npx vibe-testing@latest init --editor claude-code windsurfAfter init, edit VIBE.md with your login URL and test credentials.
CLI commands
# Set up in current project
npx vibe-testing@latest init
# Install or repair the matching browser build
npx vibe-testing@latest install-browser
# Run tests against a URL
npx vibe-testing@latest run http://localhost:3000
npx vibe-testing@latest run https://staging.myapp.com --mode deep
npx vibe-testing@latest run http://localhost:3000 --codebase /path/to/project --scope /login /dashboard
# Iterative testing until coverage thresholds
npx vibe-testing@latest converge http://localhost:3000
npx vibe-testing@latest converge http://localhost:3000 --max-rounds 6 --target-pass-rate 0.95
# Open last report in browser
npx vibe-testing@latest report
# Reset memory and screenshots for a clean run
npx vibe-testing@latest resetrun options
Option | Default | Description |
|
|
|
| - | Run browser headless (default: visible) |
| cwd | Path to project root |
| all | Test only specific routes |
|
| Config file path |
run and converge fail with a nonzero exit when no scenarios are discovered
or an execution batch is empty. MCP run_full_test and run_converge return
isError: true for these cases. Empty runs do not write a successful run
snapshot or replace the prior report.
converge options
Option | Default | Description |
|
| Max follow-up rounds after baseline |
|
| Stop when pass rate reaches this (0-1) |
|
| Stop when critical plus important gaps fall to this |
VIBE.md, project guidance
Create VIBE.md in your project root. vibe-testing reads it automatically on every run.
## Login URL
/login
## Test Credentials
- Email: test@example.com
- Password: TestPass123!
## Never Automate
- delete account
- cancel subscription
- [data-testid="danger-zone"]
- .billing-section
## Known Flaky
- /notifications (WebSocket dependent, skip or expect retry)
- /live-feed
## Notes
- Admin panel at /admin, use admin@example.com / adminpass
- Dashboard data loads async, wait for [data-loaded="true"]
- Profile page: click "Edit Profile" before form fields appearSee VIBE.example.md for the full template.
Configuration
vibe.config.json
Created automatically by init with auto-detected URL. Edit as needed:
{
"url": "http://localhost:3000",
"mode": "deep",
"auth": {
"strategy": "credentials",
"login_url": "/login",
"credentials": {
"email": "test@example.com",
"password": "TestPass123!"
}
},
"never_interact": [
"delete account",
"cancel subscription",
"[data-testid='danger-zone']"
],
"scope": {
"include": ["/**"],
"exclude": ["/admin/**", "/api/**"],
"max_routes": 30,
"seed_routes": ["/live/dev-mode-a-now"]
},
"browser": {
"headed": true,
"slowMo": 40,
"timeout": 30000
}
}Key | Description |
| App URL, localhost or staging. Auto-detected by |
|
|
|
|
| Explicit login route for non-standard paths keyword matching would miss |
| Login credentials, used for login and for generated scenarios, persisted across runs |
| Text patterns or CSS selectors to skip during exploration |
| Route patterns to include; default |
| Route patterns to exclude from testing |
| Cap how many routes are tested per run |
| Concrete URLs for dynamic-segment routes the parser can't enumerate (e.g. |
|
|
| Milliseconds between actions (useful for debugging) |
|
|
Supported frameworks
Framework | Routes | API endpoints | Forms |
Next.js App Router | yes | yes | yes |
Next.js Pages Router | yes | yes | yes |
Next.js (src/ variant) | yes | yes | yes |
React SPA (react-router) | yes | - | yes |
Vue + Vite (vue-router) | yes | - | yes |
Nuxt | yes | yes | yes |
SvelteKit | yes | yes | yes |
Express / Fastify | - | yes | yes |
Monorepos (Turborepo, pnpm, Lerna) | yes | yes | yes |
Existing test files are also read to build a coverage map: Jest, Vitest, Playwright, and Cypress suites are all parsed.
Memory and regression detection
vibe-testing learns across runs and stores state in .vibe/:
Working selectors: remembers
[name='email']worked on/login, uses it next runRoute timings: adjusts timeouts based on measured load times
Auth credentials: saved after first login, reused automatically
Flaky routes: tracks high fail-rate routes, marks them for retry
Skip routes: routes that consistently error (need URL params) are auto-skipped
Route manifest (
.vibe/route-manifest.json): every scan diffs against the previous one; new and removed routes surface asroute_changesonscan_codebaseresults so the AI can cover them immediatelyRun snapshot (
.vibe/run-snapshot.json): every run captures per-route pass/fail and diffs against the prior run;snapshot_diffflagsnewly_passing(fixes),newly_failing(regressions),still_failing, plus added and removed routes
{
"snapshot_diff": {
"newly_passing": ["/login"],
"newly_failing": ["/checkout"],
"still_failing": [],
"new_routes": ["/admin/users"],
"removed_routes": []
}
}run_converge returns the same shape, so iterative runs in your editor highlight what you just broke.
Reset with npx vibe-testing@latest reset to start fresh.
What a session looks like
When you ask your editor to "test the login flow", here is what it does:
User: "Test the login flow"
AI calls:
scan_codebase({ codebase_path: ".", url: "http://localhost:3000" })
-> Finds /login route, LoginForm component, POST /api/auth/login endpoint
-> Returns 8 generated test scenarios
get_context({ feature: "login" })
-> Returns src/app/login/page.tsx (has name="email", name="password" fields)
-> Returns src/app/api/auth/login/route.ts (POST handler, returns { token })
-> AI now knows the real selectors: [name='email'], [name='password']
login({ email: "test@example.com", password: "pass123" })
-> Opens Chromium, fills the form, clicks submit
-> Returns: { success: true, final_url: "/dashboard", tokens_found: 2 }
-> Returns screenshot of post-login dashboard
execute_scenario({ scenario: { name: "Login with invalid password", ... } })
-> Returns screenshot showing error state
generate_report()
-> Writes .vibe/report.html, opens in browser
AI reports: "Login works. Invalid password shows an error. All 3 login scenarios passed."FAQ
Does vibe-testing use an LLM internally? No. It uses heuristic verification (URL changes, toast detection, API errors). Your editor's model is the brain: it sees screenshots and decides what to test next. Runs have no API cost.
What's the difference between explore_page and execute_scenario?
explore_page is broad: it clicks every button and input it finds and reports the results. execute_scenario is precise: you give it specific steps and it follows them exactly. Use explore_page to find what's on a page, then execute_scenario to test specific flows.
What's get_context for?
It returns the actual source code for a feature, so the AI knows [name='email'] instead of guessing #email-input. Always call it before writing test steps for a specific feature.
Does it handle SPAs with client-side routing? Yes. Playwright navigates the real browser, so client-side routing (React Router, Vue Router, and the rest) works naturally.
Does it handle login / authentication?
Yes. The login tool fills credentials in a real browser, captures auth tokens from localStorage/cookies, and keeps that session alive for authenticated tests. Credentials are persisted in .vibe/memory/ and reused automatically.
Will it click "Delete Account" or other destructive buttons?
No. Set never_interact in vibe.config.json or VIBE.md to blocklist dangerous actions. Any button whose text or selector matches is skipped during exploration.
Can I use it without an AI editor?
Yes. vibe-test run https://your-app.com runs standalone. It scans, generates scenarios, executes them, and produces an HTML report without needing an editor.
How do I test a staging environment?
Set url in vibe.config.json to your staging URL, or pass it as a CLI argument: npx vibe-testing@latest run https://staging.myapp.com.
Does it work with monorepos?
Yes. init detects Turborepo/pnpm/yarn workspaces and finds the frontend app automatically.
Requirements
Node.js >= 20 (the test suite uses vitest 4.x which requires Node 20+)
Playwright Chromium.
initinstalls the matching build. To install or repair it directly:npx vibe-testing@latest install-browser
run and converge check for the browser before scanning your project and print the same recovery command when it is missing.
Docker
A Node 20 + Chromium image is included for environments that prefer container-based MCP servers:
docker build -t vibe-test .
# wire into your editor's MCP config:
# { "command": "docker", "args": ["run", "--rm", "-i", "vibe-test"] }Contributing
git clone https://github.com/AishwaryShrivastav/vibe-testing.git
cd vibe-testing
npm install
npx playwright install chromium
npm run build # tsc -> dist/
npm run dev # run CLI without building
npm run mcp # run MCP server without building
npm test # unit and isolated integration tests
VIBE_REAL_BROWSER=1 npm test # also run real Chromium CLI/MCP regressionsSee CHANGELOG.md for version history. Bug reports and feature requests: GitHub issues.
License
MIT, Aishwary Shrivastav
Links
Issues: https://github.com/AishwaryShrivastav/vibe-testing/issues
Official MCP registry: listed as
io.github.AishwaryShrivastav/vibe-testingat https://registry.modelcontextprotocol.io
Available Tools
14 toolscleanupA
Close all open browsers and reset the session state. Call when done testing.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the burden of behavioral disclosure. It states the tool closes browsers and resets session state, which implies destructive/state-changing behavior. However, it doesn't disclose potential side effects like losing unsaved data or whether it affects other sessions. The description is adequate but could be more transparent about consequences.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two short sentences with no wasted words. The primary action is stated first, followed by usage guidance. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter tool with no output schema, the description is mostly complete. It states what it does and when to call it. However, it doesn't mention what happens after cleanup (e.g., can the session be resumed?) or whether this is reversible. Given the tool's simplicity, this is a minor gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so there are no parameter semantics to document. The description correctly implies no inputs are needed. Baseline 4 is appropriate for a zero-parameter tool.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: closing all open browsers and resetting session state. It uses a specific verb ('close', 'reset') and identifies the resource (browsers, session state). It doesn't explicitly differentiate from siblings, but the action is distinct enough among the listed tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when to use the tool: 'Call when done testing.' This implies it should be used at the end of a testing session, which is useful guidance. It doesn't explicitly mention alternatives or when not to use it, but the context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
configureB
Idempotently configure a project and detect its framework, running server, and authentication method. Existing VIBE.md and vibe.config.json files are preserved.
| Name | Required | Description | Default |
|---|---|---|---|
| url | No | Optional application URL. If omitted, project configuration and bounded local server detection are used. | |
| codebase_path | Yes | Absolute path to the project root directory. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses idempotency and preservation of existing VIBE.md and vibe.config.json files, which are useful behavioral notes. However, it does not mention side effects, error conditions, authentication requirements, or what happens when files are absent. The disclosure is partial but not contradictory.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that front-loads the core purpose and adds two key behavioral guarantees (idempotency and file preservation). There is zero waste; every clause earns its place. Structure is optimal for a tool with such a narrow scope.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool lacks an output schema, so the description should explain what the tool returns or how the agent should interpret the result. It does not. It also omits details about failure modes, the impact of the optional url parameter, and any prerequisites like existing project structure. For a tool that performs detection and configuration, this is incomplete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description adds no extra meaning to the parameters beyond what the schema already states. It neither clarifies the url parameter's role nor the codebase_path format. The schema is adequate, so a 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: configuring a project and detecting its framework, running server, and authentication method. It names the resource (project) and the actions (configure, detect). While it does not explicitly distinguish from siblings, the sibling list contains no overlapping tools, so clarity is high.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives or when not to use it. There is no mention of exclusions, prerequisites, or conditions that would lead an agent to choose this tool over another. The purpose is implied but not explicitly framed against other options.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
execute_scenarioA
Execute a single test scenario (a sequence of navigate/fill/click/assert steps) and return detailed results with step-by-step logs, screenshots after each state-changing step, API errors observed, and the final page state. The editor LLM can construct scenarios based on scan_codebase output or create custom ones.
| Name | Required | Description | Default |
|---|---|---|---|
| scenario | Yes | A test scenario object with id, name, route, steps[], and expected_outcome. | |
| authenticated | No | Whether to use authenticated context. Default: inferred from scenario.requires_auth. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the behavioral disclosure burden. It does well by detailing what executing a scenario produces: step-by-step logs, screenshots after each state-changing step, API errors observed, and final page state. It does not mention potential side effects on the application under test, but the disclosed execution details are substantive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences: the first front-loads the core action and outputs; the second adds provenance guidance. No filler, repetition, or unnecessary detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The schema richly covers the nested scenario and step fields, and the description covers return behavior since there is no output schema. It is generally complete for calling the tool, though it would be stronger with explicit guidance on when to choose this versus run_full_test and a note about side effects.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description itself adds little parameter-level semantics beyond noting that scenarios may be constructed from scan_codebase output, but the schema already documents scenario, authenticated, and all nested step fields thoroughly.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly names the verb ('execute'), the resource ('a single test scenario'), and enumerates the step types (navigate/fill/click/assert) and result contents (logs, screenshots, API errors, final page state). The word 'single' helps distinguish it from sibling run_full_test, which suggests a broader full-suite execution.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives useful context that scenarios can be built from scan_codebase output or created custom, which implies a valid usage pathway. However, it does not explicitly state when to prefer this tool over alternatives like run_full_test or run_converge, nor does it provide exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
explore_pageA
Perform a full interactive exploration of a page: discover all elements, click buttons, fill inputs, test tabs, observe API calls, and report what happened. Returns detailed interaction outcomes, API observations, and screenshots. This is the "senior tester" mode — it tries every element and reports what works and what breaks.
| Name | Required | Description | Default |
|---|---|---|---|
| route | Yes | The route path to explore (e.g. /dashboard). | |
| authenticated | No | Whether to use the authenticated context. Default false. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure. It does disclose the interactive actions, API observation, and output format. However, it does not warn that clicking buttons and filling inputs may mutate data, trigger side effects, or navigate away from the current state—an important gap for an autonomous exploration tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is reasonably concise, front-loaded with the core action, and provides useful detail about outputs. The second sentence adds the 'senior tester' framing but partially repeats the earlier 'tries every element and reports' idea, creating slight redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description explains return values (interaction outcomes, API observations, screenshots) because no output schema exists. However, for a tool that performs full interactive exploration and can alter state, it is incomplete without noting prerequisites (e.g., app must be running), potential side effects, or the destructive nature of the actions it performs.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents both route and authenticated. The description adds no parameter-level detail beyond that, such as when authenticated should be true or how the route interacts with the app's base URL. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: 'Perform a full interactive exploration of a page' and enumerates concrete actions (click buttons, fill inputs, test tabs, observe API calls). It is clearly distinguished from siblings like scan_page_elements or take_screenshot by presenting itself as the 'senior tester' mode that tries every element and reports what works and breaks.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use it: when you need exhaustive interactive exploration rather than a static scan. However, it never explicitly contrasts it with siblings such as scan_page_elements or execute_scenario, nor does it state when not to use it (e.g., when a non-destructive read-only scan is safer).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
generate_reportA
Generate a self-contained HTML test report with embedded screenshots from all collected results, explorations, and coverage data. Returns the report file path and a text summary. The report includes: pass/fail results per scenario, step-by-step screenshots, element exploration findings, API error monitoring, and coverage gap suggestions. Call this after executing scenarios and explorations. The report auto-opens in the browser.
| Name | Required | Description | Default |
|---|---|---|---|
| title | No | Optional custom title for the report. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses that the report auto-opens in the browser (a side effect) and that it is self-contained, which is valuable behavioral context beyond what annotations would provide. It doesn't mention if it requires prior data or if it's safe, but it does add useful behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a well-structured paragraph that starts with the core purpose, then lists contents, then gives usage guidance. It is not overly verbose, but the list of contents could be trimmed for brevity. Still, each sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is relatively simple with a single optional parameter and no output schema, so the description covers the essential information: what it produces, when to call it, and a key behavior (auto-open). It does not explain the return format in detail, but that's not required given no output schema. It's adequate for an agent to invoke it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% as the only parameter 'title' is described as 'Optional custom title'. The description doesn't add any additional meaning beyond this, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool generates an HTML test report with embedded screenshots, and lists the specific contents (pass/fail results, screenshots, exploration findings, etc.), which distinguishes it from siblings like take_screenshot or get_coverage.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says to call this after executing scenarios and explorations, which provides clear when-to-use guidance. However, it doesn't mention when not to use it or name alternatives, so it loses a point for lack of exclusions or alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_contextA
Retrieve the most relevant source files for a given feature or route. Returns actual source code (budget-capped) so you understand real field names, API endpoints, and component structure before writing test steps. Call this after scan_codebase when you want to write precise test scenarios for a specific feature — it eliminates guesswork about selectors and form fields.
| Name | Required | Description | Default |
|---|---|---|---|
| feature | Yes | Feature name or route path to retrieve context for (e.g. "login", "checkout", "/dashboard"). | |
| max_files | No | Max source files to return (default 5, max 8). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the full behavioral burden. It mentions that output is budget-capped, which is a limitation, but it doesn't disclose whether the operation is read-only, if it has side effects, or how errors are handled. Since it retrieves data, the read-only nature is implied but not stated. This is a significant gap for a tool with zero annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with no filler. It front-loads the core purpose and then adds usage guidance. It is concise and structured effectively, though slightly longer than strictly necessary, it earns its length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description should explain what the tool returns. It says 'Returns actual source code' but doesn't specify the format (e.g., list of files with content, metadata) or how budget-capping affects the output. Error behavior and edge cases (e.g., feature not found) are not covered. For a moderately simple tool, this is acceptable but leaves some ambiguity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, and both parameters are well-documented in the schema. The description adds no extra semantic detail about the parameters themselves; it only contextualizes the return value. With full schema coverage, a baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (retrieve) and the resource (source files for a given feature or route). It explicitly says it returns actual source code to understand field names, endpoints, and structure, which distinguishes it from siblings like scan_codebase or scan_page_elements by focusing on a specific feature.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly instructs to call this after scan_codebase and when writing precise test scenarios for a specific feature. It provides a clear usage condition and rationale, though it doesn't explicitly name alternative tools for different use cases. The context is sufficient for an agent to decide when to invoke it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_coverageA
Return the current test coverage map, identified gaps, and suggested tests. Prerequisite: scan_codebase must have been called first. Returns JSON with: coverage entries per route (tested/untested, test frameworks used), gap analysis with priority scores (high/medium/low), and concrete test suggestions for missing coverage. Use this to understand what has been tested and what still needs testing.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses the prerequisite, the JSON return structure, and the three main components (coverage entries, gap analysis, suggestions). It does not state what happens when the prerequisite is unmet, but for a read-only lookup this is reasonably complete.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences and front-loads the core function. The final sentence restates the purpose somewhat redundantly, but overall the length is appropriate for the complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and no parameters, the description compensates well by spelling out the return shape and prerequisite. The overlap with suggest_tests is not directly addressed, leaving minor ambiguity about when to prefer this over that sibling.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters)Skip baseline 4 applies since the description does not need to compensate for missing parameter documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Return'), a clear resource ('test coverage map'), and the additional outputs ('gaps' and 'suggested tests'). Despite the sibling suggest_tests, the description clearly frames this as a coverage-status tool rather than a test-generation tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly names the prerequisite (scan_codebase must have been called first) and gives a direct use case ('Use this to understand what has been tested and what still needs testing'). It does not name alternatives or exclusions, but the context is clear enough for an agent to select it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
loginC
Establish an authenticated browser session by executing a login scenario. Returns the post-login URL, token state, and a screenshot. Uses saved credentials from previous runs if available, or accepts provided credentials.
| Name | Required | Description | Default |
|---|---|---|---|
| No | Email/username to log in with. If omitted, uses saved credentials. | ||
| password | No | Password. If omitted, uses saved credentials. | |
| login_url | No | Login page URL path (e.g. /login). Defaults to /login. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must fully disclose behavioral side effects. It states it establishes an authenticated session and returns data, but omits critical side effects such as session persistence, cookie/token storage, or modification of saved credentials. Insufficient for a stateful mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with purpose and return values, followed by credential behavior. No unnecessary words or repetition; all content earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a stateful login tool with no annotations and no output schema, the description is incomplete: it omits failure modes, prerequisites, side effects, and does not differentiate from sibling tools. An agent cannot fully predict the tool's impact or limitations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers all parameters at 100%, so baseline 3 applies. The description adds minor value by explaining the saved-credential fallback behavior for email and password, but otherwise repeats schema information.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clear verb 'establish' and resource 'authenticated browser session' with mention of login scenario and return values. Does not explicitly distinguish itself from sibling execute_scenario, but purpose is apparent from the login-specific scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Offers no guidance on when to use this tool versus sibling tools like execute_scenario or explore_page. The credential fallback behavior is useful but does not help an agent choose between tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
run_convergeA
Iterative coverage: runs the full baseline suite, then automatically runs follow-up rounds targeting coverage gaps and failed scenarios until pass rate and gap thresholds are met (or max rounds reached). Use for "keep testing until coverage is good". Returns results across all rounds and opens the final HTML report.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Base URL of the running application (e.g. http://localhost:3000). | |
| mode | No | fast = quick heuristic scan. deep = full feature extraction with dialogs and CRUD detection. | deep |
| headed | No | Show the browser window during testing. Defaults to headless (false) when used via MCP. Set true to watch the browser. | |
| codebase_path | No | Absolute path to the project root directory. Defaults to the current working directory. | |
| target_pass_rate | No | Stop when last batch pass rate reaches this 0–1 (default 0.92). | |
| max_followup_rounds | No | Max extra rounds after baseline (default 4). | |
| max_high_severity_gaps | No | Stop when critical+important gaps <= this (default 2). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full disclosure burden. It discloses the iterative process (baseline then follow-up rounds), stopping conditions (pass rate and gap thresholds or max rounds), and output behavior (returns results across all rounds and opens the final HTML report). It doesn't mention potential side effects on the app or resource usage, but for a testing tool the core behavior is well covered.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with zero waste. The core function is front-loaded, followed by the use-case and output behavior. Every clause adds value, making it highly scannable for an agent.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description must explain the return value, and it does ('Returns results across all rounds and opens the final HTML report'). It also explains the iterative process and stopping criteria, covering all essential behavioral aspects. The schema handles parameter details, so the description is complete for an agent to invoke it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so all parameters already have clear descriptions. The tool description doesn't add any parameter-specific meaning beyond what the schema provides, which meets the baseline for high coverage. The mention of 'targeting coverage gaps' indirectly relates to target_pass_rate and max_high_severity_gaps, but it's not explicit enough to exceed baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('runs iterative coverage') and resource (full baseline suite plus follow-up rounds), and clearly distinguishes it from siblings by mentioning 'targeting coverage gaps and failed scenarios' until thresholds are met. The use-case phrase 'keep testing until coverage is good' further clarifies its unique role.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'Use for "keep testing until coverage is good"', which is a clear when-to-use signal. It doesn't explicitly contrast with alternatives like run_full_test or execute_scenario, but the iterative nature and explicit use-case give sufficient guidance for an agent to select it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
run_full_testA
Run a complete end-to-end test suite: scan codebase → generate scenarios → execute all → explore pages → generate report. This is the all-in-one command. For more granular control, use the individual tools (scan_codebase, login, explore_page, execute_scenario, generate_report).
| Name | Required | Description | Default |
|---|---|---|---|
| url | No | Base URL of the running application (e.g. http://localhost:3000 or https://staging.myapp.com). | |
| mode | No | fast = quick heuristic scan. deep = full feature extraction with dialogs and CRUD detection. | deep |
| headed | No | Show the browser window during testing. Defaults to headless (false) when used via MCP. Set true to watch the browser. | |
| codebase_path | No | Absolute path to the project root directory. Defaults to the current working directory. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must carry behavioral disclosure. It outlines the high-level steps but omits potential side effects (e.g., whether it modifies the codebase, creates artifacts, or requires a running application) and does not mention duration or resource usage. It adds moderate value by describing the pipeline but lacks critical operational details for a complex, state-changing operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the main purpose and step list, followed by the alternative. Zero wasted words. The structure is optimal for quick parsing by an agent.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex tool with 4 optional params and no output schema, the description adequately covers purpose and usage guidance. It does not explain what the generated report contains or where it is output, and it omits prerequisites like a running application. However, given the schema covers all parameters, the description is largely complete; the missing behavioral details keep it from a 5.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers 100% of parameters with descriptions, so the baseline is 3. The tool description adds no parameter-specific guidance beyond the schema, e.g., it doesn't explain the interaction between mode and the pipeline steps. It neither improves nor degrades the schema's clarity, so a baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool runs a complete end-to-end test suite, enumerating the pipeline steps (scan → generate scenarios → execute → explore → report). It explicitly labels itself as the 'all-in-one' command, distinguishing it from the granular sibling tools. The verb 'run' and resource 'complete end-to-end test suite' are specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It directly tells the agent when to use this tool (for full pipeline) and when not to (for granular control), naming the specific sibling tools to use instead. This is explicit exclusion guidance, leaving no ambiguity about selection criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
scan_codebaseA
Analyze a project's codebase to understand its structure, routes, forms, components, existing tests, and coverage gaps. Returns a ProductModel with routes, behaviours, coverage map, gaps, and generated test scenarios. Call this first before any testing.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The base URL of the running application (e.g. http://localhost:3000 or https://staging.myapp.com). If the app is running on a staging/dev URL, provide that instead of localhost. | |
| mode | No | fast = quick heuristic scan. deep = full extraction with dialogs/features (default). | deep |
| codebase_path | Yes | Absolute path to the project root directory. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the behavioral burden. It discloses the return type (ProductModel) and the contained data, and the verb 'Analyze' implies a read-only operation. However, it does not explicitly state side effects, whether it modifies anything, or any operational constraints like time or dependency on a running app.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two dense sentences with no filler. Purpose is front-loaded, the return value is summarized, and the critical sequencing instruction is stated at the end. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a first-step analysis tool with no output schema and no annotations, the description covers the central facts: what it analyzes, what it returns, and when to call it. It could be slightly richer about mode semantics or preconditions, but those are already largely captured in the schema and this is otherwise sufficient for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the input schema already documents all three parameters (url, mode, codebase_path). The description adds no additional parameter-level meaning beyond the overall workflow, so it sits at the baseline 3 rather than higher.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Analyze') and resource ('a project's codebase'), and enumerates what is examined: structure, routes, forms, components, existing tests, and coverage gaps. It distinguishes itself from siblings like scan_page_elements (which is page-specific) by making the codebase-wide scope explicit and by positioning itself as the initial step.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The instruction 'Call this first before any testing' provides explicit when-to-use guidance, establishing this tool as the entry point before scanning pages, executing scenarios, or generating reports. It does not explicitly list alternatives or state when not to use it, so it stops short of a full 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
scan_page_elementsA
Navigate to a specific page and discover all interactive elements (buttons, links, inputs, selectors, checkboxes, tabs). Returns a structured list of elements with their types, text, selectors, and disabled state. Also returns a screenshot of the page. Use this to understand what's on a page before deciding what to test.
| Name | Required | Description | Default |
|---|---|---|---|
| route | Yes | The route path to scan (e.g. /dashboard, /settings). Will be appended to the base URL. | |
| authenticated | No | Whether to use the authenticated browser context. Default false. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must carry the behavioral disclosure. It reveals that the tool navigates, scans elements, and returns a screenshot, but it does not state whether it is read-only, whether navigation persists, or any limitations such as dynamic content handling. This is adequate but not rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences with no unnecessary words. It front-loads the purpose, then describes the output, then gives the usage context. Every sentence contributes to agent understanding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 2-parameter tool with full schema coverage, the description covers purpose, output structure, and usage context. It lacks an output schema but describes the return values in sufficient detail. It does not mention how the authenticated parameter affects behavior, but that is a minor gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds no parameter-specific meaning beyond what the schema already provides for route and authenticated. No compensation is needed because the schema documents both parameters fully.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool navigates to a page and discovers interactive elements, and lists the returned data (types, text, selectors, disabled state, screenshot). This is specific, but it does not explicitly contrast with siblings like take_screenshot or explore_page, leaving some potential overlap ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly says to use this to understand a page before deciding what to test, which gives a clear usage context. It does not mention when not to use it or name alternative tools, but the context is sufficient for selecting this tool in many cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
suggest_testsA
Analyze codebase features, existing test coverage, and results from previous runs to suggest concrete test scenarios that should be written or executed. Returns prioritized, executable scenario objects with steps. Use this after scan_codebase to understand what testing is missing and get ready-to-run scenarios.
| Name | Required | Description | Default |
|---|---|---|---|
| route | No | Optional: focus suggestions on a specific route (e.g. /dashboard). If omitted, analyzes all routes. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full behavioral burden. It does disclose the output shape ('prioritized, executable scenario objects with steps') and signals this is an analysis/suggestion tool rather than an execution one, but it does not explicitly state side effects or non-mutating behavior. This is adequate but not rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, both necessary: the first defines purpose and output, the second provides usage context. It is front-loaded with the core function and avoids filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 1-parameter, no-output-schema suggestion tool, the description covers the key context: what inputs it considers, what it returns, and when to call it relative to scan_codebase. It is slightly light on explicit return-value detail, but the phrase 'prioritized, executable scenario objects with steps' gives enough shape for an agent to use the result.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already documents the sole optional 'route' parameter with 100% coverage, including an example and default behavior. The description adds no additional parameter-level meaning, which aligns with the baseline of 3 when the schema carries the load.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Analyze'), a clear resource ('codebase features, existing test coverage, and results from previous runs'), and a concrete deliverable ('concrete test scenarios... Returns prioritized, executable scenario objects with steps'). It distinguishes itself from siblings like scan_codebase and execute_scenario by framing itself as the suggestion step after analysis and before execution.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit sequencing guidance: 'Use this after scan_codebase to understand what testing is missing.' It makes the intended context clear, though it does not explicitly list exclusions or directly contrast with alternatives like get_coverage or execute_scenario.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
take_screenshotA
Navigate to a URL and take a screenshot. Returns the screenshot as a base64 data URI that the editor LLM can see and reason about. Use this for quick visual verification.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Full URL or route path to screenshot. | |
| full_page | No | Capture full page or just viewport. Default false. | |
| authenticated | No | Use authenticated context. Default false. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and does a good job by disclosing the key behavioral output: the screenshot is returned as a base64 data URI that the editor LLM can see. It does not cover failure modes or authentication quirks, but the observable behavior most relevant to an agent is stated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no wasted words. The action is front-loaded, followed by the output format and use case. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple, all three parameters are documented by the schema, and the description supplies the missing output explanation (base64 data URI). An agent has enough information to invoke it correctly and interpret the result.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so url, full_page, and authenticated are already fully documented. The description does not add parameter-specific detail or examples, so the baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb and resource: 'Navigate to a URL and take a screenshot.' It also names the output format (base64 data URI) and the intended use ('quick visual verification'), which clearly distinguishes it from analysis-oriented siblings like scan_page_elements and explore_page.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use the tool: 'Use this for quick visual verification.' It provides clear context but does not name sibling alternatives or state when not to use it, so it misses the top-tier criterion of explicit exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
2 tool updates
v0.4.6- Added
configure - Changed
run_full_test1 field changed- changed
Input schema / requiredPrevious value: -[ - "url" -]New value: +[]
13 tool updates
v0.1.0- First observed
cleanup - First observed
execute_scenario - First observed
explore_page - First observed
generate_report - First observed
get_context - First observed
get_coverage - First observed
login - First observed
run_converge - First observed
run_full_test - First observed
scan_codebase - First observed
scan_page_elements - First observed
suggest_tests - First observed
take_screenshot
TDQS
Scored across 14 tools
Most tools have clearly distinct purposes, but scan_codebase, get_coverage, and suggest_tests all return coverage gaps and test suggestions, creating real selection ambiguity. The descriptions clarify prerequisites somewhat, but an agent could still reasonably call the wrong one.
The majority of tools follow a clear imperative verb_noun snake_case pattern like scan_codebase, execute_scenario, and generate_report. However, login, cleanup, and configure are bare verbs, and run_converge uses a verb where a noun object would be more consistent with run_full_test.
Fourteen tools is within a reasonable range for a comprehensive testing workflow and most map to distinct phases. It feels slightly heavy because several tools overlap around coverage/suggestions and there is an all-in-one orchestrator, but no tool feels entirely superfluous.
The core web-testing lifecycle is well covered: configuration, login, codebase scanning, element discovery, scenario execution, coverage analysis, reporting, and cleanup. Minor gaps include no explicit way to retrieve past test results outside of generated reports or to abort a long-running convergence run.
Maintenance
Related MCP Connectors
Browser-based QA for AI-built software. Test pages with real browsers via agents.
Browser-backed QA with evidence and fix-ready reports for coding agents.
AI QA tester — real browsers scan sites for bugs, SEO, perf, and accessibility issues via chat.
Hosted browser for AI agents: screenshots, post-JS DOM, console, WCAG. No install, no API key.
Related MCP Servers
- AlicenseAqualityCmaintenanceMCP server + Chrome extension that gives AI coding agents control of your real browser with existing sessions, logins, and cookies. Works with Cursor, Claude, Windsurf.1873 npm51MIT
- AlicenseNot gradedqualityBmaintenanceAn agentic QA framework that authors, generates, triages, and self-heals Playwright tests for any web app, usable from Claude Code/Desktop as an MCP server or from CI as a CLI.5MIT
- FlicenseAqualityBmaintenanceAn MCP server that gives Claude direct control of a real browser via Playwright, enabling AI-driven web testing, autonomous test execution, and live failure analysis through natural language.84,622 npm3-
- AlicenseAqualityCmaintenanceAI-native browser testing, directly from your coding agent.3MIT