Skip to main content
Glama

blop — The MCP-Native Release Confidence Control Plane

Most AI testing tools can click through pages. Teams still do not know whether they should ship.

blop turns browser execution into release decisions by combining business-critical journey context, evidence-heavy QA runs, and risk governance in one MCP-native control plane.

You do not write test code. You ask in chat, then ship with auditable evidence, prioritized risk, and a clear go/no-go recommendation.

Compatible MCP clients: Cursor, Claude Code, and other clients that support MCP tool/resource workflows.

Quick Navigation


Related MCP server: AgentsGate

Product thesis

  • Core belief: teams do not have a "generate more tests" problem; they have a release confidence problem.

  • What competitors miss: bug detection without business weighting creates noisy output and weak ship/no-ship decisions.

  • What blop uniquely does: connect journeys, evidence, release history, and governance into a single risk narrative that leaders can act on.

Documentation contract


What does it actually do?

  1. Captures context with inventory + graph resources agents can read cheaply

  2. Discovers and records business-critical flows from that context

  3. Replays flows asynchronously to catch regressions before release

  4. Correlates evidence across screenshots, traces, run health, and telemetry signals

  5. Scores risk so teams can prioritize or gate releases with confidence

It plugs into Cursor or Claude Code as an MCP tool — meaning you just ask it to run tests in a chat window, the same way you'd ask a colleague.

Product map

  • P1 OSS Core: this repo's main focus — local MCP runtime, journey discovery/recording/replay, and evidence capture.

  • P2 Hosted Workflow: sync, release dashboard, history, sharing, and the system of record where release confidence lives for teams.

  • P3 Governance Engine: policy, ship/hold/block reasoning, ownership, and signoff.

  • P4 Intelligence Layer: impact taxonomy, recurring insight, and telemetry correlation once aggregate signals are trustworthy.

Who this helps

  • QA + developers: quickly discover, record, and replay business-critical flows with deterministic evidence.

  • Engineering managers: tie regressions to business impact with release-risk scoring, clusters, and remediation guidance.

Trust and operations at a glance

  • Read-only context: resources (blop://...) are for low-token retrieval and planning.

  • Action tools: tools execute browser actions, replays, recording, and risk analysis.

  • Artifact storage: runs, screenshots, traces, and logs are persisted locally (.blop/ and runs/).

  • Local-first runtime, broader product: the OSS runtime works fully offline by default, while the broader product thesis also includes hosted sync, release history, dashboarding, and sharing workflows.

  • Auth behavior: auth sessions are cached and validated; expired sessions are surfaced before critical runs.

Roadmap-aware release confidence

blop is not just a browser runner. The broader product thesis is that release decisions should reflect product context such as customer goals, acceptance criteria, and release scope. This OSS repo mainly implements the local execution and evidence plane today, but the docs and interfaces should still frame those runs as inputs into a roadmap-aware release-confidence workflow.


Why blop vs generic AI testing agents?

Generic browser agents optimize for test execution throughput. blop optimizes for decision quality under uncertainty.

Dimension

Generic AI browser runner

blop MCP-native approach

Output shape

Mostly conversational text

Structured contracts + typed envelopes via blop_v2_get_surface_contract and v2 resources

Context handling

Re-run flows to recover context

Read blop://... resources first (inventory, context-graph, artifact-index, stability-profile)

Ops model

One-shot execution focus

Async run lifecycle with health stream, run states, and artifact indexing

Release decisions

Manual interpretation

Risk scoring, incident clustering, remediation drafts, telemetry correlation

Client portability

Tooling-specific patterns

Standard MCP tool/resource model across Cursor, Claude Code, and compatible clients

Proof points in implementation:

  • Contract definitions and stable resource envelope in src/blop/tools/v2_surface.py

  • Correlation/risk persistence in src/blop/storage/sqlite.py

  • Structured run reporting in src/blop/reporting/results.py

If your release process needs a deterministic answer to "can we ship this safely?", blop is purpose-built for that question.


Before you start — what you'll need

What

Where to get it

Takes

Python 3.11 or newer

python.org/downloads

5 min

uv (fast Python installer)

Run curl -LsSf https://astral.sh/uv/install.sh | sh in Terminal

1 min

Google API key (free tier works)

aistudio.google.com/app/apikey

2 min

Cursor or Claude Code

cursor.com or npm i -g @anthropic-ai/claude-code

5 min

Chromium runtime

Installed by playwright install chromium --with-deps --no-shell

2-5 min


Installation — step by step

Open Terminal and run these commands one at a time:

# 1. Go into the blop folder
cd /path/to/blop-mcp

# 2. Create a Python environment
uv venv && source .venv/bin/activate

# 3. Install blop
uv pip install -e .

# 4. Install the browser blop controls
playwright install chromium --with-deps --no-shell

You should see no errors. If you do, check the Troubleshooting section below.

Release packaging

blop-mcp now supports a standard Python distribution path for PyPI publishing while preserving the blop and blop-mcp CLI commands.

Local maintainer smoke check:

uv pip install -e ".[dev]"
python -m build
python -m venv /tmp/blop-dist-smoke
source /tmp/blop-dist-smoke/bin/activate
pip install dist/*.whl
blop --help
blop-mcp --help

This should build both the sdist and wheel, install the wheel into a clean environment, and verify the console entrypoints start.

Release instructions:

  • Build and verify distributions locally using the smoke check above.

  • Upload the verified artifacts to PyPI with python -m twine upload dist/*.

  • Full maintainer notes: docs/releasing.md


Configure and connect blop

1) Configure credentials

Copy the example config file and fill it in:

cp .env.example .env

Then open .env in any text editor and fill in your values:

# Required — get this from aistudio.google.com/app/apikey
GOOGLE_API_KEY=your_key_here

# Your app's URL
APP_BASE_URL=https://your-app.com

# Login details (only needed for testing authenticated pages)
LOGIN_URL=https://your-app.com/login
TEST_USERNAME=your@email.com
TEST_PASSWORD=your_password

Everything else can stay as-is.

Optional Blop Cloud sync:

# Leave these unset for local-only mode.
BLOP_HOSTED_URL=https://app.blop.dev
BLOP_API_TOKEN=blop_sk_...
BLOP_PROJECT_ID=00000000-0000-0000-0000-000000000000

validate_release_setup treats Blop Cloud sync as non-blocking: it warns on partial config and probes /api/v1/sync/connection when all three values are present.

2) Connect to your IDE

Cursor

  1. Open Cursor

  2. Go to Settings → MCP

  3. Click Add MCP Server and paste this (update the path to where you cloned blop). You can start from .cursor/mcp.json.example and copy it to .cursor/mcp.json locally if you prefer a file-based setup (the real mcp.json is gitignored).

{
  "mcpServers": {
    "blop": {
      "command": "uv",
      "args": ["--directory", "/path/to/blop-mcp", "run", "python", "-m", "blop.server"],
      "env": {
        "GOOGLE_API_KEY": "your_key_here",
        "APP_BASE_URL": "https://your-app.com",
        "LOGIN_URL": "https://your-app.com/login",
        "TEST_USERNAME": "your@email.com",
        "TEST_PASSWORD": "your_password"
      }
    }
  }
}
  1. Restart Cursor. You should see blop listed as a connected tool in Settings → MCP.

Claude Code

Run this once in your terminal:

claude mcp add blop /path/to/blop-mcp/.venv/bin/blop-mcp \
  -e GOOGLE_API_KEY="your_key_here" \
  -e APP_BASE_URL="https://your-app.com" \
  -e LOGIN_URL="https://your-app.com/login" \
  -e TEST_USERNAME="your@email.com" \
  -e TEST_PASSWORD="your_password"

Type /mcp in Claude Code to verify — you should see blop: connected.


Production setup (local managed stdio)

blop production is optimized for a client-managed stdio MCP process (Cursor/Claude launching blop-mcp), with strict runtime/path validation and least-privilege tool exposure.

Recommended production posture:

  • BLOP_ENV=production

  • BLOP_REQUIRE_ABSOLUTE_PATHS=true

  • absolute BLOP_DB_PATH, BLOP_RUNS_DIR, and BLOP_DEBUG_LOG

  • BLOP_ALLOW_INTERNAL_URLS=false (default-safe URL policy)

  • BLOP_CAPABILITIES_PROFILE=production_minimal

  • BLOP_ENABLE_COMPAT_TOOLS=false unless explicitly required


Your first MCP-native run in 5 minutes

Open a chat window in Cursor or Claude Code and paste this (swap in your app URL):

Use blop to test https://your-app.com

1. Call validate_release_setup(app_url="https://your-app.com") and stop if status is "blocked"
2. Call discover_critical_journeys(app_url="https://your-app.com") to find the release-gating journeys
3. Read blop://journeys and pick the gated journeys that matter for this release
4. Record or refresh those journeys with record_test_flow(...) so release checks run against real saved flows
5. Call run_release_check(app_url="https://your-app.com", journey_ids=[...], mode="replay")
6. Poll get_test_results(run_id="...") until status is terminal
7. Read blop://release/{release_id}/brief and blop://release/{release_id}/artifacts
8. If the decision is not SHIP, call triage_release_blocker(run_id="...") and summarize blockers, evidence, and next actions

That is the canonical MVP loop: validate, discover, record, replay, and triage.

Control-plane workflow (business context + QA)

  1. Preflight: confirm readiness with validate_release_setup.

  2. Discover: identify release-gating paths with discover_critical_journeys.

  3. Record: capture or refresh the gated journeys with record_test_flow.

  4. Execute: run run_release_check in replay mode against recorded flows.

  5. Triage: use triage_release_blocker plus blop://release/* resources to turn failures into decisions.

targeted mode is still available for one-off exploratory checks, but it is a shortcut, not the golden path for release gating. For larger public sites, you can raise its one-shot budget with BLOP_TARGETED_MAX_STEPS (default 40).

Auth session guidance

For protected apps, the most reliable path is:

  1. Capture auth with capture_auth_session(...)

  2. Run validate_release_setup(app_url="https://your-app.com", profile_name="your_profile")

  3. Only then launch run_release_check(..., mode="replay")

If a run returns waiting_auth or validation says the session is expired:

  • Re-run capture_auth_session(...) to refresh the saved session.

  • Re-run validate_release_setup(...) to confirm the session lands inside the app.

  • Retry the replay only after validation is clean.

If failure output points to stale recordings, refresh the affected journey with record_test_flow(...) before trusting replay failures as real regressions.

Production Client Quickstarts

Canonical MCP-client setup guides:

Operational references:

For each candidate release, summarize:

  • Decision: ship or hold

  • Risk level and score (from run outcomes + criticality weighting)

  • Top 3 risks with direct evidence paths

  • Immediate mitigation actions and target owner(s)


Playwright-MCP compatible mode

blop now includes an additive compatibility layer for prompt portability across MCP clients that expect Playwright-style browser_* tools (Cursor, Claude, Codex, Copilot, Windsurf, etc.).

Enable it by adding compat_browser to capabilities:

export BLOP_CAPABILITIES=core,auth,debug,compat_browser

Optional compatibility env vars:

Variable

Default

Purpose

BLOP_COMPAT_OUTPUT_DIR

.playwright-mcp

Where compatibility artifacts (snapshots/screenshots/storage-state files) are written

BLOP_COMPAT_HEADLESS

true

Run compatibility browser session headless or headed

BLOP_COMPAT_TEST_ID_ATTRIBUTE

data-testid

Preferred test id attribute used when building element selectors

BLOP_COMPAT_SNAPSHOT_MODE

incremental

Snapshot mode hint for compatibility workflows

Typical interop flow:

  1. browser_navigate(url=...)

  2. browser_snapshot() to obtain ref handles

  3. browser_click(ref=...) / browser_type(ref=..., text=...)

  4. browser_tabs(action="list"|"new"|"select")

  5. browser_console_messages(...), browser_network_requests(...)

  6. browser_close()

Auth bridge behavior:

  • If a profile_name is passed to browser_navigate, blop resolves that saved auth profile first.

  • If no profile is provided, compatibility mode falls back to env-driven auth state resolution.

  • This keeps compatibility tools aligned with existing blop auth workflows (save_auth_profile, capture_auth_session).

  • browser_* cookie/state/route tools act on the shared compat session only; use get_browser_cookies, set_browser_cookie, save_browser_state, and mock_network_route when you want URL-scoped storage operations or regression-run mocks.

Tool confusion matrix (use this / not that):

If you want to...

Use this

Not that

Why

Inspect cookies for a specific URL/profile

get_browser_cookies(app_url, profile_name?)

browser_cookie_list()

get_browser_cookies is URL-scoped and runs in an ephemeral context; browser_cookie_list reads the shared compat session.

Set a cookie for URL-scoped auth state

set_browser_cookie(app_url, ...)

browser_cookie_set(...)

set_browser_cookie persists URL/profile state for blop auth flows; browser_cookie_set mutates only the shared compat session.

Save URL/profile storage state to disk

save_browser_state(app_url, ...)

browser_storage_state(...)

save_browser_state captures URL-scoped state; browser_storage_state exports the current shared compat session.

Mock APIs during regression replay runs

mock_network_route(...)

browser_route(...)

mock_network_route applies during regression execution; browser_route only affects the shared compat browser session.

Capture one-off exploratory QA output

evaluate_web_task(...)

record_test_flow(...)

evaluate_web_task returns immediate report output for one-off checks (see full reference); record_test_flow creates reusable flow artifacts for regression.

Create reusable regression flow IDs

record_test_flow(...)

evaluate_web_task(...)

record_test_flow is the source of reusable flow_id values consumed by run_release_check(..., mode="replay").

Inspect page interaction structure from crawling context

get_page_structure(app_url, url?)

browser_snapshot(...)

get_page_structure is crawl/discovery-oriented; browser_snapshot is for current shared compat session state.

Start/stop the long-lived compat browser interaction loop

browser_navigate(...), browser_snapshot(...), browser_click(...)

discover_critical_journeys(...)

browser_* tools are imperative session controls; journey discovery is planning/crawl output, not interactive control.


Real-world testing scenarios for B2B SaaS

Scenario 1 — New app, zero knowledge

Use blop to discover and test https://new-saas-app.com from scratch.

1. Call validate_release_setup(app_url="https://new-saas-app.com")
2. Call discover_critical_journeys with business_goal="Find all revenue-critical journeys including signup, checkout, and onboarding"
3. Record the top suggested gated journeys with `record_test_flow`
4. Call run_release_check in replay mode and summarize anything that blocks shipping

Scenario 2 — Before a release

We're about to ship a new version. Use blop to run a pre-release check on https://staging.myapp.com:

1. list_recorded_tests — show what flows we have
2. run_release_check on all release-gating flows against staging
3. get_test_results — compare pass/fail to last run
4. triage_release_blocker or debug_test_case on anything that changed from pass to fail

Scenario 3 — Test the full authenticated product

Test the authenticated product experience on https://app.myapp.com:

1. save_auth_profile("prod-user", "env_login", login_url="https://app.myapp.com/login")
2. record_test_flow for: dashboard load, core feature (e.g. "create new project"), settings page, and billing/upgrade journey
3. run_release_check with profile_name="prod-user"
4. get_test_results — show me the full breakdown

Scenario 4 — Investigate a specific bug report

A user reported the checkout button isn't working. Use blop to investigate:

1. record_test_flow("https://myapp.com", "checkout_bug", "Navigate to pricing, click the Pro plan CTA, and verify checkout loads")
2. run_release_check on that flow
3. get_test_results — check step_failure_index and assertion_failures
4. debug_test_case on the failure to get screenshots and a plain-English explanation

Scenario 5 — Auth-gated flows with SSO

Option A — Interactive capture (recommended): Use capture_auth_session so blop opens a browser and you log in once; it saves the session and creates the profile automatically.

capture_auth_session(
  profile_name="sso-session",
  login_url="https://your-app.com/login",
  success_url_pattern="/dashboard",
  timeout_secs=120
)

For providers (e.g. Google, LinkedIn) that block "headless" or fresh contexts, use a persistent profile:

capture_auth_session(
  profile_name="sso-session",
  login_url="https://your-app.com/login",
  success_url_pattern="/dashboard",
  user_data_dir=".blop/chrome_profile_myapp"
)

Option B — Manual export: Log in manually, export the session (e.g. Playwright or Chrome DevTools -> Application -> Storage), then:

save_auth_profile(
  profile_name="sso-session",
  auth_type="storage_state",
  storage_state_path="/path/to/my-session.json"
)

MCP resources + v2 surface summary

Canonical release-confidence resources:

  • blop://health

  • blop://journeys

  • blop://release/{release_id}/brief

  • blop://release/{release_id}/artifacts

Use resources for cheap context retrieval before action:

  • blop://inventory/{app}

  • blop://context-graph/{app}

  • blop://run/{run_id}/artifact-index

  • blop://flow/{flow_id}/stability-profile

Use v2 for release-level governance:

  • contracts: blop_v2_get_surface_contract

  • release risk: blop_v2_assess_release_risk

  • journey health: blop_v2_get_journey_health

  • incidents/remediation: blop_v2_cluster_incidents, blop_v2_generate_remediation

  • telemetry correlation: blop_v2_get_correlation_report

Detailed resources and v2 references are provided below in the full reference sections.


Full tool reference

The canonical release-confidence workflow uses:

  • validate_release_setup

  • discover_critical_journeys

  • record_test_flow

  • run_release_check

  • triage_release_blocker

Legacy names such as validate_setup, discover_test_flows, and run_regression_test still appear below for compatibility context, but they should not be treated as the default path in new docs or prompts.

Recommended order for reliable MCP workflows:

  1. Preflight and contract

    • validate_release_setup

    • blop_v2_get_surface_contract

  2. Context before action

    • explore_site_inventory, get_page_structure

    • read blop://inventory/..., blop://context-graph/...

  3. Execution

    • discover_critical_journeys, record_test_flow, run_release_check

  4. Observability and governance

    • triage_release_blocker, get_run_health_stream, get_test_results, get_risk_analytics

    • v2: release risk, journey health, incident clustering, remediation, correlation

Detailed tool behavior is below.

1. discover_critical_journeys"What should gate this release?"

Crawls your app and asks AI to figure out what the important user journeys are. Returns a list of suggested flows with descriptions of what to verify.

Basic usage:

discover_critical_journeys("https://your-app.com")

With more context (gets better results):

discover_critical_journeys(
  app_url="https://your-app.com",
  business_goal="Find all revenue-critical flows like checkout and upgrade",
  max_depth=2
)

Parameters:

Parameter

What it does

Example

app_url

The website to scan

"https://app.example.com"

business_goal

Tell it what matters most to your business or release scope

"Focus on checkout and onboarding"

profile_name

Use a logged-in account to scan private pages

"my-auth-profile"

max_depth

How deep to crawl (1 = homepage only, 2 = homepage + linked pages)

2

max_pages

Max pages to crawl before planning flows

20

seed_urls

Start crawl from specific same-origin URLs

["https://app.example.com/pricing"]

include_url_pattern

Regex: only crawl URLs that match

`"/(pricing

exclude_url_pattern

Regex: skip noisy URLs

`"/(blog

What you get back: Journeys include business context and release-gating hints so you can decide what must be recorded and replayed before ship/no-ship decisions.

{
  "flows": [
    {
      "flow_name": "user_login",
      "goal": "User logs in with email and password and reaches the dashboard",
      "severity_if_broken": "blocker",
      "confidence": 0.92,
      "business_criticality": "activation"
    },
    {
      "flow_name": "pricing_page_upgrade",
      "goal": "Visitor views pricing, clicks Pro plan CTA, reaches checkout",
      "severity_if_broken": "high",
      "confidence": 0.85,
      "business_criticality": "revenue"
    }
  ],
  "inventory_summary": {
    "auth_signals": ["sign in", "/login"],
    "business_signals": ["pricing", "checkout"]
  },
  "quality": {
    "passed": true,
    "warnings": []
  }
}

2. explore_site_inventory"Map the interface before planning tests"

Runs inventory-only discovery (no Gemini flow planning) so you can inspect routes, forms, headings, auth signals, and business signals first. It now also includes page_structures, a compact per-page list of interactive ARIA nodes (role + name) to give agents layout context.

explore_site_inventory(
  app_url="https://your-app.com",
  max_depth=2,
  max_pages=20,
  include_url_pattern="/(pricing|signup|dashboard)"
)

Use this when you want deterministic topology mapping before discover_critical_journeys.


3. get_page_structure"Give me structure for one route right now"

Captures a single-page interactive structure snapshot using Playwright's accessibility tree. Useful before recording or debugging when you want context for one URL without running a full crawl.

get_page_structure(
  app_url="https://your-app.com",
  url="https://your-app.com/pricing",  # optional; defaults to app_url
  profile_name="my-auth-profile"       # optional
)

Returns a flattened interactive_nodes list so MCP agents can reason about what controls are available before choosing actions.


4. save_auth_profile"Here are my login credentials"

Saves your login details so blop can test pages that require being signed in. Your password is only stored locally on your machine.

Basic usage (username + password from your .env file):

save_auth_profile(
  profile_name="my-app-login",
  auth_type="env_login",
  login_url="https://your-app.com/login"
)

Auth types explained:

Type

When to use

Example

env_login

You have a username + password

Standard email/password login

storage_state

You have a browser session file from Playwright

SSO, OAuth, MFA flows

cookie_json

You have exported browser cookies

When you can't automate login

Tips:

  • blop caches sessions for 1 hour — it won't re-login every time you run tests

  • Your credentials are read from environment variables, never stored as plain text in the database

  • For SSO/Google login, use storage_state or the interactive capture_auth_session tool (see below)

  • Use user_data_dir when the login provider (e.g. Google, LinkedIn) blocks fresh browser contexts — blop will use a persistent Chromium profile


5. capture_auth_session"Log in once in a browser, I'll save the session"

Opens a visible browser at your login URL. You complete Google/GitHub OAuth, MFA, or any flow by hand. The tool polls the page URL every 500ms and, when it detects success, saves the Playwright storage state and creates an auth profile automatically. No manual session export needed.

Basic usage:

capture_auth_session(
  profile_name="my-app-sso",
  login_url="https://your-app.com/login",
  success_url_pattern="/dashboard"
)

Parameters:

Parameter

What it does

Example

profile_name

Name for the saved auth profile

"my-app-sso"

login_url

URL to open (your app's login or OAuth start)

"https://app.example.com/login"

success_url_pattern

URL substring that means "logged in" (optional)

"/dashboard" — if omitted, any URL change away from the login page counts as success

timeout_secs

Max seconds to wait for you to complete login (default 120)

180

user_data_dir

Path to a persistent Chromium profile dir (optional)

.blop/chrome_profile_myapp — use when OAuth providers treat a fresh browser as a bot

Returns: status is "captured" (session saved, profile ready for record_test_flow and run_release_check) or "timeout" (no success detected in time). On success you get storage_state_path; the profile is already stored — just pass profile_name to other tools.


evaluate_web_task"Run one exploratory task and get a full report now"

Runs a one-shot browser agent evaluation and returns results immediately in the same call. Use this for exploratory QA checks, quick validation, and ad-hoc investigation. If you want a reusable regression artifact (flow_id) for later replay, use record_test_flow instead.

Parameters:

Parameter

Type

Default

Description

app_url

str

required

Base URL to test.

task

str

required

Natural-language objective for the agent.

profile_name

str | null

null

Optional saved auth profile to run as a logged-in user.

headless

bool

false

Run browser headless or visible.

max_steps

int

25

Agent action budget before termination.

capture

list[str] | null

["screenshots","console","network","trace"]

Evidence channels to persist (screenshots, console, network, trace). Invalid values are ignored.

format

str

"markdown"

Output format: markdown, text, or json.

save_as_recorded_flow

bool

false

Whether to promote successful agent actions into a saved RecordedFlow.

flow_name

str | null

null

Optional flow name when save_as_recorded_flow=true.

Return schema (high level):

  • status: "completed" on normal completion, otherwise error status.

  • success: boolean pass/fail signal from evaluation outcome.

  • pass_fail: normalized result (pass, fail, or error).

  • metrics: execution stats such as elapsed time and step counts.

  • agent_steps: normalized step summaries (step, action, description).

  • evidence: logs/artifacts (console_errors, network_failures, screenshots, trace paths when captured).

  • error: populated when input validation/auth/bootstrap/agent execution fails.

  • report: formatted text/markdown block when format requests human-readable output.

Side effects:

  • Writes run artifacts to local storage (runs/...) when evidence capture is enabled.

  • Persists run metadata and health events in the local SQLite store.

  • Optionally creates and saves a reusable recorded flow when save_as_recorded_flow=true.

Example call:

evaluate_web_task(
  app_url="https://your-app.com",
  task="Open pricing, click Pro plan CTA, and verify checkout loads",
  profile_name="my-app-login",
  max_steps=30,
  capture=["screenshots", "console", "network"],
  format="json"
)

Example output (abridged):

{
  "status": "completed",
  "success": true,
  "pass_fail": "pass",
  "metrics": {"elapsed_secs": 18.4, "steps_taken": 9},
  "agent_steps": [
    {"step": 1, "action": "navigate", "description": "Navigate -> https://your-app.com/pricing"},
    {"step": 2, "action": "click_element", "description": "Click element (index 4)"}
  ],
  "evidence": {"console_errors": [], "network_failures": [], "screenshots": ["..."]},
  "error": null
}

6. record_test_flow"Watch and learn this flow"

Runs an AI agent in a real browser to accomplish a goal, and saves every step it takes. You can then replay this recording as many times as you want.

Use record_test_flow when you want a reusable regression artifact (flow_id). For one-off exploratory checks, use evaluate_web_task.

Basic usage:

record_test_flow(
  app_url="https://your-app.com",
  flow_name="user_signup",
  goal="Sign up for a new account with email and verify the welcome screen appears"
)

With authentication and business criticality (for flows behind login):

record_test_flow(
  app_url="https://your-app.com",
  flow_name="create_new_project",
  goal="Log in, create a new project called 'Test Project', and verify it appears in the project list",
  profile_name="my-app-login",
  business_criticality="revenue"
)

business_criticality (optional) is one of revenue, activation, retention, support, other. It is used in results and severity labels (e.g. "BLOCKER in revenue flow: checkout") so you can triage by business impact.

Writing good goals — what makes the difference:

❌ Vague (gets generic results)

✅ Specific (gets reliable tests)

"Test the login"

"Log in with email and password, verify the dashboard shows the user's name in the top right corner"

"Check pricing"

"Navigate to pricing, verify Free, Pro ($35/mo), and Enterprise tiers are visible, click the Pro CTA and confirm it leads to checkout"

"Test the form"

"Fill in the contact form with name, company email, and message, submit it, and verify a confirmation message appears"

What gets captured per step:

  • The element clicked or filled (selector + visible text)

  • A screenshot at that moment

  • The URL before and after

  • Final assertions generated by AI from the end state


7. run_release_check"Can we ship this safely?"

Replays recorded flows against your app and returns a release-confidence decision path. In replay mode it returns immediately with a run_id — poll for results with get_test_results. Run status moves: queuedrunning → (completed | failed | cancelled). If the auth profile cannot be resolved, status is waiting_auth and no flows run until you fix the profile and retry.

flow_ids must come from record_test_flow (or list_recorded_tests) and all IDs must be valid for the run to start.

Basic usage:

run_release_check(
  app_url="https://your-app.com",
  flow_ids=["abc123", "def456"]
)

With auth and hybrid mode:

run_release_check(
  app_url="https://your-app.com",
  flow_ids=["abc123", "def456"],
  profile_name="my-app-login",
  mode="replay",
  run_mode="hybrid"
)

Run modes explained:

Mode

What it does

When to use

hybrid (default)

Tries saved steps first; if a selector breaks, AI repairs that single step

Best for most replay runs

strict_steps

Follows saved steps exactly — fails immediately if anything doesn't match

CI/CD where you want strict enforcement

goal_fallback

Ignores saved steps, replays using only the original goal description

Drift recovery only, not the normal release gate

For release gating, prefer run_release_check(..., mode="replay") with recorded flow_ids. goal_fallback is useful for recovery and diagnosis when a recorded flow has drifted too far, but it should not be the default ship/no-ship path.


8. get_test_results"What broke?"

Polls a running test and returns structured results. Call it repeatedly every 2-5 seconds until status is "completed", "failed", "cancelled", or "waiting_auth". When status is waiting_auth, the response includes waiting_auth_message explaining that the auth profile could not be resolved (check credentials or re-run capture_auth_session if the session expired).

get_test_results("your-run-id-here")

Understanding the results:

{
  "status": "completed",
  "severity_counts": {
    "blocker": 1,
    "high": 0,
    "medium": 2,
    "pass": 4
  },
  "failed_cases": [
    {
      "flow_name": "checkout_flow",
      "severity": "blocker",
      "replay_mode": "hybrid_repair",
      "step_failure_index": 3,
      "assertion_failures": ["Payment page should load after clicking Subscribe"],
      "repro_steps": ["Go to /pricing", "Click 'Get Pro Plan'", "Observe: redirected to homepage instead of checkout"]
    }
  ]
}

Severity levels — what they mean for your business:

Level

Meaning

Action

🔴 blocker

Core workflow is completely broken. Users can't complete a key task.

Fix before shipping anything

🟠 high

Major feature broken, significant user impact

Fix in current sprint

🟡 medium

Partial issue, workaround possible

Fix in next sprint

🟢 low

Cosmetic or edge case

Backlog

pass

Everything worked

No action needed

replay_mode tells you how the test ran:

  • strict_steps — selector matched, step ran exactly as recorded

  • hybrid_repair — original selector broke, AI found the element another way

  • goal_fallback — step-by-step replay failed entirely, fell back to full agent replay

Look for replay trust cues in the response:

  • replay_trust_summary tells you whether replay stayed on the golden path or needs manual review.

  • failure_classification explains whether the failure looks like auth, drift, infra, or product.

  • stale_flow_guidance appears when an old recording should be refreshed before trusting the failure.


9. list_runs"What runs are active or recent?"

Lists recent regression runs, optionally filtered by status.

list_runs(limit=20, status="running")

Useful when you lost a run_id or need to inspect background runs.


10. get_run_health_stream"What happened during this run, step by step?"

Returns control-plane run events (queued, started, per-case completion, completed/failed) so you can inspect lifecycle and timing without opening artifacts first.

get_run_health_stream(
  run_id="your-run-id",
  limit=500  # optional
)

Useful for quick triage when a run exits unexpectedly or when you want to inspect replay/healing metadata at event granularity.


11. get_risk_analytics"Where are our biggest regression risks?"

Aggregates recent runs into high-signal diagnostics:

  • flaky step leaderboard

  • top failing transitions

  • failure rates by business_criticality (revenue/activation/retention/support/other)

get_risk_analytics(limit_runs=30)

Use this to prioritize stabilization work across many runs instead of triaging one run at a time.


12. list_recorded_tests"What tests do I have?"

Compatibility-oriented listing of every recorded flow. For canonical planning context in the release workflow, prefer blop://journeys.

Lists every flow you've ever recorded.

list_recorded_tests()

Returns a list with flow_id, flow_name, app_url, goal, and created_at. Use the flow_id values in run_release_check.


13. debug_test_case"Why exactly did this fail?"

Re-runs a specific failed case in a visible browser window (not headless), captures every screenshot, and generates a plain-English explanation of what went wrong.

debug_test_case(
  run_id="your-run-id",
  case_id="the-failed-case-id"
)

What you get back:

{
  "status": "fail",
  "step_failure_index": 3,
  "replay_mode": "hybrid_repair",
  "assertion_failures": ["Dashboard should show user inbox after login"],
  "why_failed": "The login succeeded but the session cookie was not persisted between the auth step and the dashboard navigation. The app redirected to /login again instead of /dashboard.",
  "repro_steps": ["Navigate to /auth", "Fill email and password", "Click Sign In", "Session lost — redirect back to /auth"],
  "screenshots": ["runs/screenshots/run123/case456/step_003.png"]
}

14. validate_release_setup"Is everything ready to gate a release?"

Checks preconditions before you run flows: GOOGLE_API_KEY, Chromium installed, SQLite DB, optional app_url reachability, and optional auth profile (including whether a storage_state session is still valid). Use it after changing env vars or before a big run.

validate_release_setup(app_url="https://your-app.com", profile_name="my-app-login")

Returns: status is "ready" (all checks passed), "warnings" (e.g. app URL unreachable but you can still run), or "blocked" (e.g. missing API key or Chromium). The checks array lists each condition and whether it passed; blockers and warnings give short messages. If an auth profile's session has expired, the message will suggest re-running capture_auth_session.


MCP resources — low-token context for agents

blop now exposes read-only MCP resources so agents can pull structured context without triggering heavy tool workflows.

blop://inventory/{app}

Latest saved inventory for an app URL.

  • URL-encode the full app URL in {app}.

  • Example:

    • blop://inventory/https%3A%2F%2Fapp.example.com

blop://context-graph/{app}

Latest persisted SiteContextGraph snapshot (nodes, edges, archetype, freshness/confidence metadata).

  • Example:

    • blop://context-graph/https%3A%2F%2Fapp.example.com

blop://run/{run_id}/artifact-index

Artifact index for a run (artifact metadata + case ids), useful before drilling into screenshots/traces.

  • Example:

    • blop://run/abc123/artifact-index

blop://flow/{flow_id}/stability-profile

Flow-level stability profile derived from historical cases (pass/failure rates, replay-mode distribution, stability score).

  • Example:

    • blop://flow/def456/stability-profile

  1. Read inventory + context-graph resources.

  2. Run discover_critical_journeys/record_test_flow using that context.

  3. Run run_release_check.

  4. Read artifact-index + stability-profile.

  5. Use get_risk_analytics for cross-run prioritization.


MCP v2 surface (control plane)

blop v2 expands beyond regression execution into change intelligence, journey health, incident clustering, and remediation orchestration.

New v2 tools

  • blop_v2_get_surface_contract — returns machine-readable request/response schemas + examples for all v2 tools.

  • blop_v2_capture_context — captures a context graph snapshot and structural diff summary.

  • blop_v2_compare_context — compares two graph versions and returns structural/business impact.

  • blop_v2_assess_release_risk — release-level risk score and top risks from context/run evidence.

  • blop_v2_get_journey_health — SLO-like health view for business journeys over time.

  • blop_v2_cluster_incidents — deduplicates failures into incident clusters with blast radius.

  • blop_v2_generate_remediation — emits issue-ready remediation drafts (repro + evidence).

  • blop_v2_ingest_telemetry_signals — ingests external signals (error rate/latency/conversion).

  • blop_v2_get_correlation_report — correlates failures with telemetry changes for prioritization.

New v2 resources

  • blop://v2/contracts/tools

  • blop://v2/context/{urlencoded_app_url}/latest

  • blop://v2/context/{urlencoded_app_url}/history/{limit}

  • blop://v2/context/{urlencoded_app_url}/diff/{baseline_graph_id}/{candidate_graph_id}

  • blop://v2/release/{release_id}/risk-summary

  • blop://v2/journey/{urlencoded_app_url}/health/{window} (window: 24h, 7d, 30d)

  • blop://v2/incidents/{urlencoded_app_url}/open

  • blop://v2/incident/{cluster_id}

  • blop://v2/incident/{cluster_id}/remediation-draft

  • blop://v2/correlation/{urlencoded_app_url}/{window}

Compatibility strategy (v1 + v2)

  • Compatibility aliases remain available for migration (discover_test_flows, run_regression_test, validate_setup), but new workflows should prefer the canonical release-confidence surface.

  • v1 responses now include related_v2_resources links so agents can progressively adopt v2 context.

  • v2 resources use a stable envelope:

{
  "resource_version": "v2",
  "generated_at": "2026-03-18T12:00:00Z",
  "app_url": "https://app.example.com",
  "data": {}
}

How auth profiles work

Your .env file                     blop
─────────────────────────────────────────────────────────────
TEST_USERNAME=user@company.com  →  Reads at runtime
TEST_PASSWORD=secret            →  Never stored in DB
                                   ↓
                              Opens login page
                              Fills credentials
                              Saves session cookie
                                   ↓
                         .blop/auth_state_profile.json
                         (valid for 1 hour, then re-logs in)

blop tries these login field selectors automatically, in order:

  1. input[name="username"]

  2. input[name="email"]

  3. input[type="email"]

  4. #email

  5. Any input with "email" in the placeholder

You can override with env vars TEST_USERNAME_SELECTOR and TEST_PASSWORD_SELECTOR if your login form is unusual.


Where your test data lives

your-project/
├── .env                          ← Your credentials (never commit this)
├── .blop/
│   ├── runs.db                   ← All test history (SQLite)
│   └── auth_state_*.json         ← Cached login sessions
└── runs/
    ├── screenshots/
    │   └── <run_id>/<case_id>/
    │       ├── step_000.png       ← Screenshot at each step
    │       └── step_001.png
    ├── traces/
    │   └── <run_id>/<case_id>.zip ← Playwright trace (open with trace viewer)
    └── console/
        └── <run_id>/<case_id>.log ← Browser console errors

Troubleshooting

"blop not found" or "command not found" Make sure you've activated the virtual environment: source .venv/bin/activate

"GOOGLE_API_KEY not set" Check your .env file exists in the blop-mcp folder and has your key on the GOOGLE_API_KEY= line. Alternatively, set it directly in the MCP config JSON.

Login keeps failing

  1. Double-check your TEST_USERNAME and TEST_PASSWORD in .env

  2. Try visiting your LOGIN_URL manually to confirm the credentials work

  3. If your login uses unusual field names, add TEST_USERNAME_SELECTOR=input[name="your-field"] to .env

  4. For SSO/MFA, use capture_auth_session (opens a browser so you log in once; session is saved automatically) or auth_type="storage_state" with an exported session file

  5. Run validate_release_setup(profile_name="your-profile") to verify the profile; if the session expired, re-run capture_auth_session or refresh your storage state file

Tests are all passing but you know something is broken The regression engine uses AI vision to evaluate assertions — it shouldn't produce false positives. If something is marked pass that looks wrong, use debug_test_case to re-run it with full screenshot capture and see what the browser actually showed.

"MCP server not connected" in Cursor

  1. Check the path in mcp.json points to where you actually cloned blop-mcp

  2. Make sure uv is installed (which uv in Terminal)

  3. Restart Cursor fully (Cmd+Q, reopen)

The browser opens but does nothing / hangs Your BLOP_MAX_STEPS limit (default 50) may be too low for complex flows. Add BLOP_MAX_STEPS=100 to .env.


Exploration profiles (simple but flexible)

blop uses a deterministic-first architecture with adaptive repair fallback. For easier tuning across different interfaces, you can choose a profile:

  • BLOP_EXPLORATION_PROFILE=default — balanced defaults for most apps.

  • BLOP_EXPLORATION_PROFILE=saas_marketing — tuned for async SPAs, heavy client-side editors, and cross-origin handoffs like rendley.comapp.rendley.com.

You can still override individual knobs with env vars (BLOP_NETWORK_IDLE_WAIT, BLOP_SPA_SETTLE_MS, BLOP_AGENT_MAX_FAILURES, BLOP_AGENT_MAX_ACTIONS_PER_STEP, BLOP_DISCOVERY_MAX_PAGES).

Design baseline references:


Environment variables — full reference

Variable

Required

Default

What it does

GOOGLE_API_KEY

Yes

Gemini API key. Get one free at aistudio.google.com

APP_BASE_URL

No

Default app URL (used as fallback if no URL passed to tools)

LOGIN_URL

No

Where blop navigates to log in

TEST_USERNAME

No

Login email/username

TEST_PASSWORD

No

Login password

TEST_USERNAME_SELECTOR

No

auto-detected

CSS selector for the username input field

TEST_PASSWORD_SELECTOR

No

auto-detected

CSS selector for the password input field

STORAGE_STATE_PATH

No

Path to a saved Playwright session (for SSO/OAuth)

COOKIE_JSON_PATH

No

Path to exported browser cookies (JSON array)

BLOP_DB_PATH

No

.blop/runs.db

Where blop stores its database

BLOP_HEADLESS

No

true

false = show browser window during tests (useful for debugging)

BLOP_MAX_STEPS

No

50

Max steps the AI agent takes per flow

BLOP_ALLOW_SCREENSHOT_LLM

No

false

Privacy guard for visual-regression triage. When false, baseline/current screenshots are never base64-encoded or sent to external LLMs.

BLOP_ENV

No

development

Environment mode (production enables stricter validation expectations)

BLOP_REQUIRE_ABSOLUTE_PATHS

No

false (true in production recommended)

Require absolute paths for DB/runs/log values

BLOP_ALLOW_INTERNAL_URLS

No

false

Block private/internal app URLs unless explicitly enabled

BLOP_ALLOWED_HOSTS

No

Optional host allowlist for app_url validation

BLOP_RUN_TIMEOUT_SECS

No

0

Total run timeout in seconds (0 disables timeout)

BLOP_STEP_TIMEOUT_SECS

No

45

Per-step replay timeout in seconds

BLOP_DEBUG_LOG

No

.blop/blop.log

JSON log destination path

BLOP_CAPABILITIES_PROFILE

No

env-dependent

Predefined capability profile (production_minimal, production_debug, full)

BLOP_ENABLE_COMPAT_TOOLS

No

false

Registers legacy/compat MCP tool surface when true

BLOP_EXPLORATION_PROFILE

No

default

Tuning preset (default or saas_marketing) for discovery and replay behavior

BLOP_DISCOVERY_MAX_PAGES

No

profile-driven

Default crawl page cap for discovery tools

BLOP_AGENT_MAX_FAILURES

No

profile-driven

Max recoverable action failures before agent aborts recording

BLOP_AGENT_MAX_ACTIONS_PER_STEP

No

profile-driven

Max agent actions per reasoning step during recording

BLOP_NETWORK_IDLE_WAIT

No

2.0

Seconds to wait for network idle after page load (increase for WebGL/WASM or slow dashboards)

BLOP_SPA_SETTLE_MS

No

1500

Extra settle time in ms after SPA navigation (for pushState / client-side routing)


Powered by Browser Use and Google Gemini


Origins / Attribution

blop was initially developed as a fork of browser-use/vibetest-use. The codebase has since been entirely rewritten with a new architecture, engine, tool surface, and storage layer. This repository (blop-mcp) is the canonical home for blop going forward.

If the upstream vibetest-use project's license requires attribution, see the upstream repository for license details.

Available Tools

28 tools
cancel_runB

Cancel a running test and mark it as cancelled.

Args: run_id: The run_id to cancel

Returns: dict with run_id, previous_status, new_status

ParametersJSON Schema
NameRequiredDescriptionDefault
run_idYes

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full disclosure burden. It states the effect ('mark it as cancelled') and the return structure (previous_status, new_status). However, it does not disclose side effects, irreversibility, or permission requirements, which are relevant for a cancellation operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is succinct and well-structured with clear Args and Returns sections. Every sentence adds value, and the action is front-loaded. No redundancy or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter tool with no output schema, the description provides the essential information: what the tool does, the required argument, and the expected return. It is complete enough for a straightforward cancel operation, though it could mention preconditions (e.g., run must be in progress).

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema provides no parameter descriptions (0% coverage), so the description must compensate. It explains that run_id is 'the run_id to cancel', which adds minimal meaning beyond the schema's title 'Run Id'. It clarifies the role of the parameter but lacks additional context like format or constraints.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Cancel a running test') and the specific outcome ('mark it as cancelled'), which is a distinct operation from sibling tools. It does not explicitly name sibling tools, but the verb and resource are specific enough to avoid confusion.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when to use this tool versus alternatives. The description implies usage (when you need to cancel a running test) but does not state prerequisites, limitations (e.g., cannot cancel completed tests), or mention any related tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

capture_artifactA

Capture screenshot, dom_snapshot, or network_log; optional run_id routes under runs/.

ParametersJSON Schema
NameRequiredDescriptionDefault
kindYes
metadataNo

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden. It discloses that an optional run_id routes under runs/, which is a useful behavioral detail. However, it omits side effects, required permissions, or response behavior, leaving the agent with only partial transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence that conveys purpose and a key routing behavior with no wasted words. Every piece of text adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple 2-parameter tool, the description covers the core purpose and kind values, but leaves gaps: metadata contents are not clearly specified, and the return value or confirmation behavior is not mentioned. Given the lack of annotations and output schema, a bit more detail would be needed for full completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate. It explicitly lists the allowed values for the required 'kind' parameter (screenshot, dom_snapshot, network_log), which is critical since the schema has no enums. It also hints at metadata semantics via the run_id routing note, though the metadata parameter itself remains vaguely defined.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool captures specific artifact types (screenshot, dom_snapshot, network_log) and mentions routing under runs/. The verb 'Capture' plus explicit resource types makes the purpose specific and distinguishes it from sibling tools like capture_auth_session.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by listing artifact kinds, but there is no explicit guidance on when to prefer this tool over alternatives or when not to use it. No exclusions or comparisons to sibling tools are provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

capture_auth_sessionA

Open a headed browser for interactive OAuth/MFA login and save the session state.

A browser window opens — complete Google/GitHub OAuth or any MFA flow manually. The tool polls the URL every 500ms and saves storage state automatically once login succeeds.

Args: profile_name: Name to save the auth profile under login_url: URL of the login page to open success_url_pattern: URL substring that indicates successful login (e.g. "/dashboard") If omitted, any URL change away from login_url counts as success timeout_secs: Max seconds to wait for login (default: 120) user_data_dir: Optional path to a persistent Chromium profile dir (for OAuth providers that detect fresh browser contexts as bots, e.g. Google, LinkedIn)

Returns: dict with profile_name, requested_profile_name, status ("captured" | "timeout" | "error"), storage_state_path, note

ParametersJSON Schema
NameRequiredDescriptionDefault
login_urlYes
profile_nameYes
timeout_secsNo
user_data_dirNo
success_url_patternNo

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full burden and delivers: it discloses the headed browser behavior, polling interval (500ms), automatic storage state save, success detection via URL pattern or any URL change, timeout default, and the return statuses ('captured' | 'timeout' | 'error'). This is rich behavioral disclosure.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with a clear overview, an Args block, and a Returns block. Every sentence adds relevant information, with no filler or redundant repetition of schema details.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex interactive browser tool with 5 parameters and no output schema, the description covers the full flow: setup, behavior, success criteria, and return format. It provides everything an agent needs to invoke and interpret the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, but the description fully compensates by explaining every parameter: profile_name's purpose, login_url, success_url_pattern with its default behavior, timeout_secs with default, and user_data_dir with rationale for OAuth providers. This adds significant meaning beyond the bare schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: 'Open a headed browser for interactive OAuth/MFA login and save the session state.' This specific verb+resource phrasing distinguishes it from siblings like 'save_auth_profile' and 'capture_artifact'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context for when to use the tool (interactive OAuth/MFA flows that require manual login) and includes practical details like using user_data_dir for providers that detect bots. However, it doesn't explicitly name alternatives or state when not to use it, so it stops short of full explicit guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

debug_test_caseA

Re-run a failed test case in headed mode with verbose evidence capture.

Shows the exact step that failed, repair attempt results, per-step screenshots, and a plain-English "why this failed" explanation with concrete next actions.

Args: run_id: The run_id containing the failure case_id: The case_id of the specific failure to debug

Returns: dict with case_id, run_id, status, screenshots, console_log, repro_steps, step_failure_index, replay_mode, assertion_failures, why_failed

ParametersJSON Schema
NameRequiredDescriptionDefault
run_idYes
case_idYes

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden. It discloses headed mode, verbose evidence capture, repair attempts, screenshots, and returns a detailed 'why failed' explanation. It does not mention potential side effects like re-execution cost or auth requirements, but still provides substantial behavioral detail.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured and front-loaded: a one-sentence summary, a brief details paragraph, then clearly formatted Args and Returns lists. Every sentence adds value with no redundancy or fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers purpose, parameters, and return values comprehensively, which is critical given no output schema. It lacks explicit caveats like behavior with invalid run_ids or prerequisites, but for a debug tool it is largely complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has no parameter descriptions (0% coverage), so the description's Args section compensates by explaining each parameter in context: run_id is 'the run containing the failure' and case_id is 'the case_id of the specific failure to debug.' This fully clarifies the parameters beyond the bare schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Re-run a failed test case in headed mode with verbose evidence capture.' It then details what the tool shows (exact failing step, repair attempts, screenshots, explanation), clearly distinguishing it from siblings like get_test_results which likely just retrieve results.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly implies usage for debugging a failed test case by re-running it and providing diagnostic insights. However, it does not explicitly name alternatives or state when not to use this tool, so it lacks explicit exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

discover_critical_journeysB

Crawl app_url and plan 3-8 critical user journeys in business language.

Returns CriticalJourney objects with why_it_matters and include_in_release_gating fields so you can immediately scope which journeys gate a release. Revenue and activation journeys are automatically flagged for release gating.

ParametersJSON Schema
NameRequiredDescriptionDefault
app_urlYes
max_depthNo
max_pagesNo
seed_urlsNo
profile_nameNo
business_goalNo
exclude_url_patternNo
include_url_patternNo

TDQS

B3.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the transparency burden. It discloses a useful behavioral trait: 'Revenue and activation journeys are automatically flagged for release gating.' However, it does not mention auth requirements, persistence effects, or rate limits, which are particularly relevant for a crawling tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description consists of two tight paragraphs with no filler. The first sentence front-loads the primary action ('Crawl app_url and plan 3-8 critical user journeys'), and the second explains the output value. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 8 parameters, no annotations, and no output schema, the description must compensate by explaining parameters and operational behavior. It covers the return value and auto-flagging rule but omits parameter semantics, crawl boundaries, failure modes, and any side effects, making it incomplete for a tool of this complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, and the description only mentions app_url (in the first sentence). The other seven parameters (max_depth, max_pages, seed_urls, profile_name, business_goal, exclude_url_pattern, include_url_pattern) are entirely unexplained, leaving the agent to guess their meaning and valid values.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool crawls app_url and plans 3-8 critical user journeys in business language, which is a specific verb+resource combination. It also indicates the output includes meaningful fields, but it does not explicitly differentiate from sibling tools like get_journeys_for_release or navigate_to_journey.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides a clear use case: 'so you can immediately scope which journeys gate a release.' It does not explicitly say when not to use the tool or mention alternatives, but the release-scoping context implies its intended role in the workflow.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

evaluate_web_taskA

Run a browser agent for a natural-language task and return a rich evaluation report.

One-shot evaluator — give it a URL and a task, get back a structured report with screenshots, console errors, network failures, and an agent step timeline. No need to discover/record/replay first.

Args: app_url: The website URL to evaluate task: Natural-language description of what to test (e.g. "Try the signup flow and note UX issues") profile_name: Optional auth profile name for authenticated pages headless: Run browser in headless mode (default: False — shows the browser) max_steps: Maximum agent steps (default: 25) capture: Evidence to capture: "screenshots", "console", "network", "trace" (default: all four) format: Report format: "markdown" (default), "text", or "json" save_as_recorded_flow: If True, promote the evaluation into a recorded flow for regression flow_name: Flow name to use when saving as recorded flow (auto-generated if omitted)

Returns: dict with summary, agent_steps, evidence (console_errors, network_failures, screenshots, trace_path), pass_fail, run_id, and formatted_report

ParametersJSON Schema
NameRequiredDescriptionDefault
taskYes
formatNomarkdown
app_urlYes
captureNo
headlessNo
flow_nameNo
max_stepsNo
profile_nameNo
save_as_recorded_flowNo

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full burden. It discloses the tool launches a browser, shows the browser by default (headless=false), can save as a recorded flow via save_as_recorded_flow, and captures specific evidence types. It could further mention side effects, resource usage, or auth prerequisites, but covers the major behaviors well.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Structured with an opening summary, 'Args' list, and 'Returns' section. Although lengthy, every line adds value for a 9-parameter tool, and the front-loaded purpose sentence lets an agent quickly identify the tool's role.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema and no annotations, the description is remarkably complete. It covers the input parameters, return value fields, and a key usage scenario (one-shot evaluation). The description fully compensates for the missing structured metadata.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema provides no descriptions (0% coverage), but the description documents all 9 parameters with meanings and defaults. It adds critical semantics like capture options, format choices, and the save_as_recorded_flow behavior, fully compensating for the schema gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool runs a browser agent for a natural-language task and returns a rich evaluation report. It specifies the resources (URL and task) and distinguishes it from sibling tools like record_test_flow by framing it as a one-shot evaluator.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides clear context: use for one-shot evaluation without prior recording, explicitly noting 'No need to discover/record/replay first.' While it implies alternatives like record_test_flow, it does not explicitly name alternatives or say when not to use this tool, so it falls just short of full guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

export_run_traceA

Export OTLP-shaped JSON (resourceSpans) for a run — local SQLite only, no network upload.

ParametersJSON Schema
NameRequiredDescriptionDefault
run_idYes

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure. It does reveal a key safety trait ('no network upload') and the local data source, but it does not explicitly state whether the operation is read-only or whether it writes a file, leaving some ambiguity about side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence that front-loads the action and format, then adds a concise constraint. Every word contributes value, with no repetition or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter tool, the description covers the essential purpose, output format, and storage context. It does not describe the exact response wrapper or explicitly state that data is not modified, but the format statement provides useful context in the absence of an output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has 0% description coverage and only one parameter, run_id, with the generic title 'Run Id'. The description adds 'for a run', which minimally clarifies that run_id identifies the trace source, but it does not provide format, origin, or constraints, so it fails to compensate for the missing schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific action: 'Export OTLP-shaped JSON (resourceSpans) for a run'. It identifies the resource (run trace), the output format, and the scope, distinguishing it from sibling tools like get_test_results or capture_artifact.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear context by specifying 'local SQLite only, no network upload', indicating an offline export use case. However, it does not explicitly name alternative tools or state when not to use this tool, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_journeys_for_releaseC

List recorded journeys filtered by release brief app_url or explicit app_url.

ParametersJSON Schema
NameRequiredDescriptionDefault
app_urlNo
release_idNo

TDQS

C2.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full responsibility for behavioral disclosure. It only says 'List recorded journeys' without indicating whether this is purely read-only, what data it returns, or if any setup (e.g., a release brief) is required. The phrasing implies no side effects but does not explicitly state them.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single short sentence with no redundant words, making it highly concise and front-loaded. However, it is so brief that it under-specifies essential information, so it is not a perfect fit, but it earns a 4 for efficient use of words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with two parameters, no output schema, and no annotations, the description should provide substantial context about what 'recorded journeys' are, the role of each parameter, and the expected result. The current text is too vague and incomplete, leaving the agent uncertain about how to use the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must explain the parameters. It mentions 'app_url' (albeit unclearly as 'release brief app_url or explicit app_url') but completely omits the 'release_id' parameter, which appears in the schema. This leaves a key parameter unexplained and hinders correct invocation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a verb ('List') and a resource ('recorded journeys'), but the distinguishing filter 'release brief app_url or explicit app_url' is ambiguous and does not clarify how this differs from sibling tools like get_release_and_journeys. The purpose is clear at a basic level but lacks specificity and differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is given about when to use this tool versus alternatives such as get_release_and_journeys or discover_critical_journeys. The description provides no situational context, prerequisites, or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_mcp_capabilitiesA

O(1) probe: package version, surface flags, registered tool count, and canonical tool names.

Use this or the blop://health resource before heavier discovery or replay work.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It discloses performance (O(1)) and the read-only nature through the word 'probe.' It also lists exactly what data is returned. However, 'surface flags' is ambiguous, and there is no explicit statement about side effects or auth requirements, but the probe framing implies safety.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences long with no redundancy. The first sentence defines the function, and the second gives usage context. Every word earns its place, making it highly concise and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simplicity of the tool (0 params, no output schema), the description covers the essentials: what it does, what it returns, and when to use it. It lacks details about response format or error conditions, but these are not critical for a lightweight probe. The mention of the blop://health resource adds useful context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so the description cannot add parameter-specific meaning. Per the baseline for 0 parameters, a score of 4 is appropriate. The description focuses on what the tool returns rather than params, which is correct given the empty schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: an O(1) probe that returns package version, surface flags, registered tool count, and canonical tool names. This is a specific verb+resource combination that distinguishes it from sibling tools focused on test flows, releases, and authentication.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly says to use this tool (or the blop://health resource) before heavier discovery or replay work, providing clear when-to-use guidance. It also implies an alternative resource, giving context on positioning relative to other operations.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_page_snapshotC

Compact interactive DOM snapshot (ARIA-ish) for the current page.

ParametersJSON Schema
NameRequiredDescriptionDefault
filenameNo
selectorNo

TDQS

C2.6/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full responsibility for behavioral disclosure. It does not mention whether the tool is read-only, what side effects it might have, how the snapshot is returned, or what 'interactive' means. The description adds no behavioral details beyond the basic purpose.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence, which is concise. However, it is so sparse that it borders on under-specification. The phrase 'ARIA-ish' is unclear and does not add meaningful value. It is not overly verbose, but lacks necessary details.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has no annotations, no output schema, and minimal description. Given its likely use in a QA/testing context (sibling tools include test flows and captures), the description fails to explain how the snapshot fits into the workflow, what the output format is, or how parameters affect results. The description is incomplete for a tool with zero additional documentation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has 2 optional parameters (filename and selector), but the description does not mention them. With 0% schema description coverage, the description must compensate by explaining parameters, but it does not. The parameter names give some hint (e.g., selector targets an element), but their effect on the snapshot is left unexplained.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Compact interactive DOM snapshot (ARIA-ish) for the current page' clearly indicates the tool retrieves a DOM snapshot of the current page. It specifies the resource (current page) and the output type (DOM snapshot), distinguishing it from sibling tools like evaluate_web_task or perform_step. However, terms like 'interactive' and 'ARIA-ish' are vague and not fully explained.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided about when to use this tool versus alternatives. It does not mention prerequisites, typical scenarios, or exclusions. The description is a bare definition without any usage context, leaving the agent to infer when to invoke it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_prd_and_acceptance_criteriaC

Summaries and acceptance-style criteria from recorded flows / release brief (no external PRD yet).

ParametersJSON Schema
NameRequiredDescriptionDefault
journey_idNo
release_idNo

TDQS

C2.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It adds the caveat 'no external PRD yet', which is useful context, but it does not disclose what happens when no parameters are provided, whether it returns both summaries and criteria, or any side effects. The description is too sparse to be transparent about behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single concise sentence with no fluff, but it is under-specified. It is appropriately short but sacrifices essential information. The sentence earns its place in that it provides some context (source and caveat), but it does not cover enough ground to be considered well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness1/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (two optional parameters, no output schema, no annotations), the description is woefully incomplete. It does not explain what the return value looks like, how to choose between journey_id and release_id, or what 'summaries and acceptance-style criteria' actually means in practice. The description is a fragment that leaves too much to inference.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has two parameters (journey_id, release_id) with zero description coverage. The description does not mention either parameter or explain how they relate to recorded flows or release briefs. With 0% schema coverage, the description must compensate, but it fails entirely, leaving the agent without guidance on parameter usage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description identifies the tool's output as 'summaries and acceptance-style criteria' sourced from recorded flows or release brief, which is a specific resource. It does not explicitly use a verb like 'get' or 'retrieve', but the tool name provides that, and the description clarifies the scope and source. It is somewhat distinguished from siblings that focus on releases or journeys, but it does not explicitly differentiate itself.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no explicit guidance on when to use this tool versus alternatives. The phrase 'from recorded flows / release brief' hints at the context, but it does not state exclusions or mention sibling tools like get_release_context or get_journeys_for_release. The usage is implied, not clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_process_insightsB

Derive process-mining style variants from run health events (optional PM4Py when installed).

Uses replay_step_completed and other health events. Install blop-mcp[insights] for PM4Py stats.

ParametersJSON Schema
NameRequiredDescriptionDefault
run_idYes
include_pm4pyNo

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must convey behavioral context. It discloses that the tool uses replay_step_completed and other health events, and that PM4Py stats are optional depending on installation. However, it does not state whether the operation is read-only, what output is returned, or what happens if PM4Py is absent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three short sentences, front-loaded with the core purpose. Each sentence adds valuable information: what it does, what data it uses, and an installation note. No redundant content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with no output schema and no annotations, the description is moderately complete: it explains the main function, data source, and optional dependency. However, it omits expected output structure, any side effects, and explicit usage comparisons with sibling tools, leaving some gaps for an agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has 0% description coverage, so the description must compensate. It does not explicitly explain run_id or include_pm4py, though the PM4Py mention gives a hint about the boolean parameter. The parameter names are self-explanatory, but the description lacks clarity on expected formats or the boolean's effect.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb 'derive' with a resource 'process-mining style variants from run health events', clearly indicating the tool's function. It distinguishes itself from sibling tools like get_test_results or debug_test_case, though it does not explicitly name alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage when process-mining insights from run health events are needed, but it does not state when not to use it or mention alternative tools. The installation note provides some context but no exclusions or preferred scenarios.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_qa_recommendationsA

QA-engineering view: test pyramid health, coverage gaps, flakiness signals, and prioritized recommendations.

Aggregates recorded journeys and recent run cases for app_url, then returns a RecommendationSet plus embedded qa_context (risk matrix, defect mix, pyramid stats). Use scope to narrow the recommendation lists.

ParametersJSON Schema
NameRequiredDescriptionDefault
scopeNofull
app_urlYes
release_idNo
lookback_runsNo

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden of behavioral disclosure. It transparently explains that the tool aggregates recorded journeys and recent run cases for app_url, and describes the return structure (RecommendationSet plus embedded qa_context with specific stats). It omits side-effect safety but for a read-like view this is adequate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two concise sentences, front-loaded with a clear overview and no unnecessary words. Every phrase adds meaningful context.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is moderately complex (no output schema, no annotations, 4 params). The description covers the key output and mentions one param, but lacks semantics for release_id and lookback_runs and does not provide alternative/usage context. It is adequate for basic use but has clear gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate. It only explains 'scope' (use to narrow recommendation lists) and implicitly mentions app_url in the aggregation sentence. release_id and lookback_runs are completely unexplained, and even app_url lacks detailed semantics. This is insufficient for 4 parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a specific QA-engineering view: test pyramid health, coverage gaps, flakiness signals, and prioritized recommendations. It also names concrete outputs (RecommendationSet, qa_context with risk matrix, defect mix, pyramid stats), which distinguishes it from sibling tools like debug_test_case or triage_release_blocker.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides context for when the tool would be used (QA analysis aggregating journeys and run cases) and gives param-level guidance for scope. However, it does not explicitly mention when to prefer this tool over siblings or include exclusions, leaving usage guidance implied rather than explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_release_and_journeysC

Batch: release context plus journeys for the release app URL in one call.

ParametersJSON Schema
NameRequiredDescriptionDefault
release_idYes

TDQS

C2.8/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It does not mention whether the operation is read-only, what the response structure looks like, or any permissions or side effects. The only signal is the 'get' prefix, which is insufficient to understand the tool's behavior beyond its basic purpose.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence that is easy to skim and front-loaded with the core function. It is appropriately concise, though it sacrifices detail for brevity. The structure is clean and to the point.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has a single parameter, no output schema, and no annotations, the description should provide more context about what 'release context' and 'journeys' include and what the response will look like. The current description is too thin to be fully actionable for an agent, especially since it combines two concepts.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has one required parameter, release_id, with 0% description coverage. The description fails to explain anything about the parameter, leaving the agent to infer from the name alone. Since the schema is also unhelpful, the lack of any parameter semantics in the description is a significant gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states that the tool batches release context and journeys, which distinguishes it from sibling tools like get_release_context and get_journeys_for_release. However, the verb 'Batch' is not a standard action verb and the description doesn't explicitly say 'retrieves' or 'returns', though the tool name 'get_*' implies it. The mention of 'release app URL' adds scope, making the purpose reasonably clear.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage when both release context and journeys are needed in a single call, but it does not explicitly state when to use this tool over alternatives or provide exclusions. The phrase 'in one call' hints at batching, but there is no direct guidance about when to choose this over calling the separate siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_release_contextB

Return structured release brief (decision, risk, blockers) for a release_id.

ParametersJSON Schema
NameRequiredDescriptionDefault
release_idYes

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It does not explicitly state whether this is a read-only operation, what happens if the release_id is invalid, or whether any side effects occur. The verb 'Return' implies a read, but this is not made explicit.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with zero wasted words. It conveys the action, output type, and key output components in under 15 words, making it easy to skim and understand.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter tool, the description is minimally adequate but leaves gaps. It does not explain the exact structure of the 'release brief' beyond listing three categories, nor does it provide context about when this is best used relative to sibling tools. The absence of an output schema makes the description the only source for return semantics, but it remains sparse.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It only paraphrases the parameter as 'for a release_id' without adding practical details like format, expected values, or relationships to other tools. The parenthetical list of output content is helpful but does not clarify the parameter beyond what the schema already provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb 'Return' and clearly identifies the resource as a 'structured release brief' with explicit content areas: decision, risk, blockers. This distinguishes it from sibling tools like get_journeys_for_release or get_prd_and_acceptance_criteria, which target different aspects of release context.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided about when to use this tool versus alternatives. It simply states what it does. There is no mention of when to prefer this over triage_release_blocker or get_release_and_journeys, leaving the agent to infer usage from the name and description alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_test_resultsA

Get structured results for a test run.

Prefer run_release_check + blop://release/{release_id}/brief for release gating. This tool remains the detailed run-level payload, now with summary-first fields.

Args: run_id: The run_id returned from run_regression_test

Returns: dict with run_id, status, cases (with assertion_results, replay_mode_used, step_failure_index, artifact_paths), severity_counts, failed_cases, next_actions

ParametersJSON Schema
NameRequiredDescriptionDefault
run_idYes

TDQS

A4.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It describes the return payload in detail (run_id, status, cases with subfields, severity_counts, etc.) and notes the 'summary-first fields' behavior. However, it does not explicitly state that the operation is read-only or has no side effects, though the verb 'Get' strongly implies this.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with a summary sentence, usage guidance, and clearly labeled Args/Returns sections. Every sentence provides useful information without waste. The return field list is detailed but organized and necessary for the agent.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter tool with no annotations or output schema, the description is remarkably complete. It covers the return structure, parameter origin, and relationship to sibling tools. It gives the agent everything needed to correctly invoke and interpret the tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema only provides the parameter name 'run_id', but the description adds crucial context: 'run_id: The run_id returned from run_regression_test.' This tells the agent exactly where to obtain the value, going beyond the schema's bare definition.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Get structured results for a test run.' It also distinguishes itself from the sibling run_release_check by noting it remains the 'detailed run-level payload' while release gating should use run_release_check.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit guidance is provided: 'Prefer run_release_check + blop://release/{release_id}/brief for release gating.' This tells the agent when NOT to use this tool and names the alternative. It also clarifies that this tool is for the detailed run-level payload.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_ux_taxonomyA

Static UX/criticality hints for planning (cached, small JSON).

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden, and it discloses 'static', 'cached', and 'small JSON', which imply a read-only, cheap operation. However, it does not explicitly state side effects or permission requirements, and the meaning of 'criticality hints' is left undefined.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is one concise sentence, front-loading the core concept ('Static UX/criticality hints') and adding useful modifiers ('cached', 'small JSON'). Every word contributes, and there is no fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter tool with no output schema, the description covers the key aspects: what it returns (hints), its usage (planning), and its nature (static, cached, small). It could be more detailed about the JSON structure, but it is adequate for such a simple utility.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so the description does not need to add parameter semantics. The baseline of 4 applies, and the description appropriately avoids inventing parameter details.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description identifies the resource as 'UX/criticality hints' with the purpose 'for planning', which is specific and not a tautology. It distinguishes itself from sibling tools by highlighting 'static' and 'cached', but does not elaborate on the exact meaning of the hints.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'for planning' gives a clear context of when to use this tool, but it does not mention alternatives or exclusions. There is no comparison to sibling tools like get_process_insights, leaving the agent to infer the intended use.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_workspace_contextA

Return compact workspace metadata, resource URIs, and discovery defaults.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A3.6/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of disclosing behavioral traits. It only states what is returned but does not indicate side effects, permissions, rate limits, or whether the operation is read-only. The verb 'Return' implies a read operation, but this is not explicit.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence that efficiently conveys the tool's purpose without wasted words. It fits the conciseness standard for a low-complexity tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given that the tool has no parameters and no output schema, the description provides a reasonable outline of return contents (metadata, URIs, defaults). It is adequate for a simple getter, though 'discovery defaults' is slightly vague and could be expanded.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, and the input schema is empty. Per the baseline for 0 params, a score of 4 is appropriate. The description therefore does not need to add parameter semantics beyond what is already absent.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Return') and identifies a distinct resource ('workspace context') along with the concrete contents (metadata, resource URIs, discovery defaults). This clearly differentiates it from sibling tools like get_release_context or get_mcp_capabilities.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It does not mention any context or exclusions, and there are no sibling references or usage hints.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

package_authenticated_saas_baselineA

Package reusable authenticated SaaS goldens into strict-step release-gate flows.

Use this after discovery or live exploration when you know the stable semantic path you want to gate on. It promotes curated recipes into recorded flows that replay in strict_steps mode and are ready for run_release_check(mode="replay").

Supported recipe_type values:

  • role_click_to_url

  • role_click_to_text

  • selector_then_role_to_url

  • text_click_to_text

  • text_then_text_to_text

  • text_then_selector_to_text

ParametersJSON Schema
NameRequiredDescriptionDefault
app_urlYes
recipesYes
profile_nameNo
baseline_nameYes

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the transparency burden. It does disclose that the tool 'promotes curated recipes into recorded flows that replay in strict_steps mode' and prepares them for run_release_check, giving some insight into the resulting state. However, it omits potential side effects, prerequisites like auth profiles, and any error behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is succinct and front-loaded. The first sentence captures the core purpose, the second provides usage timing, and the list of recipe types is essential. No filler or redundant content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description gives a solid high-level understanding of the tool's role in the workflow (after discovery, before release gate). However, missing parameter documentation, output behavior (no output schema exists), and behavioral details make it only partially complete for a tool with 4 parameters.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It does provide supported recipe_type values, which partially explains the 'recipes' parameter, but it gives no explanation for app_url, baseline_name, or profile_name. This leaves much of the parameter semantics unexplained.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function with a specific verb ('Package') and resource ('reusable authenticated SaaS goldens'), plus defines the outcome ('strict-step release-gate flows'). It also lists supported recipe types, which differentiates the tool from discovery-related siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit when-to-use context: 'Use this after discovery or live exploration when you know the stable semantic path.' It also notes the downstream integration with run_release_check, but does not explicitly mention when not to use the tool or contrast with specific alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

perform_stepC

One structured step: click | type | wait | press_key | navigate (see PerformStepSpec).

ParametersJSON Schema
NameRequiredDescriptionDefault
step_specYes

TDQS

C2.8/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of disclosing side effects. It lists action types like 'click' and 'navigate' that imply browser state changes, but does not describe consequences such as page navigation, waiting behavior, or whether the operation is destructive. The safety profile is unclear.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence of about 13 words, front-loaded with the key information ('One structured step:') and a concise list of operations. Every word earns its place, with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has a complex nested object parameter, no output schema, and no annotations, yet the description is extremely sparse. It names the allowed action types but omits details about input format, return value, or practical usage, leaving the agent under-informed for a general-purpose execution tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has a single object parameter 'step_spec' with no property descriptions (0% coverage). The description only hints that the spec can contain one of the listed step types, but does not explain the required structure or any nested fields, providing minimal compensation for the schema gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb (perform) and resource (a structured step), and enumerates the specific allowed operations (click, type, wait, press_key, navigate), which distinguishes it from sibling tools like record_test_flow or get_process_insights. However, the reference to 'PerformStepSpec' is not expanded, leaving some ambiguity about the exact step format.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives no explicit guidance on when to use this tool versus alternatives such as navigate_to_url or record_test_flow. The mention of 'One structured step' implies a low-level execution tool, but there is no clarification about prerequisites, intended scenarios, or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

record_run_observationB

Idempotent agent observation keyed by (run_id, observation_key).

ParametersJSON Schema
NameRequiredDescriptionDefault
run_idYes
observation_keyYes
observation_payloadYes

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses a key behavioral trait—idempotency—and the unique composite key, which adds value beyond the tool name. However, with no annotations, it leaves out critical behavior such as what happens when the same key is reused with a different payload, whether it overwrites or errors, and any permission or side-effect implications.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single, front-loaded sentence with no redundant words. It efficiently communicates the essential idempotency and key structure, making every word earn its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no annotations, no output schema, and a terse description, the tool is under-specified. It lacks context about what constitutes an observation, expected payload format, duplicate handling, or return behavior, making it incomplete for safe autonomous invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The description states that run_id and observation_key form the key, giving relational meaning beyond the bare schema. It does not explain the purpose or structure of observation_payload, which remains undocumented, leaving a partial gap at 0% schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the action as recording an agent observation with an idempotency guarantee, keyed by run_id and observation_key. It specifies the resource and key structure, though it does not explicitly differentiate this tool from siblings like record_test_flow or capture_artifact.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus alternatives, nor any mention of exclusions or prerequisites. The description only defines what the tool is, not when it should be invoked.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

record_test_flowA

Record a test flow by running a Browser-Use agent to accomplish a goal.

Captures each action with selector, target_text, dom_fingerprint, per-step screenshots, and generates final assertion steps from a Gemini screenshot analysis.

Args: app_url: The website URL to test flow_name: Short name for this flow (used as identifier) goal: Plain-English description of what to accomplish profile_name: Optional auth profile name (from save_auth_profile) command: Optional natural language command for additional context business_criticality: "revenue" | "activation" | "retention" | "support" | "other"

Returns: dict with flow_id, flow_name, step_count, status, artifacts_dir

ParametersJSON Schema
NameRequiredDescriptionDefault
goalYes
app_urlYes
commandNo
flow_nameYes
profile_nameNo
business_criticalityNoother

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Without annotations, the description carries the burden. It discloses that the tool runs an agent, captures step details, and uses Gemini for assertion generation. However, it does not mention side effects like resource consumption, rate limits, or failure modes. The level of detail is moderate but not comprehensive.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with a brief overview, an Args list, and a Returns line. Every sentence provides useful information without redundancy. It is concise yet informative, fitting within a compact paragraph with a clear hierarchy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description provides a return dict overview and parameter details, but lacks edge cases, error handling, and behavior under different conditions. With no annotations or output schema, more contextual information (e.g., timeouts, intended use cases) would improve completeness. However, it covers the core functionality adequately.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema has 0% description coverage, so the description's Args section provides necessary semantics. It explains each parameter clearly, including the enum values for business_criticality and the source of profile_name. This adds value beyond the schema, though some parameters (e.g., command) could use more nuance.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Record a test flow by running a Browser-Use agent to accomplish a goal.' It specifies the resource (test flow) and the action (record), and differentiates from sibling tools like evaluate_web_task by mentioning capture details and assertion generation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage context is implied through the description (recording flows, optional auth profile from save_auth_profile), but no explicit alternatives or exclusions are given. The reference to 'from save_auth_profile' hints at a prerequisite, but the description does not clearly state when to use this tool versus other test-related tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

run_release_checkA

Flagship release confidence tool: replay critical journeys and return a SHIP / INVESTIGATE / BLOCK decision.

In replay mode (default), queues a regression run and returns immediately with run_id for polling. In targeted mode, runs a one-shot agent evaluation synchronously as a shortcut smoke check.

Args: journey_ids: deprecated alias for flow_ids. flow_ids: recorded flow IDs to replay. If omitted, uses all flows matching criticality_filter. criticality_filter: defaults to ["revenue", "activation"]. release_id: optional caller-supplied release identifier (auto-generated if omitted). mode: "replay" (default, golden path for release gating) or "targeted" (one-shot eval). smoke_preflight: Optional advisory smoke sweep before replay. Does not block the release on its own.

ParametersJSON Schema
NameRequiredDescriptionDefault
modeNoreplay
app_urlYes
flow_idsNo
headlessNo
run_modeNohybrid
release_idNo
journey_idsNo
profile_nameNo
smoke_preflightNo
criticality_filterNo

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It discloses important behaviors: returns a decision, queues asynchronously in replay mode, runs synchronously in targeted mode, and smoke_preflight is advisory and non-blocking. It also explains the deprecated alias. However, it omits potential side effects like whether runs are persisted or if specific permissions are required.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured, front-loading purpose and then breaking down modes and arguments clearly. It is appropriately sized for the tool's complexity, with each line adding useful information. The deprecated alias note is valuable but adds a bit of length.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

While the description covers the main modes and return types (decision, run_id), it leaves gaps for a tool of this complexity. The required app_url is not mentioned, and there is no output schema to fall back on. The behavior of headless, run_mode, and profile_name is undocumented, making the description incomplete for full understanding.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must explain all parameters. It explains several (journey_ids, flow_ids, criticality_filter, release_id, mode, smoke_preflight) but completely misses app_url (the only required parameter), headless, run_mode, and profile_name. Additionally, the stated default for criticality_filter (['revenue','activation']) conflicts with the schema's null default, adding confusion.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Flagship release confidence tool: replay critical journeys and return a SHIP / INVESTIGATE / BLOCK decision.' This specifies the verb (replay), resource (critical journeys), and outcome (decision), distinguishing it from sibling tools by its central role in release gating.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It clearly describes the two modes (replay and targeted) and when each is appropriate, positioning replay as the 'golden path for release gating' and targeted as a 'shortcut smoke check.' However, it does not explicitly mention alternative sibling tools or state when not to use this tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

save_auth_profileA

Save an authentication profile for use in test runs.

Args: profile_name: Unique name for this profile auth_type: One of "env_login", "storage_state", or "cookie_json" login_url: Login page URL (required for env_login) username_env: Name of env var holding the username (default: TEST_USERNAME) password_env: Name of env var holding the password (default: TEST_PASSWORD) storage_state_path: Path to a Playwright storage_state.json file cookie_json_path: Path to a JSON file containing cookie objects user_data_dir: Optional path to a persistent Chromium profile directory (helps with anti-bot OAuth)

Returns: dict with profile_name, auth_type, status, note

ParametersJSON Schema
NameRequiredDescriptionDefault
auth_typeYes
login_urlNo
password_envNoTEST_PASSWORD
profile_nameYes
username_envNoTEST_USERNAME
user_data_dirNo
cookie_json_pathNo
storage_state_pathNo

TDQS

A3.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden for behavioral disclosure. It states the return dict but does not mention side effects, persistence location, overwrite behavior, or required permissions. For a save operation, this is a notable gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with an Args block and a Returns line, front-loading the core purpose. Every line earns its place by adding parameter semantics or clarifying expected output without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For an 8-parameter tool with no annotations and no output schema, the description supplies strong invocation guidance: all parameters, defaults, conditional requirements, and return keys. It falls short only on side effects and tool-selection context, but is largely sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has 0% description coverage, yet the description compensates thoroughly by explaining every parameter, its purpose, default values, and conditional requirements. It adds meaningful context such as 'user_data_dir helps with anti-bot OAuth' that the schema alone cannot convey.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool saves an authentication profile for use in test runs, with a specific verb and resource. It does not explicitly distinguish itself from sibling tools like capture_auth_session, so it misses the top score.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides conditional usage details such as login_url being required for env_login and default environment variable names. However, it never explicitly says when to use save_auth_profile versus alternatives like capture_auth_session, so usage guidance is only implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

triage_release_blockerA

Root-cause evidence + next actions for a release blocker.

Accepts any of: run_id, release_id, flow_id, journey_id, incident_cluster_id (at least one required). Returns BlockerTriage with likely_cause, evidence_summary, user_business_impact, recommended_action, suggested_owner, and linked_artifacts.

ParametersJSON Schema
NameRequiredDescriptionDefault
run_idNo
flow_idNo
journey_idNo
release_idNo
incident_cluster_idNo
generate_remediationNo

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It clearly describes the return payload (likely_cause, evidence_summary, etc.) and input flexibility, which is helpful. However, it does not disclose potential side effects, read-only guarantees, error cases, or permissions needed for a tool that analyzes release blockers.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is exceptionally concise: two sentences that front-load the purpose, then cover input requirements and output contents. Every sentence adds meaningful information with no wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the absence of an output schema, annotations, and schema descriptions, the description does a reasonable job of conveying the tool's core contract. But it omits the generate_remediation parameter, fails to explain edge cases like invalid or multiple IDs, and does not situate this tool relative to siblings such as run_release_check or get_release_context. It is adequate but not fully complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It lists five of six parameters and correctly states that at least one identifier is required, which contradicts the schema's optional-looking defaults. However, it omits generate_remediation entirely and gives no semantic detail about what each ID references or how they relate, leaving the agent with only names.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Root-cause evidence + next actions for a release blocker.' It clearly distinguishes this tool from sibling tools like run_release_check or get_release_context by focusing on triage and root-cause analysis, and it further defines the output type (BlockerTriage).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context: use this when you need root-cause evidence and next actions for a release blocker. It also lists acceptable identifier types and notes the 'at least one required' constraint. However, it does not explicitly compare against alternatives or state when NOT to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

validate_release_setupA

Preflight check before a release: verifies API key, Chromium, DB, app reachability, and auth profile.

This is the canonical MVP entry point — run this before discover_critical_journeys or run_release_check.

Args: check_mobile: If True, also checks Appium server reachability for mobile testing.

ParametersJSON Schema
NameRequiredDescriptionDefault
app_urlNo
check_mobileNo
profile_nameNo

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full transparency burden. It discloses the scope of verification (API key, Chromium, DB, app reachability, auth profile) and mentions the check_mobile behavior. However, it does not explicitly state side effects, failure modes, or return value, leaving some ambiguity for a validation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded with the primary purpose. The Args section includes only one of three parameters, making the structure slightly unbalanced, but the prose is efficient and clear.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema and bare parameters, the description should explain all inputs and expected outputs. It omits two parameters and does not describe what the tool returns or how success/failure is communicated, leaving it incomplete for an agent to invoke accurately.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema descriptions cover 0% of parameters, and the description only documents check_mobile. The app_url and profile_name parameters are left entirely unexplained, so the description fails to compensate for the missing schema details.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly defines the tool as a preflight check for releases, listing specific resources it verifies (API key, Chromium, DB, app reachability, auth profile). It distinguishes from siblings by framing it as the canonical MVP entry point before discover_critical_journeys or run_release_check.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit guidance is provided: 'run this before discover_critical_journeys or run_release_check'. This names specific sibling tools and establishes clear ordering, giving the agent a strong sense of when to invoke this tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 28 tool updatesv0.4.0
    • First observedcancel_run
    • First observedcapture_artifact
    • First observedcapture_auth_session
    • First observeddebug_test_case
    • First observeddiscover_critical_journeys
    • First observedevaluate_web_task
    • First observedexport_run_trace
    • First observedget_journeys_for_release
    • First observedget_mcp_capabilities
    • First observedget_page_snapshot
    • First observedget_prd_and_acceptance_criteria
    • First observedget_process_insights
    • First observedget_qa_recommendations
    • First observedget_release_and_journeys
    • First observedget_release_context
    • First observedget_test_results
    • First observedget_ux_taxonomy
    • First observedget_workspace_context
    • First observednavigate_to_journey
    • First observednavigate_to_url
    • First observedpackage_authenticated_saas_baseline
    • First observedperform_step
    • First observedrecord_run_observation
    • First observedrecord_test_flow
    • First observedrun_release_check
    • First observedsave_auth_profile
    • First observedtriage_release_blocker
    • First observedvalidate_release_setup

TDQS

B3.3/5.0

Scored across 28 tools

Disambiguation4/5

Most tools have clear, distinct purposes, but there is some overlap between record_test_flow and evaluate_web_task (both use a browser agent) and between save_auth_profile and capture_auth_session (both deal with auth profiles). The descriptions help clarify, but a few tools could still be confused.

Naming Consistency5/5

All tool names follow a consistent snake_case verb_noun pattern (e.g., record_test_flow, run_release_check, get_release_context). Even longer names like package_authenticated_saas_baseline maintain the convention, making the set predictable.

Tool Count2/5

With 28 tools, the server exceeds the 25+ threshold for 'too many' per the calibration. While the broad QA/release scope justifies some density, the count feels heavy and could overwhelm agents without a clear need for this many distinct operations.

Completeness4/5

The tool surface covers the core QA workflow well: recording flows, running releases, getting results, debugging, and gaining insights. Minor gaps exist, such as no explicit update/delete for flows or auth profiles and no direct list of all flows, but these are workable via existing tools.

Maintenance

ActivityActive
ResponsivenessUnresponsive

Related MCP Connectors

Related MCP Servers

  • F
    license
    A
    quality
    D
    maintenance
    Automated Playwright E2E test repair powered by a self-improving, governed MCP server that runs failing tests, collects failure artifacts, reasons about root causes, validates and applies fixes, and re-runs to verify.
    12
    -
  • A
    license
    Not graded
    quality
    A
    maintenance
    Enables AI agents to securely call MCP tools with risk scoring, checkpoints, rollback, and approval workflows.
    1 npm
    MIT
  • F
    license
    Not graded
    quality
    B
    maintenance
    MCP server for storing and managing QA workflows, encrypted credentials, and browser test run history. Enables agents to create targets, workflows, and retrieve decrypted credential bundles for test execution.
    -