teardown-observe
The Teardown observe server lets an agent black-box observe a target, formalize its behavior into checked rules/claims, track coverage and unknowns, and release a sanitized spec through the gate.
Open CLI, web, Android, or desktop targets with intake metadata and optional strict mode.
Configure mapper rules and run/reset action sequences to capture raw observations or symbols.
Discover CLI interfaces and probe CLI features with sweeps, pairwise tests, and mined claims.
Discover UI screens/controls, probe form fields, and get suggested probes for coverage gaps.
Define named actions and learn state-machine flows under an oracle budget.
State and falsify claims, check observer-authored rules, retract rules, and manage unknowns.
Track coverage, web API contracts, and blocked third-party hosts.
Ingest public docs so copied wording is flagged by the gate.
Compile spec drafts, review gate findings, and release specs (with approvals/gap handling in supported modes).
Answer clean-side tickets and verify the signed audit log hash chain.
Enables black-box observation and analysis of Android apps, capturing controls by role and name with state flags, visible text, and system dialogs. Can inject network faults via airplane mode. Requires Android SDK and an emulator or device via adb.
Enables black-box observation of Linux desktop applications via the AT-SPI tree, capturing windows, controls, text, and list items. Experimental; runs in a Docker container with Xvfb and AT-SPI.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@teardown-observeTreat localhost:3000 as a black box and observe the signup flow."
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Teardown
Clean-room reimplementation by black-box observation.
See a feature you like. Rebuild it from how it behaves, never from its code.
Quick start · How Teardown works · What you can analyze · Results · FAQ · Documentation
uv tool install "teardown-cleanroom[web]" && teardown demo
Teardown is early software (0.3). You declare the purpose of every analysis at intake and own its legal assessment. ReadResponsible use first.
See a feature in an app that you want in your own product? Teardown lets you rebuild it without ever looking at its implementation. One agent, the observer, studies the app as a black box. It records what the app accepts, refuses and defaults to, what it does at the edges, and how its screens are sequenced. Teardown turns that into falsifiable rules and claims and releases them as a sanitized spec. A second agent, the clean implementer, sees only that spec and writes its own implementation. It can check its work against the original, but it only ever gets back abstract pass/fail verdicts.
The valuable knowledge about a feature is behavioral. Tracing the implementation through binaries, minified bundles or obfuscated code is slow, brittle, and copies expression you don't want. Teardown never captures code, decompilation or stack traces. What it cannot observe becomes an explicit unknown, not a silent gap.
Both sides can run enforced, not just instructed. The observer works in a container whose only way to the target is Teardown's drivers. The implementer works in a container that sees only the spec. Every crossing goes through the gate and is recorded in a signed, hash-chained audit log.
Quick start
Install
uv tool install "teardown-cleanroom[web]" # Python 3.12+; the command is `teardown`
teardown install-browser # Chromium for web targets
teardown demo # observe -> gate -> clean -> verdict on a built-in examplepip install "teardown-cleanroom[web]" works too, and
uv tool install "teardown-cleanroom[web] @ git+https://github.com/lemonadesaltbagel/teardown" installs the
latest unreleased version. Docker is needed for CLI targets (each run is isolated in a container)
and for the enforced rooms; an installed copy builds the room images from files it carries. Android needs the SDK
and an emulator or device reachable through adb.
To work on Teardown itself, use a checkout: git clone https://github.com/lemonadesaltbagel/teardown.git, then
uv sync and uv run teardown ....
Set up your agent
Register the observe server with your agent (Claude Code shown; any MCP client works):
claude mcp add teardown-observe -- teardown observe --project /path/to/projectFrom a checkout, use uv run --directory /path/to/teardown teardown observe ... instead.
Then ask:
Open web:http://localhost:3000/ as a research target and follow the observe_target playbook:
discover the UI, probe every field, inject API faults, state rules, close the coverage gaps
and release the spec.The playbook (docs/observer-playbook.md) is served as the observe_target MCP
prompt; the observer guide (docs/observer-guide.md) details every tool.
Review and release
teardown review --project P --feature F # gate findings by class, coverage gaps
teardown approve --project P --feature F --by you@example.com --class expected-output --reason test-oracle
teardown release --project P --feature F
teardown export-clean --project P --out KIT # the kit: released spec + build/verify scaffoldingIn strict mode every release needs your approval at a terminal, and the observer's tools cannot approve anything. In practice that is a few class-level decisions per release: approve the app text the clone must reproduce, approve the action names, acknowledge the coverage gaps the observer explained (docs/strict-gate.md).
Rebuild in the clean room
teardown cleanroom up --project P --kit KIT --claude-token-file ~/.config/teardown-claude-token
teardown cleanroom check --project P # isolation probes, run from inside the container
teardown cleanroom shell --project P -- claude # the implementer sees only /kit; `verify` asks the gate
teardown cleanroom down --project Pteardown observerroom confines the observer the same way
(docs/clean-runtime.md).
Related MCP server: apire
How Teardown works
Observe. Drivers run the target and capture behavior only: outputs and file effects for CLIs, accessibility trees and API exchanges for web apps, accessibility trees for Android. A mapper turns each observation into a compact, wording-agnostic symbol. For web apps that symbol is versioned (
web.aria/v10) and records everything a user-facing Playwright test can tell apart.Generalize. UI discovery crawls every screen. The observer probes fields by equivalence class and boundary, injects faults (server errors, network failures, malformed bodies, unreadable files, airplane mode) and states rules in its own words. Teardown checks every rule against the original and records it as corroborated or falsified. Coverage lists what is left: each control, field, endpoint × fault, data state and live output, every route signed in and signed out, and record pages in each status. A release is refused until every item is exercised or explained.
Gate. The release is checked for copied expression (verbatim and near-verbatim matches against everything observed and every ingested document) and for real names, which become a codename. In strict mode, findings need your recorded approval.
Rebuild. The clean agent builds from the spec and asks the gate to verify its build. The gate runs the original and the clone through the same tests and returns pass/fail with abstract symbols, never the original's text or evidence.
What you can analyze
Target | What Teardown observes | Faults it can inject | Requirements | Status |
Command-line tools | Exit code, stdout, diagnostic presence and errno class, file effects | Closed/full stdio, unreadable files, read-only directories | Docker (isolated runner) or local | Stable |
Web apps | Everything a user-facing Playwright locator or assertion can tell apart (the accessibility tree with roles, names, levels, states and nesting; field values and properties; titles, focus, status messages, landmarks text), plus API method/path/status and public-contract selectors | 5xx, network failure, timeout, malformed body, pending requests, changed responses (server modes); API setup calls to reach deep screens | Playwright Chromium; optional backend reset command | Working: 96.9–99.2% of hidden tests on the last three held-out apps (see Results) |
Android apps | Controls by role and name with state flags, visible text, system dialogs | Network off (airplane mode) | Android SDK, emulator or device via | Working |
Linux desktop apps | AT-SPI tree: windows, controls, text, list items | — | Docker (Xvfb + AT-SPI container) | Experimental |
macOS, Windows, iOS | — | — | — | Not started |
Teardown refuses to bypass protections and never captures implementation-tier data (see Responsible use).
Results
Each result was scored on a hidden test suite the observer never saw, against the original's own score under the same harness. How the evaluation works, and every result with its caveat, is in docs/evaluation.md.
target | hidden suite |
A command-line tool no model has seen | 397/397 from the spec alone |
Web apps no model has seen (the last three held-out apps) | 129/130, 126/130, 125/127 |
Android apps no model has seen | 113/118 and 100/119 (the second built without any verification feedback) |
RealWorld "Conduit" web frontend (tooling tuned against it) | 99/101 |
Flipt v1.61.0, a real open-source web app, on its own suite | 19/55 in the fourth round (from 9 in the first): an application many times larger; 21 of its tests need a server mode the observer could not find, and the other misses are sections the coverage report listed as unexplored (see the evaluation notes) |
The clones reproduced their specs. What the hidden suites caught were things the spec did not contain.
The web abstraction is built from a principle. It records every difference a user-facing Playwright locator or assertion can see, checked by an audit of page pairs that must differ and pairs that must not.
The strict-gate rounds took a few class-level decisions at the person's own terminal per release. The audit log records each one.
Famous targets measure the model's memory too. On
jq, the same model with no spec at all did as well as with it. That is why the benchmarks use apps written for the purpose.
FAQ
Any agent that supports MCP. The observe server runs over stdio or HTTP (teardown observe --port). The
enforced rooms ship a container with Claude Code and register only the room's Teardown server. Any other agent
that can run in the container and use a proxy works too.
No. Drivers capture behavior-tier data only. Stack traces are redacted at capture, Android resource ids and activity names never enter observations, and web API bodies are reduced to value-free shapes. In the observer room, the observer agent cannot reach the target's files, bundles or the internet at all: only Teardown's tools.
The implementer's container mounts only the kit. Its only network peer is an egress proxy that allows the gate
and the model API. The gate accepts only clones built from the kit and returns abstract verdicts. Unexpected
errors come back generic, so dirty-side detail cannot leak through them. cleanroom check proves the isolation
from inside the container.
No container can close that gap. If the implementer's model already knows the target, part of the result is memory. Teardown asks implementers whether they recognized the target and reports it. The uncontaminated benchmarks use apps written for the purpose.
Teardown runs locally. Your agents send their tool results to their model provider, which has its own data policy.
That depends on your jurisdiction, the target's terms and what you build. Teardown records your declared purpose and shows applicable warnings, but it does not give legal advice. A clean room addresses copying of expression. It does not address patents, trademarks or trade dress.
Documentation
Observer playbook: the loop the observer agent follows (also an MCP prompt).
Observer guide: driving the dirty side for CLI, web and Android targets.
Strict gate: reviewing findings and approving releases.
Clean runtime: the enforced clean room and observer room.
Evaluation: the benchmark method and every result with its caveats.
Responsible use
Teardown never:
bypasses CAPTCHAs, bot protection, certificate pinning, root or attestation checks, or FLAG_SECURE;
flips Electron fuses or re-signs binaries;
uses fake accounts or exceeds configured rate limits;
captures implementation-tier data (code bodies, decompilation, stack traces).
These refusals are not configurable. Observe only software you are allowed to observe, within its terms. You declare your purpose at intake and own the legal assessment.
Contributing
Issues and pull requests are welcome. See CONTRIBUTING.md for the ground rules, and report anything that could let information cross the clean-room barrier privately through SECURITY.md. Changes are listed in CHANGELOG.md.
uv sync
uv run ruff check src tests
uv run pytest -q # Android and desktop tests run when an emulator / the desktop image is availableLicense
Apache License 2.0. See NOTICE.
The license covers Teardown's own code and documents. Teardown grants no rights in the software you analyze: observing a target, and building and distributing anything from what you observed, remain subject to that target's terms and the law that applies to you.
Available Tools
31 toolsadd_quoteC
Standard mode only: allow one literal through the gate, with the reason it is functionally necessary.
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes | ||
| approved_by | Yes | ||
| compat_reason | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description carries the full burden. 'Allow' implies a mutating permission grant, but the description doesn't say what is changed, whether it is reversible, what approval flow the approved_by field triggers, or what happens if an existing quote conflicts. It does hint that a functional-necessity justification is required, which is modest value.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
It is a single front-loaded sentence with the mode restriction first, so it is compact. However, the terseness comes at the cost of intelligibility: 'literal' and 'gate' are undefined jargon rather than efficient shorthand.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
This is a likely mutation tool with three required parameters, zero schema coverage, and no annotations; the output schema covers return values but nothing else. The description omits prerequisites, approver semantics, mode definitions, and the effect of the operation, so it is not sufficient to call the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and all three parameters are required with bare titles. The description roughly maps to compat_reason ('the reason it is functionally necessary') and maybe 'text' ('one literal'), but approved_by is completely unaddressed, so it fails to compensate for the coverage gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description says 'allow one literal through the gate,' which is metaphor rather than a concrete verb+resource. An agent cannot tell what artifact is modified (a rule? a spec? a compatibility exception?) or what the effect on the system is, so the purpose is only vaguely inferable from the name and sibling set.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
'Standard mode only' implies a conditional context, but it never defines what standard mode is or what the alternative is. No sibling is named, and with tools like propose_claim, compile_spec, and check_rules present, an agent gets no signal for choosing add_quote over them.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
answer_ticketC
Answer a clean-side question; the answer is gate-checked.
| Name | Required | Description | Default |
|---|---|---|---|
| answer | Yes | ||
| ticket_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description carries the full burden. It hints that the answer is 'gate-checked', which is genuinely behavioral context, but does not say whether a failed gate rejects the answer, raises an error, mutates ticket state, or what permissions are required. The single behavioral hint is left undefined.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single front-loaded sentence with no wasted words, but its brevity comes at the cost of clarity – the jargon-dense phrasing is under-specified rather than crisp. Compact but cryptic.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The output schema covers return values, but with no annotations and zero parameter descriptions, the description needed to explain the 'clean-side' concept, gate semantics, and error behavior. Given the specialized domain vocabulary in the sibling tool set, it leaves the agent unable to call this confidently.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% for both required parameters. The description does not say what format ticket_id takes, whether the answer is free text or structured, length limits, or how 'gate-checked' constrains the answer content. The only hint is that an 'answer' exists and is validated.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The verb 'answer' and resource 'question/ticket' are present, so the basic action is identifiable. However, the two defining terms 'clean-side' and 'gate-checked' are unexplained domain jargon, and nothing distinguishes this from sibling tools like list_tickets or check_rules. Purpose is implied rather than stated.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to invoke this versus alternatives (e.g., list_tickets to obtain a ticket_id, or check_rules to see gate criteria). No prerequisites, ordering constraints, or exclusions are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
api_contractB
Web: the value-free backend contract observed so far (paths, methods, statuses, body shapes). Fold data-bearing path segments with target_options.api_templates.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It does disclose that the contract is value-free and reflects what has been observed so far, which is useful behavioral context, but it does not state read-only behavior, freshness, coverage limits, or what happens if no contract has been observed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences with no filler. The core resource description is front-loaded and the folding instruction follows immediately.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need not be explained, and zero parameters keep the schema burden low. However, the description remains thin on when to use this tool versus the many sibling tools and does not clarify how the observed contract relates to compile_spec or review_spec workflows.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the baseline is 4. The description does not need to explain parameter semantics, though it references target_options.api_templates as external configuration context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description identifies a specific resource — the observed backend API contract — and enumerates contents: paths, methods, statuses, body shapes. It is clearer than the bare name, though it lacks an explicit verb and does not distinguish itself from siblings such as compile_spec or review_spec.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives an instruction about folding data-bearing path segments using target_options.api_templates, but does not say when to call api_contract instead of alternatives, nor does it state prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
approve_observed_outputB
Standard mode only: approve byte-exact output the behavior requires (default: help/usage/version output) as functional quotes. Records who approved it.
| Name | Required | Description | Default |
|---|---|---|---|
| feature | Yes | ||
| action_ids | No | ||
| approved_by | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full behavioral-disclosure burden. It says approval is recorded and restricted to standard mode, but it does not describe side effects, permissions, reversibility, conflicts, or what happens when action_ids is omitted.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two tight sentences with the mode restriction and main action front-loaded. Every sentence contributes directly to explaining the approval behavior.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need not be explained. But with no annotations, three parameters, and 0% schema description coverage, the description is too sparse about parameter meaning, approval side effects, and alternative tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and all three parameters lack schema descriptions. The description loosely implies 'who approved it' maps to approved_by, but it does not explain feature or action_ids, leaving most parameter meaning undocumented.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb, 'approve', and identifies the resource as byte-exact observed output required by behavior. It also scopes the tool to standard mode and mentions default help/usage/version output, but it does not explicitly distinguish this approval action from related siblings such as add_quote or propose_claim.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'Standard mode only' gives a usage condition, and the parenthetical default identifies common applicable output types. However, the description does not name alternatives or explain when not to use this tool compared with related quote/claim/spec tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
audit_verifyB
Verify the audit log's hash chain.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full behavioral-disclosure burden. It does not explain what a failed verification looks like, whether it mutates state, whether special authorization is required, or how the hash chain is traversed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no filler. Every word contributes to the stated purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter verification tool with an output schema, the description is nearly complete: it identifies the object and action. The remaining gap is behavioral context around failure and security implications, which is not covered by annotations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters and a fully described empty schema, so there are no parameter semantics to explain. A baseline of 4 is appropriate because the description is not expected to add parameter detail for a no-argument tool.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: verify the audit log's hash chain. It is clear what the tool does, though it does not differentiate itself from any sibling because no sibling appears to overlap directly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool, when not to use it, or what preconditions must hold before verification. The agent must infer that this is a read-only integrity check from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
blocked_hostsB
Web: third-party hosts the app tried to load during one episode (reset + actions); candidates for target_options.profile.asset_hosts when layout depends on them.
| Name | Required | Description | Default |
|---|---|---|---|
| actions | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses the temporal scope of one episode (reset + actions) and implies an observational read, but it does not explicitly state side effects, permissions, or whether the operation is read-only.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single dense sentence that front-loads the core resource and includes the key usage condition. It is efficient, though the semicolon-heavy structure could be slightly easier to parse if split.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need not be described. However, with no annotations and zero schema description coverage, the description leaves the actions parameter unexplained and does not fully address behavioral traits such as read-only status.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the sole optional parameter, actions, is not explained. The description mentions 'actions' only as part of the episode context, not as a parameter with type, format, or default behavior.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the resource as third-party hosts the app tried to load during a web episode, which is more specific than the bare tool name. It does not explicitly distinguish this from sibling web-probing tools, so it earns a 4 rather than a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives a clear usage condition: these hosts are candidates for target_options.profile.asset_hosts when layout depends on them. It does not mention when not to use the tool or name alternative sibling tools, so it falls short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
check_rulesC
Check observer-authored rules against the original. Each rule: {statement (your own words), origin: documented|inferred-intent|observed, kind, checks: [{kind: outcome|same|differs, a: invocation, b: invocation (same/differs), expect: subset (outcome), keys: [..] (optional)}]}. Falsified documented rules are reported as documentation/behavior conflicts.
| Name | Required | Description | Default |
|---|---|---|---|
| rules | Yes | ||
| feature | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description must carry the full burden of behavioral disclosure. It mentions that falsified documented rules are reported as documentation/behavior conflicts, which is a useful behavioral trait, but it omits critical details like whether the tool mutates state, what permissions are required, or how it handles non-documented rules.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the purpose and then efficiently describes the rule format. It uses compact notation and avoids unnecessary words, though the dense structure may be slightly cryptic for quick parsing.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no annotations, an output schema, and complex rules parameter, the description is partially complete. It thoroughly covers the rules parameter but fails to explain the 'feature' parameter or clarify what 'original' refers to, which are critical for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It provides a detailed explanation of the 'rules' parameter, including the structure and allowed values for each rule, which adds significant meaning beyond the schema. However, it completely ignores the required 'feature' parameter, leaving a substantial gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description begins with 'Check observer-authored rules against the original,' which states a verb and resource, but 'original' is undefined and the tool is not clearly distinguished from siblings like falsify_claim or list_rules. The subsequent rule format details add specificity but don't fully clarify the tool's core purpose relative to its ecosystem.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus alternatives such as falsify_claim or list_rules. The description only explains the rule structure, leaving the agent to infer usage from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
compile_specC
Compile a spec draft (model + claims + unknowns) for a feature.
| Name | Required | Description | Default |
|---|---|---|---|
| feature | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full behavioral burden and says nothing about side effects — whether a compile writes or overwrites a draft, is idempotent, or requires prior claims/unknowns to exist. Only the existence of an output schema saves the agent from total blindness about results.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One short sentence, front-loaded with the verb and the output artifact. No filler. The brevity is only a problem because nothing else compensates, not because the sentence is padded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with zero annotation coverage and no parameter documentation, the description should explain the lifecycle position and side effects. The output schema covers return values, but nothing tells the agent what must exist beforehand or what this changes.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% for the single 'feature' parameter. The phrase 'for a feature' weakly signals that the argument is a feature identifier, but gives no format, expected naming, or whether it must match an existing feature record.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The verb 'compile' and resource 'spec draft' are named, and the parenthetical 'model + claims + unknowns' hints at what gets produced. But 'compile' is jargon — it's unclear whether this assembles, validates, or generates — and there's no explicit contrast with siblings like review_spec or release_spec that sit in the same lifecycle.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no when-to-use guidance, no prerequisites, and no mention of alternatives. An agent can infer from sibling names that a propose → compile → review → release pipeline may exist, but the description itself offers nothing to route on.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
coverageC
Coverage of a feature inventory {item: [patterns]} by probes and rule checks (options are added automatically). Uncovered items become unknowns.
| Name | Required | Description | Default |
|---|---|---|---|
| feature | Yes | ||
| inventory | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must carry the behavioral burden. It usefully discloses a side effect (uncovered items become unknowns) and that options are added automatically, but it omits permissions, idempotency, whether existing unknowns are updated vs created, and other mutation semantics.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences keep it front-loaded and efficient. The parenthetical 'options are added automatically' is somewhat cryptic but not verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema reduces the need to describe return values, but for a tool with a mutation side effect and no annotations, the description is still incomplete: the 'feature' parameter is undefined, null inventory behavior is unexplained, and the side effect is only partially described.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description should explain both parameters. It clarifies the inventory shape as {item: [patterns]}, but never defines the required 'feature' parameter or explains what a null inventory means.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description conveys that this tool computes coverage of a feature inventory using probes and rule checks, and that uncovered items become unknowns. However, it is phrased as a noun phrase rather than a clear verb+resource, and it overlaps conceptually with siblings like check_rules and list_feature_unknowns without explicitly differentiating itself.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It offers no explicit when-to-use or when-not-to-use guidance and does not name alternatives. The note that uncovered items become unknowns describes an outcome, not a selection criterion.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
define_actionsB
Define named actions (CLI invocations: argv, stdin, files, env, stdin_mode, stdout_mode, timeout_s) for learning flows of stateful CLIs.
| Name | Required | Description | Default |
|---|---|---|---|
| actions | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It lists the fields of an action but does not state whether defining actions persists them, whether existing actions are overwritten, what permissions are needed, or how errors are handled. This is a significant gap for a mutation-style definition tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single tightly written sentence that front-loads the core purpose and follows with the relevant field list. There is no filler or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need not be explained. However, the tool accepts a complex nested object with arbitrary properties and no annotations, and the description does not clarify the mapping structure, required sub-fields, or the semantics of fields such as stdin_mode and stdout_mode. It is minimally adequate but leaves important details to inference.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It enumerates the per-action fields (argv, stdin, files, env, stdin_mode, stdout_mode, timeout_s), which is meaningful semantic information absent from the schema. It still does not explain value formats or required sub-fields, but it substantially improves understanding of the nested parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb and resource: "Define named actions" for "learning flows of stateful CLIs." It does not explicitly distinguish itself from sibling tools like learn_flow or run_sequence, but the stated purpose is clear enough for an agent to understand what the tool does.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives the context ("for learning flows of stateful CLIs") but offers no explicit guidance on when to use this tool versus alternatives like run_sequence, learn_flow, or discover_cli. There are no exclusions, prerequisites, or alternative-routing instructions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
discover_cliB
Discover a CLI's interface: help structure confirmed by behavior probes. Help prose is kept as dirty-side evidence only; the result is structure (options, arity, value types).
| Name | Required | Description | Default |
|---|---|---|---|
| cmd | Yes | ||
| takes_files | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description does add real behavioral context: help prose is downgraded to 'dirty-side evidence' and only confirmed structure is returned, so the agent knows not to trust raw help text. It omits important traits for a tool that must execute a CLI to probe it — whether the command is actually run, side effects, safety, or any permission/auth requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences with the purpose front-loaded and no padding. 'Dirty-side evidence' is slightly jargon-y but earns its place by signaling the trust level of help output.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return structure need not be spelled out, and the description correctly frames the result. But for a tool that executes an external binary, missing execution semantics and an unexplained second parameter leave meaningful gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must carry the parameter burden and does not. 'cmd' is self-evident, but 'takes_files' (boolean, default true) is never explained — the agent cannot tell whether it declares that the target CLI accepts file arguments or whether the tool itself should pass files.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Discover a CLI's interface') and even describes the shape of the result (options, arity, value types), which is more than most definitions offer. It does not, however, distinguish itself from sibling probe_cli or discover_ui, leaving the agent to infer which discovery tool to pick.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is only implied: the phrase 'help structure confirmed by behavior probes' hints that this is a discovery step feeding later probing, but there is no explicit when-to-use, when-not-to-use, or named alternative among siblings like probe_cli or discover_ui.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
discover_uiA
UI targets: crawl from the start screen (or after the named start actions, e.g. a sign-in) clicking every enabled control; screens are identified by route, headings and controls. Defines a click[role:name] action for every control found and records the inventory that coverage and suggest_probes use. Start here, then probe fields and state rules.
| Name | Required | Description | Default |
|---|---|---|---|
| fill | No | ||
| start | No | ||
| feature | Yes | ||
| max_depth | No | ||
| max_states | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the load and does add real context: it discloses that it clicks every enabled control, how screens are identified, and that it persists an inventory consumed by later tools. However it omits material behavior for a stateful crawler — that clicking live controls can mutate application state, that a target session must already exist, and that runs are bounded by state/depth caps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two dense sentences with the purpose front-loaded and the sequencing instruction at the end; every clause carries information. The middle sentence runs long with several comma-separated behaviors, which slightly hurts scannability.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The output schema removes the need to describe return values, and the crawl concept is conveyed. But for a five-parameter, potentially state-mutating crawl with zero annotation coverage, leaving four parameters and the session/prerequisite situation unexplained leaves the definition only partially complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% across five parameters, so the schema offers no help, and the description only explains `start` (named actions such as a sign-in). The meaning and effect of `fill`, `feature`, `max_depth` and `max_states` — especially the crawl bounds — are left entirely to inference.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource — crawl the UI from the start screen, clicking every enabled control — and defines what it produces (a click[role:name] action per control plus an inventory). This clearly separates it from discover_cli and probe_web_field, and it names the downstream consumers (coverage, suggest_probes).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
"Start here, then probe fields and state rules" gives explicit ordering relative to the probe tools and establishes the entry-point role. It stops short of naming exclusions (e.g. when to choose discover_cli or open_target first), so not a full when/when-not treatment.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
falsify_claimC
Try to falsify a claim with fresh, uncached probes.
| Name | Required | Description | Default |
|---|---|---|---|
| repeats | No | ||
| claim_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full disclosure burden. It does reveal that probes are fresh and uncached (likely meaning repeated live execution against the target, with cost and non-determinism), but it never states side effects, whether the claim's state changes, what happens on inconclusive results, or any rate/permission constraints for a tool that appears to run live probes.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One tight sentence with no filler and the core action front-loaded. It is efficient, though the terseness is part of why key semantics are missing rather than a model of well-structured brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Placeholder — see justification correction below.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate and does not. Neither claim_id nor repeats is explained; in particular the meaning of repeats (default 3) — how many probe attempts per falsification run, and whether they must all fail to falsify — is left entirely opaque.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('falsify a claim') and adds a mechanism note ('fresh, uncached probes'), which distinguishes it from propose_claim and list_claims. However, it does not explicitly contrast with adjacent tools like suggest_probes or check_rules, so differentiation is only partial.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no statement of when to prefer this over suggest_probes, check_rules, or audit_verify, and no preconditions (e.g. the claim must exist and be in a falsifiable state). The only usage signal is the implied 'use this to test a claim', which an agent must infer.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
ingest_docsB
Register public documentation you read (dirty side). Its paragraphs join the gate corpus so copied wording is flagged; write rules in your own words.
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes | ||
| source | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It usefully discloses the key side effect: paragraphs join the gate corpus and copied wording is flagged. It leaves out operational details such as whether ingestion is idempotent, what qualifies as public, and whether entries can be removed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The definition is a single compact sentence with no wasted language. The core action and intended effect are front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The presence of an output schema reduces the need to explain return values. Still, for an ingestion/mutation tool with no annotations and undocumented parameters, the description is only minimally complete: it explains the corpus effect but not enough about accepted inputs or operational constraints.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must explain the two required parameters. It only gives vague context: source and text are not defined, and it does not say whether source is a URL, citation, or label, nor what text should contain.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description gives a specific verb-resource pair: registering public documentation and adding its paragraphs to the gate corpus. It clearly explains the copy-detection purpose, though it does not explicitly differentiate this tool from nearby claim, quote, or rule tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It implies the right usage context: register documentation you have read so copied wording is flagged, and write rules in your own words. However, it gives no explicit when-not-to-use guidance, prerequisites, or named alternative tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
learn_flowC
Learn a state machine for a feature with KV or L# under an explicit oracle budget.
| Name | Required | Description | Default |
|---|---|---|---|
| feature | Yes | ||
| alphabet | Yes | ||
| walk_len | No | ||
| algorithm | No | kv | |
| walks_per_state | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full behavioral burden and falls well short: it doesn't say whether this mutates the rule/claim store, how long a learn run takes, what 'oracle budget' means operationally, or what happens when the budget is exhausted. The one hint of cost-awareness is not actionable.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
It is a single front-loaded sentence with no padding, which is structurally sound. However, the space that is used is spent on unexplained abbreviations and a budget concept that maps to no input, so brevity comes at the cost of usefulness rather than from efficiency.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values needn't be described, but for a 5-parameter, no-annotation, presumably long-running learning operation the description leaves out mutation side effects, budget semantics, algorithm choices, and parameter meanings. It is not complete enough for an agent to invoke this correctly on the first attempt.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% across 5 parameters, so the description must compensate and does not: 'feature' is only implicitly referenced, 'KV or L#' loosely maps to the 'algorithm' parameter without naming it or listing valid values, and 'alphabet', 'walk_len', and 'walks_per_state' are entirely unaddressed — including the required 'alphabet' array.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The verb+resource ('learn a state machine for a feature') is identifiable, but the qualifiers 'KV or L#' and 'explicit oracle budget' are unexplained jargon and the budget concept has no corresponding parameter. An agent can guess the general purpose but cannot confidently predict what the tool will do with its inputs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no statement of when to use this versus the many sibling tools (check_rules, coverage, discover_ui, suggest_probes, etc.), nor any prerequisite or follow-up guidance. The phrase 'under an explicit oracle budget' hints at a cost constraint but never says when that constraint applies or how to satisfy it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_claimsC
List claims, optionally for one feature.
| Name | Required | Description | Default |
|---|---|---|---|
| feature | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. "List" implies a read, but it says nothing about volume, pagination, ordering, or what happens when the supplied feature does not exist (empty result vs error). For an unannotated tool this is a substantial disclosure gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One short sentence, front-loaded with the verb and resource, with zero filler. Brevity is appropriate, though it is terse to the point of omitting behavior rather than being padded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists so return shape needn't be described, but with no annotations, an undocumented filter parameter, and no statement of scoping or failure behavior, the definition is too thin for an agent to call it confidently.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With a single parameter at 0% schema description coverage, the description must compensate; it does state that feature is optional and acts as a filter, which is the main semantic. It stops short of saying whether the value is a feature name/ID or how unknown or non-matching features are handled.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The verb+resource pair ("List claims") is specific, and the optional feature scope is stated, but the domain term "claim" is never explained and nothing distinguishes this from siblings like list_rules or list_feature_unknowns. An agent can guess it enumerates claim objects but not what a claim is in this system.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
"optionally for one feature" hints at a filtering use case but gives no when-to-use/when-not guidance and never names an alternative (e.g., when to inspect claims via list_rules or a spec tool instead). The agent must infer selection criteria entirely.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_feature_unknownsB
List unknowns (yours and the ones coverage generated) with id, question and status, so they can be answered with update_unknown.
| Name | Required | Description | Default |
|---|---|---|---|
| feature | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must carry behavioral disclosure itself. "List" implies a read and the mention of returned fields (id, question, status) and the two provenance sources add useful context, but nothing is said about the optional feature filter's scoping effect, ordering, volume, or permissions required.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with no filler; the verb and scope lead. The parenthetical about provenance is slightly dense but earns its place by clarifying that coverage-generated entries are included.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so field-level return detail is redundant (though harmless), and the follow-up action is covered. The gap is the undocumented `feature` parameter combined with zero annotations, meaning the description says nothing about filtering behavior or result scope.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% for the single `feature` parameter, and the description never mentions it. This leaves an unlabeled optional filter whose semantics — whether it narrows results to one feature or defaults to all — must be guessed from the parameter name alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ("List unknowns") and scopes it ("yours and the ones coverage generated"), which separates it from record_unknown and update_unknown siblings. It does not explicitly contrast with list_claims or list_rules, but the resource noun is distinct enough that selection is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives a clear workflow cue: these unknowns exist "so they can be answered with update_unknown," routing the agent to the correct follow-up tool. It does not state when not to use this tool or what the list omits, so it falls short of full when/when-not guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_rulesB
List the rules recorded for a feature: id, statement, status, failed checks.
| Name | Required | Description | Default |
|---|---|---|---|
| feature | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the burden. 'List' implies a read-only operation and it discloses the returned fields (id, statement, status, failed checks), which is useful, but it says nothing about authorization, pagination, or whether the read is recursive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single tight sentence, front-loaded with the verb and resource, with the returned fields appended. Every clause is informative.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists so return-value detail is largely covered, and the tool is a simple one-parameter read. However, with no annotations the description should do more to establish the read-only nature and the meaning/format of the feature argument.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% and the single 'feature' parameter has no schema description. The description clarifies that 'feature' scopes the query, which compensates somewhat, but gives no format or example for the identifier.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (List) and resource (rules) scoped to a feature, and names the returned fields. It does not differentiate from siblings like list_claims or the related check_rules/retract_rule, which prevents a top score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no explicit when-to-use or when-not guidance, and no mention of alternatives such as check_rules (which presumably validates rules) or retract_rule. Usage is only implied by 'recorded for a feature'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_ticketsC
List questions from the clean side.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full behavioral burden and delivers almost nothing: no pagination, ordering, filtering, or scope boundaries are stated. The phrase "clean side" gestures at a scoping concept but never defines it, so it adds ambiguity rather than transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
It is a single short sentence with no filler, so length is not the problem. However, the brevity comes from under-specification rather than economy — the one clause present is the opaque "clean side" phrase, which does not front-load actionable information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need not be described. Still, for a listing tool in a suite with list_claims, list_rules, and answer_ticket siblings, the agent needs to know what set is returned, how it is ordered, and how it relates to answering — none of which is present, and "clean side" is never defined.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, so there is no parameter semantics to document and the baseline is 4. Nothing in the schema is left unexplained by the description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
"List questions" restates the name (tickets ≈ questions) without distinguishing it from siblings like list_claims or list_rules, and the qualifier "from the clean side" is unexplained jargon rather than a scope definition. An agent cannot tell what a "clean side" is or whether it wants these results.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no when-to-use guidance, no mention of the apparently related answer_ticket sibling, and no stated preconditions. The only implied usage is that listed items can presumably be answered, which the agent must infer from the sibling list rather than the description.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
open_targetB
Open a target and record intake (declared purpose, jurisdiction, ToS snapshot). target: cli:docker: | cli:build: | cli:local | web: | android:: | desktop:docker:. strict: every release needs a person's approval at a terminal. target_options (web): reset_command, reset_check_url, contract_selectors {source, selectors}, api_templates, profile {asset_hosts, ...}; see the observer guide.
| Name | Required | Description | Default |
|---|---|---|---|
| strict | No | ||
| target | Yes | ||
| account | No | ||
| purpose | Yes | ||
| real_names | No | ||
| terms_text | No | ||
| jurisdiction | No | ||
| target_options | No | ||
| owner_declaration | No | ||
| max_actions_per_second | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full behavioral burden. It usefully discloses that intake records purpose, jurisdiction, and ToS, and that strict mode requires human approval at a terminal, but it omits what side effects occur, whether state persists, and how failures are surfaced for a stateful open operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the main action in the first sentence, followed by target formats and then strict/option details. It is dense but each sentence carries operational information; minor ellipses in the target_options profile list reduce clarity slightly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need not be explained. However, for a 10-parameter, annotation-free tool with 0% schema coverage, the description leaves several parameters and nested target_options details undefined, relying on an external observer guide.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% across 10 parameters, so the description must compensate and only partially does. It documents target formats, strict, and some target_options keys, but purpose, jurisdiction, terms_text, owner_declaration, account, real_names, and especially max_actions_per_second receive little or no explanation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a clear verb and resource: 'Open a target and record intake (declared purpose, jurisdiction, ToS snapshot).' The enumerated target forms (cli:docker, web:, android:, desktop:) make the operation concrete. It does not explicitly contrast with siblings like discover_ui or probe_cli, but the scope is distinct enough.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied as the first step before other target-scoped tools, but the description never states when to use this versus alternatives such as discover_ui, probe_cli, or learn_flow. There are no explicit exclusions or prerequisites beyond the strict-mode note.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
probe_cliC
Probe a CLI feature: one-factor sweep, pairwise over accepted classes, noise calibration (rerun + env variations), probe table, mined decision-table claims and unknowns. hints = {cmd, base, selector_options, takes_files, default_input, sweep_inputs, inputs, pair_inputs, operand_type, unicode_argv, skip_options, extra_inputs, extra_values}.
| Name | Required | Description | Default |
|---|---|---|---|
| hints | Yes | ||
| feature | Yes | ||
| pairwise | No | ||
| env_sample | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full behavioral burden. It usefully discloses the internal process (sweeps, pairwise combinations, noise calibration via rerun and env variations), but it does not state whether CLI commands are executed, what side effects or permissions are involved, or any safety constraints. The output schema covers return values, so that gap is less serious.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the tool's purpose and then enumerates key processes. It is dense but not wasteful, and the hints key list, while long, is necessary to partially compensate for the schema's lack of descriptions.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
This is a complex probing tool with a nested hints object and no annotations. While an output schema exists, the description leaves too much unspecified about parameter semantics, execution behavior, and safety for an agent to invoke it confidently without inspecting the implementation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and there are four parameters including a nested hints object with many keys. The description lists the hints keys, which is helpful, but provides no meaning, type, constraints, or examples for any of them, and top-level parameters like feature, pairwise, and env_sample are only obliquely referenced.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: probing a CLI feature. It enumerates the probing modes (one-factor sweep, pairwise, noise calibration, probe table, mined claims/unknowns), so the purpose is clear, but it does not explicitly distinguish this tool from siblings like discover_cli or suggest_probes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no explicit guidance on when to use this tool versus alternatives such as suggest_probes, discover_cli, or probe_web_field. The only usage signal is the implied context of probing a CLI feature, with no stated prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
probe_web_fieldB
Probe one form field (web, Android or desktop UI): equivalence classes for its kind (text/email/password/number or custom classes), a length-boundary search, per-class outcomes and folded claims. prefix = named actions that reach the form; submit = the named action that submits it. Boundary classes become named actions you can add to learn_flow's alphabet. When accepted values change the outcome (computed results), say what acceptance looks like: accepted_if = {subset: {...}} and/or {text_contains: '...'}. Number fields skip the length search (their limits are ranges). Without submit, the outcome is read right after typing (live previews, totals, counters).
| Name | Required | Description | Default |
|---|---|---|---|
| kind | No | text | |
| name | Yes | ||
| role | Yes | ||
| valid | No | ||
| prefix | Yes | ||
| submit | No | ||
| classes | No | ||
| feature | Yes | ||
| boundary | No | ||
| accepted_if | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and does disclose meaningful mechanics: boundary classes become named actions, submit controls whether an action is invoked, and number fields skip length probing. It does not disclose whether submitted forms can mutate state, what permissions are required, or any safety/rate-limit considerations, which is a notable omission for a UI-probing tool that may submit forms.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and front-loads the core operation, but it is dense with domain jargon such as 'folded claims' and 'learn_flow's alphabet,' making it harder to parse. The content is not bloated, but the structure could be clearer for an agent unfamiliar with the project vocabulary.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
This is a complex 10-parameter tool with no annotations and no schema descriptions, so the description needs to be more complete. It covers several key parameters and behaviors, but required parameters and safety implications of submitting forms remain unexplained; the output schema does at least cover return values.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% across 10 parameters, so the description must compensate. It explains `prefix`, `submit`, `classes`, `kind`, and `accepted_if`, but leaves required parameters like `feature`, `role`, and `name` plus `valid` and `boundary` unexplained, forcing the agent to guess their meaning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description gives a specific verb and resource: 'Probe one form field (web, Android or desktop UI)' and states the core outputs (equivalence classes, boundary search, per-class outcomes). It is clear what the tool does, though it does not explicitly differentiate itself from siblings such as probe_cli or discover_ui.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides useful operational conditionals: number fields skip the length search, and without a `submit` action the outcome is read right after typing. However, it never states when to choose this tool over alternatives like probe_cli or discover_ui, so usage is implied rather than explicitly guided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
propose_claimC
State a falsifiable claim as a trace check (expect: symbol per step, null = any).
| Name | Required | Description | Default |
|---|---|---|---|
| kind | Yes | ||
| layer | No | unknown | |
| expect | Yes | ||
| inputs | Yes | ||
| origin | No | observed | |
| feature | Yes | ||
| statement | Yes | ||
| strictness | No | must |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It does not disclose whether proposing persists a claim, whether it is validated against rules, or what happens afterward (though an output schema exists, the mutation/profile behavior is unstated). The only behavioral hint is the 'null = any' note on expect.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
It is a single front-loaded sentence with no waste, but for a complex 8-parameter tool it is too terse, pushing semantics into a dense parenthetical rather than structuring useful guidance.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 8 parameters, 0% schema description coverage, and no annotations, the definition is far from complete. An output schema reduces the need to explain returns, but the input semantics and behavioral profile are largely missing for a tool this complex.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% across 8 parameters (5 required), and the description explains only 'expect' (symbol per step, null = any). It says nothing about kind, feature, statement, inputs, layer, origin, or strictness, leaving the majority of parameters undocumented.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a verb ('State') and a resource ('a falsifiable claim'), plus a parenthetical about 'expect'. However, it does not clarify what a 'claim' or 'trace check' actually is, and offers no differentiation from the sibling falsify_claim (the natural counterpart to a propose). An agent can guess the general idea but not the precise purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no indication of when to use this versus falsify_claim or list_claims, no prerequisites, and no exclusions. The agent is left to infer the workflow entirely.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
record_unknownC
Record something that could not be observed (missing coverage is unknown, not absence).
| Name | Required | Description | Default |
|---|---|---|---|
| feature | Yes | ||
| question | Yes | ||
| recommended_probes | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the behavioral burden. It explains the key domain rule that missing coverage is unknown, not absence, but says nothing about side effects, permissions, idempotency, or what recording changes in the system.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single front-loaded sentence with no wasted words, and the parenthetical earns its place by clarifying the domain semantics. It is appropriately concise, though its brevity contributes to the gaps elsewhere.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, reducing the need to explain return values, but the tool has three undocumented parameters and no annotations. The description adds useful conceptual framing but leaves required parameter meaning and behavioral details incomplete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not mention any of the three parameters. The required feature and question fields and the optional recommended_probes field are left completely unexplained.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action and domain concept: recording something that could not be observed, with the parenthetical clarifying that missing coverage means unknown rather than absence. It is clearer than a tautology, but it does not explicitly differentiate itself from siblings such as update_unknown or list_feature_unknowns.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus alternatives, no exclusions, and no mention of prerequisites. The parenthetical explains a semantic distinction, but it does not route the agent among the many sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
release_specA
Release a draft through the gate. Strict mode: succeeds only once a person has approved every finding and acknowledged coverage gaps (teardown approve); rewrite prose findings instead of waiting for approval. Standard mode: refused while measured coverage items are neither exercised nor explained, unless accept_gaps gives the reason they ship as unknowns.
| Name | Required | Description | Default |
|---|---|---|---|
| draft_id | Yes | ||
| accept_gaps | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description carries the behavioral burden: it discloses strict mode requirements (approval of every finding, acknowledged coverage gaps), the alternative of rewriting prose findings, and standard mode refusal conditions. It does not cover permissions or reversibility, but provides rich gating behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the core action ('Release a draft through the gate') followed by mode-specific details. Dense but every sentence adds behavioral context; no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given an output schema exists and no annotations are present, the description covers the key gating behaviors and modes. It omits what 'gate' means operationally, but overall provides enough for an agent to call correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. It explains accept_gaps meaningfully ('gives the reason they ship as unknowns'), but does not explain draft_id beyond its self-evident name. Adequate but not fully compensating.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Release') and resource ('a draft'), and explains the gating conditions. It distinguishes itself from siblings like approve_observed_output by focusing on releasing after approvals, though it does not explicitly name alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explains when the tool succeeds in strict mode and when it is refused in standard mode, and mentions accept_gaps as a condition. It gives clear context for use but does not explicitly compare to siblings or state when to prefer this over review_spec.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
retract_ruleB
Retract one of your rules (e.g. superseded by a reworded version), with a reason. Rule ids are in check_rules results.
| Name | Required | Description | Default |
|---|---|---|---|
| reason | Yes | ||
| feature | Yes | ||
| rule_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and this is a mutation operation, so the description carries the full burden. It mentions requiring a reason and where to find rule IDs, but does not disclose whether retraction is permanent, what side effects occur, what permissions are needed, or what happens to dependent rules.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two tight sentences with no wasted text. The core action is front-loaded, and the example and rule-ID hint are useful without bloating the definition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Output schema exists, so return values need not be explained. But for a no-annotation mutation tool with three required parameters and zero schema descriptions, the definition is incomplete: the feature parameter is unexplained and behavioral consequences of retraction are missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must explain all three parameters. It clarifies rule_id by saying IDs are in check_rules results and echoes that reason is required, but it gives no meaning at all for the feature parameter and adds little format detail for reason.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb and resource: retract a rule. It also gives a concrete example of when a rule might be retracted and points to check_rules for rule IDs. It is clear, though it does not explicitly distinguish itself from list_rules.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives one implied usage scenario: a rule superseded by a reworded version. It also says rule IDs come from check_rules results. However, it does not state when not to use this tool or name alternatives such as list_rules.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
review_specB
Run the gate on a draft without releasing it: findings grouped by class (expected-output / interface / prose), which are approved, and prose findings in full.
| Name | Required | Description | Default |
|---|---|---|---|
| draft_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are supplied, so the description carries the full behavioral burden. It does disclose the output shape (findings grouped by class, approval status, full prose findings) and implies a non-mutating dry run, which is genuinely useful. However, it never explicitly states that no state changes occur, whether a review record is persisted, what errors look like for an invalid draft_id, or any permission requirements — gaps that matter for a gate tool with zero annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with no filler, leading with the action and immediately qualifying it with the key constraint. It is dense with domain jargon ('the gate', finding classes), but nothing in it is wasted.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so the description need not explain return values, and the one-parameter surface keeps the tool simple. Still, 'the gate' is a domain concept left undefined, and the failure/error path for an invalid draft is unaddressed. Adequate minimum for a low-arity tool, but not thorough.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate for the single draft_id parameter — and it does not. 'On a draft' only loosely gestures at draft_id without clarifying whether it is a spec ID, version, or handle, or whether an unknown ID triggers an error. The description adds essentially no meaning beyond the bare string field.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource ('Run the gate on a draft') plus a scope qualifier ('without releasing it') that implicitly separates it from release_spec. An agent can tell this is a validation/dry-run tool, though the sibling that performs the actual release is never named, so differentiation is inferential rather than explicit.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'without releasing it' implies this is the pre-release check, which is real usage guidance against release_spec. But it never says when the gate must be run, what happens if the gate fails, or how this differs from other validation-adjacent siblings like check_rules, coverage, or audit_verify. Usage is left to be inferred.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
run_sequenceB
Reset the target, apply the inputs, and return raw observations and symbols. symbols_only drops the raw observations; keys (e.g. ["path", "status", "alerts"]) keeps only those parts of each symbol.
| Name | Required | Description | Default |
|---|---|---|---|
| keys | No | ||
| inputs | Yes | ||
| symbols_only | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden, and it does disclose the important side effect that the target is reset and that output has two layers (raw observations plus symbols). It does not say whether the reset is destructive to prior state, what happens on invalid inputs, or what 'symbols' actually contain, leaving real gaps for a state-mutating tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences, front-loaded with the core action, then optional-parameter modifiers. Every clause carries information, though the second sentence packs two modifiers together without separating them visually.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need not be described, which this description respects. However, for a tool that resets state and takes an undocumented required string array, the definition is missing the workflow context (target prerequisite, input format) needed to call it confidently.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It explains symbols_only and keys well, including a concrete example of key names, but the required 'inputs' parameter is never explained — an agent cannot tell whether these are action strings, selectors, or commands.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description gives specific verbs and a resource flow: reset the target, apply inputs, return raw observations and symbols. An agent knows it executes a sequence of inputs against an already-open target. It stops short of explicitly naming which sibling tools feed it or overlap with it (e.g., discover_ui, probe_web_field).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no statement of when to use this versus siblings like discover_ui, suggest_probes, or probe_web_field. No prerequisites are given either, even though 'reset the target' implies a target must first be opened (open_target) and inputs must come from somewhere.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
set_mapperC
Replace the mapper rules (drop/mask/rename/classify). Creates a new mapper version.
| Name | Required | Description | Default |
|---|---|---|---|
| note | No | ||
| rules | No | ||
| params | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description bears the full behavioral burden. It does disclose one valuable trait — that a new mapper version is created rather than rules being edited in place — but it omits the destructive semantics of "replace" (what existing rules are lost), permission requirements, and whether prior versions remain recoverable. For a mutation with zero annotation coverage that is a substantial gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two compact sentences with the action and the resulting side effect front-loaded; every clause carries information and there is no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
A mutation tool with no annotations, 0% parameter coverage, and no sibling differentiation needs more than this. An output schema exists so return values need not be explained, but the missing guidance on required inputs, destructive scope, and when to prefer a targeted alternative leaves the definition incomplete for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% across three parameters. The description implies the content of `rules` via the drop/mask/rename/classify list, but says nothing about the `note` field or the `params` object, nor about the structure of individual rule entries. With all defaults (nothing required), an agent has no guidance on how much it must supply.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific verb ("Replace") and resource ("mapper rules") and enumerates the rule kinds it covers (drop/mask/rename/classify), so the agent knows what is being written. It does not, however, position itself against nearby siblings such as list_rules, check_rules, or retract_rule, so fine-grained discrimination is left to inference.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no statement of when this tool should be used versus list_rules/check_rules/retract_rule, and no prerequisites (e.g., must rules be validated first, does this require a spec context). The word "Replace" only implies a write path; nothing tells the agent when a full replacement is preferable to a targeted retract_rule.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
suggest_probesB
Concrete next steps for the feature's coverage gaps: fault actions with a path that makes the call, probe_web_field arguments for unprobed fields, click paths for unexercised controls. Run after discover_ui and after each batch of rules.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | ||
| feature | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations, so the description carries the burden. It does disclose the shape of the output (fault actions with a call-making path, probe_web_field argument sets, click paths), which tells the agent these are suggestions rather than executed probes. It does not state read-only behavior, side effects, cost, or whether suggestions are deduplicated against existing rules.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One dense sentence of scope plus one short sentence of sequencing, with no filler. Front-loaded with what the tool returns, which is the right ordering. Slightly list-heavy phrasing but every clause adds information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need not be re-explained, and the description still usefully summarizes them. However, for a two-param tool with zero schema documentation, the missing meaning of 'feature' and 'limit' is a substantive gap an agent must guess at.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so both parameters are undocumented anywhere. The description only implies that 'feature' scopes the gaps; it never says what form a feature identifier takes and never mentions 'limit' (default 30) at all, leaving half the surface unexplained.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific action and resource: generating concrete next-step probes that target a feature's coverage gaps. The enumeration of three probe flavors (fault actions, probe_web_field arguments, click paths) makes the capability concrete and distinguishable from the sibling 'coverage' tool, though it never explicitly contrasts with it.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives explicit sequencing: run after discover_ui and after each batch of rules. That is a real when-to-use instruction an agent can act on. It stops short of saying when NOT to use it or naming the alternative (e.g., plain 'coverage') for a purely informational read.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
update_unknownB
Mark an unknown resolved / out-of-scope / investigating (or reopen it) with a note. Resolved unknowns leave the release; the audit log keeps the history.
| Name | Required | Description | Default |
|---|---|---|---|
| note | No | ||
| status | Yes | ||
| unknown_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden, and it does disclose two real behavioral traits: resolved unknowns leave the release, and the audit log retains history. It omits whether reopening restores the unknown to the release, whether the operation is idempotent, and any permission requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, both earning their place: the first states the action and states, the second states the consequences. Front-loaded and zero waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values needn't be explained, and the side effects on release/audit are covered. However, for a required status parameter with no enum and no schema descriptions, the missing enumeration of valid status values is a meaningful gap for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must define all three parameters. It mentions 'a note' and hints at status values, but never states which exact strings status accepts — 'reopen' reads as an action here, yet status is a required free-form string, leaving genuine ambiguity about valid values.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb (mark/reopen) plus the concrete resource (an unknown) and names the resulting states (resolved / out-of-scope / investigating). It is clearly distinguishable from record_unknown or list_feature_unknowns, though it never names a sibling explicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It implies the right context — resolving, scoping out, or reopening an unknown — but never states when to use this versus record_unknown (create) or who is allowed to do it. The guidance is inferable rather than explicit, with no exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
3 tool updates
v0.2.0- Changed
probe_web_field4 fields changed- added
Input schema / properties / submit / anyOfAdded value: +[ + { + "type": "string" + }, + { + "type": "null" + } +] - added
Input schema / properties / submit / defaultAdded value: +null - removed
Input schema / properties / submit / typeRemoved value: -"string" - changed
Input schema / requiredPrevious value: -[ - "feature", - "prefix", - "role", - "name", - "submit" -]New value: +[ + "feature", + "prefix", + "role", + "name" +]
- Changed
release_spec1 field changed- added
Input schema / properties / accept_gapsAdded value: +{ + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Accept Gaps" +}
- Changed
run_sequence1 field changed- added
Input schema / properties / keysAdded value: +{ + "anyOf": [ + { + "items": { + "type": "string" + }, + "type": "array" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Keys" +}
31 tool updates
v0.1.0- First observed
add_quote - First observed
answer_ticket - First observed
api_contract - First observed
approve_observed_output - First observed
audit_verify - First observed
blocked_hosts - First observed
check_rules - First observed
compile_spec - First observed
coverage - First observed
define_actions - First observed
discover_cli - First observed
discover_ui - First observed
falsify_claim - First observed
ingest_docs - First observed
learn_flow - First observed
list_claims - First observed
list_feature_unknowns - First observed
list_rules - First observed
list_tickets - First observed
open_target - First observed
probe_cli - First observed
probe_web_field - First observed
propose_claim - First observed
record_unknown - First observed
release_spec - First observed
retract_rule - First observed
review_spec - First observed
run_sequence - First observed
set_mapper - First observed
suggest_probes - First observed
update_unknown
TDQS
Scored across 31 tools
Most tools map to distinct workflow stages such as open_target, discover_ui, probe_web_field, and release_spec, but rule/claim checking and the multiple list_* introspection tools overlap conceptually. An agent may struggle to choose between check_rules/propose_claim/falsify_claim or list_rules/list_claims, and the large surface increases misselection risk.
Names are predominantly verb_noun (retract_rule, probe_cli, review_spec), with only coverage, blocked_hosts, and api_contract as noun-only exceptions. The probe_web_field versus probe_cli split is a minor inconsistency but overall the naming is predictable.
31 tools exceeds the 25-tool threshold and includes several narrow introspection/listing tools such as list_claims, list_rules, list_feature_unknowns, list_tickets, blocked_hosts, and api_contract that could be consolidated. The multi-target workflow explains some breadth, but the surface is heavy for an agent to manage.
The surface covers target intake, UI/CLI discovery, field/CLI probing, sequences, flows, rules/claims, coverage, unknowns, docs, spec compile/review/release, tickets, audit, and mapping. Minor gaps exist around session/target listing or closing and claim lifecycle cleanup, but the core teardown-to-release workflow is well supported.
Maintenance
Related MCP Connectors
Remote MCP for Android CLI agent build gate, structured receipts, audit logs, and reviewer-ready evi
Remote MCP for A2A failure replay MCP, structured receipts, audit logs, and reviewer-ready evidence.
Live behavioral security verifier, schema auditor, and trust oracle for A2A, MCP & x402 agents.
Collision-first MCP diagnostics for MCP, API, webhook and agent failures. Returns structured evidence, bounded repair guidance, agent compatibility checks and safe recovery routing.
Related MCP Servers
- AlicenseAqualityDmaintenanceEvidence-driven quality testing for Model Context Protocol servers. It connects as a real client, captures the complete model-facing contract, identifies agent-usability risks, measures context cost, and executes only explicitly approved target calls.8MIT
- AlicenseNot gradedqualityBmaintenanceEnables passive reconstruction of an application's HTTP endpoints, WebSocket/event surfaces, payload schemas, and process architecture purely by observing labeled traffic, logs, and process metadata — never by sending requests to the target. Findings carry evidence levels and confidence, and can be exported as OpenAPI, AsyncAPI, JSON Schema, protocol catalogs, architecture maps, and candidate MCP capability surfaces.MIT
- AlicenseNot gradedqualityBmaintenanceEnables agents to take software apart over MCP — native binaries, Go/Rust/.NET assemblies, WASM and EVM bytecode, Android and Flutter apps, firmware, databases, crashes and network captures — returning findings backed by code, references and explicit unknowns. Drives deep-analysis engines such as Hopper, Ghidra, IDA, JADX and Frida entirely locally, letting agents explain behavior or rebuild a feature in your own project.538,483 npmMIT
- AlicenseNot gradedqualityAmaintenanceServes 170 reverse-engineering tools over a local stdio MCP process, letting an agent drive disassembly, decompilation, and dynamic analysis through the engines already installed on the user's machine (Ghidra, Hopper, IDA, JEB, JADX, adb, mitmproxy, Chrome). Binaries never leave the host, and every finding comes back with an evidence contract covering evidence ids, capture tool, coverage, and explicit unknowns.MIT