Covenant AAP - hosted developer pilot

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations, the description discloses important behavior: only published fields are returned, private and unlisted records are excluded from results and totals, ranking matches the public listing API, price/inventory/identity/performance/payment are unverified, and no account, signing, publication, or provider call is required. This adds substantial context that an agent would otherwise have to infer.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the core action and then lists relevant constraints. It is dense but efficient; every sentence adds useful scope or safety information without excessive repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The annotations and description together cover the read-only, non-destructive nature and exclusion rules well. However, with no output schema and 0% schema description coverage, the definition does not explain how to use the query/category/limit/offset parameters or what the result shape looks like, leaving a clear gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, and the description does not mention any of the four parameters (limit, query, offset, category). It does not compensate for the lack of schema documentation, so parameter usage remains unclear from the description alone.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: search a shared marketplace for active indexed public listings. It also clarifies the scope by excluding private and unlisted records, which helps distinguish it from other marketplace-related siblings. Sibling alternatives are not explicitly named, so it falls short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explains what is searched and what is excluded, but gives no explicit guidance on when to use this tool versus alternatives such as covenant_marketplace__quick_trade or covenant_contract__inspect. Usage is only implied by the public-search framing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is covered. The description adds one genuinely useful behavioral fact — 'Never supplies assent' — but the remaining clauses about drafting and HTML rendering describe other actions and muddy rather than clarify this call's behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

A single run-on sentence lists four capabilities of which three do not apply, burying the operative action at the very end. The most important information (this is a read-only diff of before/after) is neither front-loaded nor separated from the noise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description should explain the diff result format (change paths, added/removed values, depth handling for nested objects), and it says nothing. For a two-param nested-object comparison tool with zero schema documentation, this leaves the agent guessing about both inputs and returns.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%: both required params 'before' and 'after' are bare untyped objects with only maxProperties constraints and no explanation. The phrase 'compare exact changes' weakly implies these are the two document versions being diffed, but the description never states their shape, nesting, or what 'exact' comparison means for nested objects.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description enumerates four unrelated capabilities (drafting, ancestry preservation, comparison, HTML rendering) and only disambiguates via the trailing 'Selected action: compare.' An agent must parse the whole list to learn this invocation diffs two documents, and the leading verb 'Draft' points at a different sibling's job.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Selected action: compare' implies the tool is the comparison entry point, but there is no statement of when to use this versus covenant_contract__inspect, covenant_contract__draft, or covenant_contract__render, and no prerequisites or exclusions are given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, and closed-world behavior, so the safety profile is largely covered. The description adds real value with "Never supplies assent," clarifying that a draft carries no legal effect, and mentions escaped HTML output. It still does not say what happens to the draft (returned only vs. persisted) or why idempotentHint is false.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

It is short and front-loaded, which is good, but the opening sentence crams four distinct capabilities together and the trailing "Selected action: draft" fragment conveys nothing actionable. Tightening to the drafting behavior plus the no-assent caveat would earn every word.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 7-parameter tool with three free-form nested objects, no output schema, and zero schema descriptions, the definition is far too thin — an agent has no basis for populating dispute, settlement, extensions, or conditions, nor any idea of the returned draft's structure.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% across 7 parameters, and the description only loosely gestures at clauses and "amendment ancestry." The nested objects (dispute, settlement, extensions) and the untyped conditions array are documented in neither schema nor description, leaving four of seven inputs opaque.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description names several concrete capabilities (draft clauses, preserve amendment ancestry, compare changes, render escaped HTML), but stacking four functions into one tool muddles the core purpose and overlaps with siblings like covenant_contract__compare and covenant_contract__render. An agent cannot easily tell whether this tool drafts only, or also compares and renders.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

"Selected action: draft" implies a caller-side action selector that does not exist in the schema (no action parameter), so it provides no actionable routing signal. There is no statement of when to prefer this tool over covenant_contract__render, __compare, or __inspect.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already declare readOnlyHint=true, openWorldHint=false, and destructiveHint=false, so the description's main additional disclosure is 'Never supplies assent,' which is useful legal context but does not explain inspect behavior. It does not clarify what is inspected, whether any state is accessed, or what the response contains; the 'Draft arbitrary clauses' language also muddies the read-only nature even though the selected action is inspect.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a one-sentence list of four distinct capabilities plus a legal disclaimer and a mode selector, none of which are front-loaded around the inspect action. Most of the text describes sibling functionality rather than earning its place in this tool's definition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has a nested free-form object body, no output schema, and no parameter documentation, yet the description provides no explanation of the request shape or the expected return. For an inspect tool with a bare 'body' object and sibling alternatives, this is not enough for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There is one required parameter named 'body' with a nested object and 0% schema description coverage, and the description never mentions it. With no property descriptions, no constraints, and no description guidance, an agent has no way to know what to put in the body for an inspect action.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description lists broad capabilities (drafting, preserving ancestry, comparing, rendering) that belong to the sibling tools, then appends 'Selected action: inspect' without saying what 'inspect' actually does. It does not state a specific verb and resource for this tool's core behavior, so an agent cannot tell what covenant_contract__inspect returns or checks.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use inspect rather than covenant_contract__compare, covenant_contract__draft, or covenant_contract__render, despite those siblings being explicitly named in the description. The phrase 'Selected action: inspect' signals a mode but does not explain selection criteria or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false. The description adds useful behavior—escaped HTML output and no assent—but leaves side effects, authorization needs, and the meaning of the multi-action phrasing unclarified. It does not clearly contradict the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

It is brief, but the structure buries the operative action at the end after listing sibling-like capabilities. 'Selected action: render' should lead rather than trail.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With a nested required body parameter, no output schema, and annotations only covering safety, the description omits the input contract entirely. It implies an escaped HTML output but does not sufficiently explain output shape or usage.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There is one required object parameter, 'body,' with 0% schema description coverage, and the description never explains what belongs in body or how clauses, amendments, or changes map to it. The schema only constrains it as an object with maxProperties 20000.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a concrete output ('escaped HTML draft') and says 'Selected action: render,' but the opening clause also claims drafting, ancestry preservation, and comparison—tasks owned by sibling tools such as covenant_contract__draft and covenant_contract__compare. That makes the tool's actual scope unclear despite the render-specific phrase.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no when-to-use or when-not-to-use guidance, and no explicit routing away from the sibling draft/compare tools despite listing those capabilities. The only constraint is 'Never supplies assent,' which is not selection guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds genuine behavioral context ('no funds or inventory are inferred', every record carries seven question dimensions), but it opens with 'Create a general market, listing...' while annotations declare readOnlyHint=true — the exact kind of write/read conflict an agent must not get wrong. The description also partially self-corrects ('prepares... in one call'), producing internal ambiguity rather than a flat, clean contradiction, which is why this is a 2 rather than a 1.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Five short sentences with no wasted clauses, and the core 'prepares the complete simple agreement in one call' is near the front. But 'Selected action: quick_trade' is pure redundancy with the tool name, and the three-object opening sentence spends space without increasing precision.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 10 parameters, nested objects (payment_asset, terms, settlement, archetype) and no output schema, the description needs to carry much more: it says nothing about the return value, the meaning of the required hex identifiers, or what 'compact' changes. For a tool of this complexity, the definition leaves material gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 30%, so the description must compensate, and it largely does not: seller, buyer, context (all 64-hex patterns), amount, compact and archetype are left unexplained. 'Every record carries seven question dimensions' and 'no funds or inventory are inferred' gesture at the model but give no field-level meaning.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence tries to cover three different objects at once ('a general market, listing or reusable transaction archetype'), which blurs what the tool actually produces; only the second sentence ('prepares the complete simple agreement in one call') pins the purpose down. Nothing distinguishes it from the sibling draft/inspect tools, and the closing 'Selected action: quick_trade' just restates the name.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Credit/escrow/custom terms require adopted execution' is a real usage constraint, hinting this tool is for simple agreements only. However, the agent is never told which sibling to use for the escrowed/complex case, nor what 'adopted execution' means operationally, so the routing guidance stays implied rather than explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already establish that this is read-only and non-destructive, but the description adds important behavioral context: a declared role is not verified permission, and the tool does not authorize signing, payment, or execution. That caveat is material and not present in the structured annotations. It does not describe return behavior, but with annotations covering safety, this is a strong addition.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short and front-loads the core purpose. The row-shape detail, exclusion, reference URL, and authorization caveat are all relevant, though the external reference creates minor dependence on outside context.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with five parameters and no schema descriptions or output schema, the definition is incomplete: four parameter families are not explained. The external reference URL may help, but the description itself does not fully equip an agent to understand all inputs. The safety caveat helps, but the parameter documentation remains thin.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must carry parameter meaning. It explains only the required agreements rows as having exactly id, kind, and parties, and leaves group, grants, policies, and expected_heads undocumented. This partially compensates for the schema gap but not enough for a higher score.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: inspect caller-supplied role summaries and local authority evidence. It also clarifies the shape of each agreements row, which makes the tool's domain more concrete. It does not explicitly differentiate itself from nearby review/evidence siblings, so it stops short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by 'Inspect caller-supplied role summaries and any supported local authority evidence,' and there is one explicit exclusion: 'Do not pass a contract-draft wrapper.' However, the description does not say when to choose this tool over covenant_review__evidence or covenant_review__review, nor does it name those alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false and idempotentHint=false, so safety is covered. The description does add genuine context beyond them — that a verified proof only covers its exact statement and assumptions, not outside truth or execution permission. However, it says nothing about what the response contains or how partially verified items are reported.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Four short sentences, front-loaded with the action and followed by operational caveats. The external reference URL is awkwardly placed but nothing is redundant.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only tool with no output schema and two undocumented required parameters, the description covers the scope-of-verification caveat and the empty-input case but omits the result shape and the role of agreement_ids. An agent could call it but could not predict what it gets back.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% for both params, so the description carries the full burden. It defers the 'items' field semantics to an external URL the agent may not be able to fetch, and never mentions 'agreement_ids' at all, leaving one required parameter entirely unexplained.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource ('Check supplied proof envelopes with the supported installed verifiers'), which tells an agent it is verifying proofs rather than reviewing or inspecting agreements. It does not distinguish itself from siblings like covenant_review__review or covenant_review__authority, and 'proof envelopes' remains jargon the agent must infer.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied — an agent must infer it should call this when it already holds proof envelopes to verify. There is no explicit statement of when to use this versus covenant_review__review or covenant_review__authority, though the note that empty input reports no proofs supplied gives a small operational cue.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly and non-destructive, so the bar is lower; the description still adds real context by ruling out signing/execution, noting that drafts are not verified agreements, and warning that case builders are SDK helpers rather than endpoint tools. Return shape and how obligations/approvals are surfaced are not described.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose is front-loaded in the first sentence and the remaining sentences are short, targeted clarifications. A few disclaimers are near-boilerplate but still earn their place as guardrails against misuse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex nested-object tool with no output schema, the external reference JSON with a complete empty example covers the case format well. However expected_id is unaddressed and nothing indicates what a review returns, so the definition is not fully self-sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Coverage is 50% and the case property carries a long schema description of its own, so the baseline is 3. The description adds a reference link for the case format but says nothing about expected_id beyond the schema pattern, leaving its role (presumably integrity pinning) unexplained.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (review) and resource (an exact coordination case, its approvals and remaining obligations), which is clearer than the bare name. It does not explicitly contrast with the sibling review tools (authority, evidence), so an agent must infer the split.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It implies usage via disclaimers ('A contract draft is not a verified agreement', 'No signing or execution') and points to a reference URL for the case format. It never says when to call this versus covenant_review__authority or covenant_review__evidence, leaving the choice to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.