RepoPilot

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, openWorldHint=true, idempotentHint=true, destructiveHint=false, so the safety profile is covered. The description adds valuable behavioral context beyond annotations: selection is bounded, not exhaustive or proof of compatibility; checked_sha pins an immutable revision; pr observes GitHub check runs; source_ranges retrieves missing context at pinned revisions. It does not contradict annotations. A 4 is appropriate because it adds meaningful behavioral nuance without redundancy.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but efficient, front-loading the core purpose and then enumerating parameter behaviors in a compact sequence. The example is valuable. It loses one point because the density makes it slightly hard to parse in a single pass, and some information (e.g., 'Requires question') is repeated in the schema. Still, every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (9 parameters, multiple modes, output schema present), the description covers the key decision points: repo vs package, question vs no question, pr vs base_sha, and source_ranges constraints. The output schema exists, so return values need not be described. It is not a 5 because the description doesn't explicitly state what happens when both repo and package are omitted or when neither question nor source_ranges is provided, though the schema's oneOf and required constraints partially cover this.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 89%, so the schema already documents most parameters well. The description adds semantic value by explaining the relationship between parameters (e.g., checked_sha selects a commit, base_sha adds before/after excerpts, pr observes check runs, source_ranges retrieves missing context) and by giving a concrete example. It doesn't fully explain every parameter interaction, but the schema plus description is strong. Baseline 3 is exceeded because the description clarifies the workflow-level meaning of the parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a clear verb and resource: 'CALL to investigate behavior in public source' and immediately distinguishes the two modes (evidence retrieval vs. cached decision brief). It names the sibling alternative (evaluate_dependency_change) for package-version judgment, and the example anchors the intended use. This is specific and differentiates from siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit when-to-use guidance: supply question and path_hints for evidence, omit question for the cached brief, pass exactly one of repo or package, and use evaluate_dependency_change for a specific package version. It also states constraints like 'Requires question' and 'Omit base_sha' for pr, which are reinforced in the schema. This is strong routing guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false, so the safety profile is covered. The description adds valuable behavioral context: output is deterministic, it never approves a merge (limit on its authority), and it notes repo caching for hotspot context. It does not detail receipts format, but with readOnlyHint covering the safety dimension, 4 is generous but warranted.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with when to call, the core value (deterministic score with receipts), input modes, and a critical behavioral caveat. Every clause carries information; zero filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Output schema exists so return format needn't be detailed. The description covers invocation timing, input-mode disjunction, scoring dimensions, determinism, and non-approval semantics. It doesn't detail the receipt structure or caching behavior, but given the good annotations and output schema, this is adequately complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 63%, and the description fills key gaps: it explains the input-mode semantics (repo+PR OR diff), states repo may accompany diff for cached hotspot context, and notes files array is 'privacy-preserving... contains no source' — clarifying that line counts may be placeholders. This adds meaning beyond the oneOf constraints.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states it computes a 'deterministic 0-10 change-shape score' with receipts for specific factors (size, spread, missing tests, sensitive paths, hotspots, blast radius). The verb 'check' plus resources (PR/diff) and the distinct scoring behavior distinguish it from siblings like check_dependency or analyze_repo.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says 'CALL before merging a pull request or after producing a local diff', giving concrete timing. It lists valid input combinations ('repo+pr OR diff; repo may accompany diff') and clarifies it 'prioritizes review and never approves a merge', which sets expectations for when it's appropriate vs not.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, and non-destructive behavior; the description adds the important caveat that a favourable result does not validate an exact package version or compatibility. It also describes what kind of report is returned (lean repository-level recommendation, CVEs, maintenance, etc.), which is useful beyond the structured fields.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded: it starts with the trigger condition, then the return value, then exclusions and constraints. Each sentence carries distinct information, and the longer list of returned fields is informative rather than filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the open-world read-only annotations and presence of an output schema, the description adequately covers triggers, parameter constraints, and the key version/compatibility limitation. It does not explicitly differentiate from all siblings like verify_dependency_change, but the 'before adoption' framing and version-routing rule are sufficient for an agent to select it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, yet the description still adds meaning by enforcing 'exactly one of repo or package' and clarifying that repo can be owner/repo or a github.com URL. The package parameter description also points users to evaluate_dependency_change for version checks, making parameter choice unambiguous.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific action ('check') and resource ('dependency before adoption') and lists concrete triggers: add, upgrade, trust, fork, or deploy. It also distinguishes itself from code navigation and from evaluate_dependency_change for version-specific checks, so an agent can identify the tool without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It opens with an explicit 'CALL when...' list of adoption scenarios and includes a 'DO NOT use for code navigation' exclusion. It further routes version-specific judgments to evaluate_dependency_change, giving clear when-to-use and when-not-to-use guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, openWorld, idempotent, non-destructive. The description adds valuable behavioral detail beyond these: it discloses that on a cache miss it uses bounded live GitHub/OpenSSF evidence with limited confidence and explicit unknowns rather than fabricated certainty. This is honest about limitations and return variability based on cache state.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, tightly packed with useful information: when to call, what's returned, accepted formats, and behavioral caveats. No wasted words or repetition of annotation content. Front-loaded with the trigger condition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Tool has an output schema and rich annotations, so description needn't cover return value structure. It covers trigger condition, target format inputs, evidence sourcing behavior, and confidence caveats. For a comparison tool with simple two-string inputs and disclosed fallback strategy, this is complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so both parameters a and b are documented. The description also adds meaning by explaining accepted formats (owner/repo, GitHub URL, npm package name, npm:@scope/pkg) which extends beyond the schema's generic 'repo or npm package target'. However, it doesn't clarify dimensional/ordering semantics (e.g., whether a/b order matters) beyond what the schema provides, so a baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool compares two repository/dependency choices and lists what it returns (preferred candidate per use case, trade-offs, confidence, evidence gaps). It explicitly scopes to 'exactly two' choices, distinguishing it from single-repo analysis siblings. The accepted target formats are enumerated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states when to CALL ('when the user is choosing between exactly two dependencies or repositories') and implies when NOT to (single choice). It describes the fallback behavior: cached analyses preferred, bounded live evidence on miss with explicit unknowns rather than guessing. This provides strong guidance for selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive and open-world, yet the description adds real behavioral context on top: everything inferred is recorded in input_adjustments, a missing project snapshot yields "unknown" compatibility which must be read as "not checked, never no problem found", and policy_profile mis-spellings are rejected rather than guessed. That is precisely the extra disclosure annotations cannot carry.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the trigger and the question it answers, followed by inputs, defaults, and the no-side-effects guarantee. It is dense but nearly every sentence carries a distinct, actionable fact; only the repeated "never nested under a change key" warning overlaps with the schema text and costs a little tightness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation-adjacent but read-only tool with a rich nested schema, an output schema, and strong annotations, the description covers the trigger, the shape of the return (blockers, warnings, recommendation, verification plan), the input-resolution behavior, and the no-write guarantee. Nothing an agent needs to invoke it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds genuine meaning: only `dependency` is required, omitting to_version evaluates the latest published version exactly as npm install would, and a dist-tag or SemVer range resolves to one exact version reported in change.to_version. It also warns that dependency is a bare string and every field is top-level, never nested under a "change" key.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource — evaluates an exact dependency change — and scopes it with concrete evidence sources (registry metadata, advisory deltas, provenance, license, repository evidence). It never names the near siblings check_dependency or verify_dependency_change, so the agent must infer which one applies, which keeps it out of the 5 band.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

"CALL immediately before adding or upgrading an npm dependency" gives a clear trigger condition, and "Evaluates only; never installs or edits anything" marks the boundary against an install action. There is no explicit when-not or named alternative among the sibling tools, so it is clear context without exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already assert read-only, idempotent, open-world, and non-destructive behavior; the description adds cache semantics ('cached CLAUDE.md-style artifact'), verify-before-trusting guidance, and a caution about context reuse. No contradiction with annotations. It doesn't address staleness or auth, but the annotation profile lowers the burden.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description front-loads the when-to-call condition, then gives output, cancellation, and input-constraint guidance in compact sentences. Every sentence earns its place, and the minimal-call example is a high-value addition rather than filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a low-complexity, 4-parameter, read-only tool with a rich schema, an output schema, and safety annotations, the description covers when, why, how to call, and when not to call. Nothing essential is missing for an agent to invoke it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3; the description adds value beyond the schema by stating that there is no artifact id, that exactly one of repo or package must be supplied, and that each must be a plain string. The minimal-call example reinforces the required shape without repeating every schema property.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a concrete resource (repository/package), a precise action (return the cached CLAUDE.md-style artifact or Cursor rules), and a clear purpose (orient the agent before substantial code work). It also clarifies that there is no artifact id, avoiding confusion with the tool name. It does not explicitly differentiate from siblings like analyze_repo or compare_repos, so it stops short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives an explicit trigger condition ('CALL before substantial code work in an unfamiliar repository when...'), a clear do-not-call rule ('Do not call again if the artifact is already in context'), and an explicit alternative for version-specific judgment (evaluate_dependency_change in the package parameter). This fully routes the agent.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already carry readOnlyHint, openWorldHint, and destructiveHint, so the bar is lower. The description still adds valuable non-obvious context: task text is sent to RepoPilot's model provider, is never persisted or echoed, and rankings are suggestions rather than authoritative actions. This goes beyond the annotations without contradicting them.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but every sentence earns its place: call-to-action, input guidance, output summary, privacy disclosure, and a caveat. It is front-loaded with the most important usage signal and contains no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers the main repo-based use case, output categories, and privacy behavior, and an output schema exists to carry return-value details. However, it ignores the package input branch of the oneOf schema, does not explain intent or max_files, and leaves the 'short-lived verification contract' vague. For a complex 8-parameter tool, this is a noticeable gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is about 50%, so the description must compensate. It adds useful meaning for task (max 4000 chars, include error and expected behaviour) and checked_sha (pass when known). But intent, max_files, path_hints, and the repo/package branch remain underdescribed, so compensation is only partial.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the tool's action and resource: planning a task against repository evidence, finding where a bug or feature lives, and returning ranked files. It is specific and useful, but it does not explicitly distinguish this tool from sibling tools such as analyze_repo or check_change_risk, so it misses the differentiator needed for a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives concrete when-to-use guidance: 'CALL before editing an unfamiliar public repository, or to find where a bug or feature lives.' It also tells the agent what to include in task text and when to pass checked_sha. However, it does not name alternatives or state when not to use this tool, so it falls short of fully explicit routing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish readOnly/idempotent/non-destructive, so the bar is lower, and the description still adds real disclosure: it refuses to run commands, does not store diff or check output, and explicitly marks receipts as caller asserted rather than verified. That trust-model statement is exactly the kind of context annotations cannot convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with the imperative call condition, then the comparison behavior, then the negative constraints. No sentence is filler or restates the tool name.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a four-parameter, deeply nested verification tool with an output schema present, the description covers when to call, what is compared, what is reported, and what the tool will not do. Return-value detail is legitimately delegated to the output schema; only fine-grained input semantics remain thin.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 25% across four nested parameters, so the description must carry more weight, and it does clarify the roles of the three inputs (evaluated target, resolved result, proof receipts) at a conceptual level. However, it adds no field-level or format guidance beyond what the schema and its embedded descriptions already state.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

Names a specific action (verifies a dependency change by comparing the evaluated target against the resolved result and caller proof receipts) and states its outputs (missing/failed evidence, residual risk, caller-asserted labeling). This is clearly distinguishable from evaluate_dependency_change, which produces the evaluation this tool consumes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

"CALL after changing the manifest/lockfile and running local checks" gives an explicit triggering condition that an agent can act on. It does not name the alternative tools or state when not to call it, so it stops short of full routing guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.