RepoPilot

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover safety (readOnlyHint, idempotentHint, destructiveHint=false), but the description adds substantive behavioral limits beyond them: selection is 'bounded, not exhaustive or proof of compatibility', four-file cap, 40-line-per-range cap, commit-pinned citations, and the PR check-run behavior. That is real added context, though it doesn't discuss rate limits or error behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with an imperative 'CALL to investigate behavior in public source', then proceeds through parameter behaviors in a logical order. It is dense and semicolon-heavy, packing multiple constraints per sentence, but nearly every clause carries a distinct, load-bearing fact rather than filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a nine-parameter tool with nested source_ranges and a declared output schema, the description covers the essential mode interactions (pr vs base_sha, question-required modes, one-of repo/package) and even summarizes repository_brief. Combined with the existing output schema, an agent has enough to invoke it correctly, with only minor gaps around failure modes.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 89%, so the baseline is 3 and the schema already documents most parameters. The description still adds cross-cutting semantics: how checked_sha/base_sha/pr/source_ranges combine, the before/after excerpt behavior, and the precise 'up to 40 lines per range' retrieval bound. This is meaningful beyond the schema's per-field text.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: investigate behavior in public source via question + path_hints, returning commit-pinned line citations. The scope (up to four files, bounded selection) is precise. It does not, however, name or distinguish itself from any sibling such as compare_repos or check_change_risk, so an agent still has to infer the boundary.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear operational context for each mode: checked_sha selects a commit, base_sha adds before/after excerpts, pr selects a public PR and observes its check runs, source_ranges fills missing context, and omitting question yields repository_brief. It also states 'Pass exactly one of repo or package.' It stops short of explicit when-not guidance or naming an alternative tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false, so the safety profile is covered. The description adds valuable behavioral context: output is deterministic, it never approves a merge (limit on its authority), and it notes repo caching for hotspot context. It does not detail receipts format, but with readOnlyHint covering the safety dimension, 4 is generous but warranted.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with when to call, the core value (deterministic score with receipts), input modes, and a critical behavioral caveat. Every clause carries information; zero filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Output schema exists so return format needn't be detailed. The description covers invocation timing, input-mode disjunction, scoring dimensions, determinism, and non-approval semantics. It doesn't detail the receipt structure or caching behavior, but given the good annotations and output schema, this is adequately complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 63%, and the description fills key gaps: it explains the input-mode semantics (repo+PR OR diff), states repo may accompany diff for cached hotspot context, and notes files array is 'privacy-preserving... contains no source' — clarifying that line counts may be placeholders. This adds meaning beyond the oneOf constraints.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states it computes a 'deterministic 0-10 change-shape score' with receipts for specific factors (size, spread, missing tests, sensitive paths, hotspots, blast radius). The verb 'check' plus resources (PR/diff) and the distinct scoring behavior distinguish it from siblings like check_dependency or analyze_repo.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says 'CALL before merging a pull request or after producing a local diff', giving concrete timing. It lists valid input combinations ('repo+pr OR diff; repo may accompany diff') and clarifies it 'prioritizes review and never approves a merge', which sets expectations for when it's appropriate vs not.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, and non-destructive behavior; the description adds the important caveat that a favourable result does not validate an exact package version or compatibility. It also describes what kind of report is returned (lean repository-level recommendation, CVEs, maintenance, etc.), which is useful beyond the structured fields.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded: it starts with the trigger condition, then the return value, then exclusions and constraints. Each sentence carries distinct information, and the longer list of returned fields is informative rather than filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the open-world read-only annotations and presence of an output schema, the description adequately covers triggers, parameter constraints, and the key version/compatibility limitation. It does not explicitly differentiate from all siblings like verify_dependency_change, but the 'before adoption' framing and version-routing rule are sufficient for an agent to select it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, yet the description still adds meaning by enforcing 'exactly one of repo or package' and clarifying that repo can be owner/repo or a github.com URL. The package parameter description also points users to evaluate_dependency_change for version checks, making parameter choice unambiguous.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific action ('check') and resource ('dependency before adoption') and lists concrete triggers: add, upgrade, trust, fork, or deploy. It also distinguishes itself from code navigation and from evaluate_dependency_change for version-specific checks, so an agent can identify the tool without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It opens with an explicit 'CALL when...' list of adoption scenarios and includes a 'DO NOT use for code navigation' exclusion. It further routes version-specific judgments to evaluate_dependency_change, giving clear when-to-use and when-not-to-use guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, openWorld, idempotent, non-destructive. The description adds valuable behavioral detail beyond these: it discloses that on a cache miss it uses bounded live GitHub/OpenSSF evidence with limited confidence and explicit unknowns rather than fabricated certainty. This is honest about limitations and return variability based on cache state.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, tightly packed with useful information: when to call, what's returned, accepted formats, and behavioral caveats. No wasted words or repetition of annotation content. Front-loaded with the trigger condition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Tool has an output schema and rich annotations, so description needn't cover return value structure. It covers trigger condition, target format inputs, evidence sourcing behavior, and confidence caveats. For a comparison tool with simple two-string inputs and disclosed fallback strategy, this is complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so both parameters a and b are documented. The description also adds meaning by explaining accepted formats (owner/repo, GitHub URL, npm package name, npm:@scope/pkg) which extends beyond the schema's generic 'repo or npm package target'. However, it doesn't clarify dimensional/ordering semantics (e.g., whether a/b order matters) beyond what the schema provides, so a baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool compares two repository/dependency choices and lists what it returns (preferred candidate per use case, trade-offs, confidence, evidence gaps). It explicitly scopes to 'exactly two' choices, distinguishing it from single-repo analysis siblings. The accepted target formats are enumerated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states when to CALL ('when the user is choosing between exactly two dependencies or repositories') and implies when NOT to (single choice). It describes the fallback behavior: cached analyses preferred, bounded live evidence on miss with explicit unknowns rather than guessing. This provides strong guidance for selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, openWorldHint=true, and idempotentHint=true, and the description aligns exactly by stating it evaluates only and never installs or edits anything. Beyond that, it discloses that every default, repair, and resolution is listed in input_adjustments, that missing project data yields compatibility "unknown" (not "no problem found"), and that to_version ranges resolve to one exact version reported in change.to_version.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is long but front-loads the action and purpose in the first sentence, then layer usage, inputs, and behavior. It is mostly dense and purposeful, though the repeated emphasis on top-level fields and the example JSON add some redundancy for an already 100%-covered schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex evaluation tool with 8 parameters, nested objects, and an output schema, the description covers the critical agent-facing behavior: when to call, what is inferred, how omitted project data is handled, that unsupported input shapes are rejected, and that the tool never mutates. With the output schema present, it does not need to enumerate return fields in detail, and its mention of blockers, warnings, recommendation, and verification plan is sufficient orientation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is already 100%, but the description adds substantial meaning beyond the schema: dependency must be a plain string (not an object or a {"change":{}} wrapper), to_version accepts exact versions, dist-tags, or SemVer ranges, from_version enables before/after advisory comparison, and policy_profile near-misses are rejected rather than guessed. It also explains dependency_type word mappings and that package_manager only affects verification commands and lockfile naming.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: evaluate a dependency change, answering "is this exact version safe to take on" from registry, advisory, provenance, license, and repository evidence. It clearly distinguishes evaluation from installation ("Evaluates only; never installs or edits anything"). However, it does not name or contrast with sibling tools such as verify_dependency_change or check_dependency, so sibling differentiation is left implicit.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives a clear trigger: "CALL immediately before adding or upgrading an npm dependency." It also clarifies the minimum input ("Only `dependency` is required — omit to_version to evaluate the latest published version"). But it provides no explicit when-not-to-use conditions or named alternatives, so the agent must infer the boundary with check_dependency and verify_dependency_change.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already assert read-only, idempotent, open-world, and non-destructive behavior; the description adds cache semantics ('cached CLAUDE.md-style artifact'), verify-before-trusting guidance, and a caution about context reuse. No contradiction with annotations. It doesn't address staleness or auth, but the annotation profile lowers the burden.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description front-loads the when-to-call condition, then gives output, cancellation, and input-constraint guidance in compact sentences. Every sentence earns its place, and the minimal-call example is a high-value addition rather than filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a low-complexity, 4-parameter, read-only tool with a rich schema, an output schema, and safety annotations, the description covers when, why, how to call, and when not to call. Nothing essential is missing for an agent to invoke it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3; the description adds value beyond the schema by stating that there is no artifact id, that exactly one of repo or package must be supplied, and that each must be a plain string. The minimal-call example reinforces the required shape without repeating every schema property.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a concrete resource (repository/package), a precise action (return the cached CLAUDE.md-style artifact or Cursor rules), and a clear purpose (orient the agent before substantial code work). It also clarifies that there is no artifact id, avoiding confusion with the tool name. It does not explicitly differentiate from siblings like analyze_repo or compare_repos, so it stops short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives an explicit trigger condition ('CALL before substantial code work in an unfamiliar repository when...'), a clear do-not-call rule ('Do not call again if the artifact is already in context'), and an explicit alternative for version-specific judgment (evaluate_dependency_change in the package parameter). This fully routes the agent.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already carry readOnlyHint, openWorldHint, and destructiveHint, so the bar is lower. The description still adds valuable non-obvious context: task text is sent to RepoPilot's model provider, is never persisted or echoed, and rankings are suggestions rather than authoritative actions. This goes beyond the annotations without contradicting them.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but every sentence earns its place: call-to-action, input guidance, output summary, privacy disclosure, and a caveat. It is front-loaded with the most important usage signal and contains no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers the main repo-based use case, output categories, and privacy behavior, and an output schema exists to carry return-value details. However, it ignores the package input branch of the oneOf schema, does not explain intent or max_files, and leaves the 'short-lived verification contract' vague. For a complex 8-parameter tool, this is a noticeable gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is about 50%, so the description must compensate. It adds useful meaning for task (max 4000 chars, include error and expected behaviour) and checked_sha (pass when known). But intent, max_files, path_hints, and the repo/package branch remain underdescribed, so compensation is only partial.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the tool's action and resource: planning a task against repository evidence, finding where a bug or feature lives, and returning ranked files. It is specific and useful, but it does not explicitly distinguish this tool from sibling tools such as analyze_repo or check_change_risk, so it misses the differentiator needed for a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives concrete when-to-use guidance: 'CALL before editing an unfamiliar public repository, or to find where a bug or feature lives.' It also tells the agent what to include in task text and when to pass checked_sha. However, it does not name alternatives or state when not to use this tool, so it falls short of fully explicit routing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive, open-world. The description adds genuinely new behavioral facts: it never executes commands, never stores diff/check output, and it labels receipts as caller-asserted rather than verified — important trust semantics not captured by annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two dense sentences, front-loaded with the imperative CALL trigger, and every clause adds information about behavior or inputs. It is long but not padded, though it could be broken into shorter sentences for scanability.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex verification tool with an output schema present (so return format need not be explained), the description covers sequencing, comparison behavior, advisory scanning, evidence-gap reporting, and non-execution. It is nearly complete, with only parameter-level detail for the nested inputs left thin.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 25%, so the description must compensate, and it does explain a few fields (resolved_changes as packages added/changed, receipts as caller-reported). However most of the large nested schema (after, checks, evaluation) gets no semantic elaboration in the description, leaving real gaps.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states concrete verbs and resources: it compares the evaluated target against the resolved result and caller receipts, checks advisories across resolved_changes, and reports missing/failed evidence and residual risk. An agent can distinguish this from evaluate_dependency_change, which produces the evaluation this tool consumes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives an explicit trigger ('CALL after changing the manifest/lockfile and running local checks'), which is exactly the when-to-use guidance an agent needs. It does not name when-not-to-call or point at a sibling alternative, so it falls short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.