Server Quality Checklist

Tool Scores

• octopus_statusA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It describes a read-only check operation (connection and token org), which is non-destructive. However, it does not disclose any additional behavioral traits, such as what happens when connection fails, whether it makes network calls, or what the exact output format is. Given the simplicity, a score of 3 is reasonable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the main action and supplemented with a clear use case. Every word earns its place; there is no fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with zero parameters and no output schema, the description adequately covers the tool's function and purpose. It lacks detail on edge cases or output specifics, but given the tool's simplicity, it is complete enough. The sibling context doesn't introduce additional complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, and the schema coverage is 100% (trivially, as there are no properties). The description clarifies that the tool requires no input, which is sufficient. With zero parameters, the baseline is 4, and the description adds the context of what it checks.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: checking the Octopus connection and revealing which organization the current API token belongs to, with the explicit use case of verifying setup. It uses a specific verb ('check') and resource ('Octopus connection'), and it distinguishes itself from siblings like octopus_review_changes and octopus_ask, which are for other actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes 'Use this to verify setup,' which provides clear guidance on when to use this tool. While it does not explicitly mention alternatives, the sibling tools' names suggest different functions, and the 'verify setup' context is sufficient to differentiate usage. Lacks explicit 'when not to use' exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• octopus_askA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It usefully states that answers come with context from indexed repositories and review history, but it does not explicitly confirm side effects, permissions, output shape, or behavior when no context is found. This is acceptable but not richly transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary action, and contains no filler. Every phrase adds relevant information about the tool's purpose and data sources.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple Q&A tool with three well-documented parameters and no output schema, the description covers the main use case and information source. It could mention how to use conversationId for follow-ups or what the response format looks like, but the schema already covers parameter mechanics and the tool is not complex.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema already clearly documents question, repoId, and conversationId. The description adds no additional parameter-level semantics, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Ask') with a clear resource ('Octopus') and a well-defined subject ('a question about your code or a past review'). It also clarifies the answer draws on indexed repositories and review history, which differentiates it from sibling tools like octopus_status and octopus_review_changes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies when to use this tool: whenever the user wants a contextual answer about code or a past review. It does not explicitly name alternatives or exclusion criteria, but the general use case is clear enough for an agent to select it appropriately among its siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• octopus_review_changesA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It clearly states what the tool does (runs git diff if repoPath is given, uses HEAD with fallback to staged) and what it returns (findings and a summary). It also mentions the optional model and title, though it doesn't explicitly state any side effects or permissions. Overall, it provides good behavioral context beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary purpose, and each sentence earns its place. It covers the main usage modes, input handling, and output summary without any fluff. Perfectly concise for the functionality.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool is moderately complex (4 params, no output schema, no annotations), the description is fairly complete: it explains the input options, fallback behavior, and return type. However, it doesn't mention what the findings look like or any limitations (e.g., diff size, performance). But since no annotations exist, the description does a good job covering the essential behavioral context. A small gap is lack of info on what 'summary' contains, but overall adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the description adds meaning: it explains the relationship between diff and repoPath (mutually exclusive alternatives), mentions the fallback to staged, and clarifies the default behavior for model. Even though the schema covers all params, the description adds practical usage semantics, which is valuable. Not a 5 because the schema descriptions are already quite detailed, but the description reinforces and clarifies the key logic.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs an Octopus code review on a unified diff, with specific verbs (run, review) and resource (Octopus, unified diff). It distinguishes from siblings: octopus_status is for status, octopus_ask is for asking questions, while this one reviews changes. The first sentence is specific and actionable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly explains when to use it: for uncommitted/working changes, and provides two usage modes: pass diff directly or pass repoPath to run git diff. It doesn't explicitly name alternatives or exclusions, but the contrast with siblings is implied by the tool name and function. The second sentence gives clear context for usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.