Server Quality Checklist

Tool Scores

• sage-opinionA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses key behavioral traits: the tool sends a prompt to an external model, handles file paths as context, uses absolute paths, and may error if too much context is included. However, it lacks details on rate limits, authentication needs, or what the 'sage-like model' entails (e.g., model type, limitations). The description doesn't contradict annotations since none exist.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with the core purpose stated first. It uses bullet-like formatting for key points (paths, absolute paths, context limits), but includes some redundancy (e.g., repeating absolute path requirement). Most sentences earn their place by clarifying usage, though it could be slightly more streamlined.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no output schema, no annotations), the description is somewhat complete but has gaps. It covers the basic operation and constraints, but lacks details on the model's behavior, error handling specifics, or output expectations. Without annotations or an output schema, more context on what 'opinion' entails would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters ('paths' and 'prompt') with descriptions. The description adds minimal value beyond the schema: it reiterates the need for absolute paths and context inclusion but doesn't provide additional syntax, format details, or examples. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Send a prompt to sage-like model for its opinion on a matter.' It specifies the verb ('send'), resource ('sage-like model'), and action ('for its opinion'). However, it doesn't explicitly differentiate from sibling tools like 'sage-plan' or 'sage-review' beyond the 'opinion' focus, which is implied but not contrasted.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: 'Include the paths to all relevant files and/or directories that are pertinent to the matter' and advises on absolute paths and context limits. It implicitly suggests using this tool for opinion-seeking tasks, but it doesn't explicitly state when to choose this over siblings like 'sage-plan' or 'sage-review', nor does it list exclusions or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sage-reviewA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explains that the tool includes 'full content of all files in the specified paths' and returns 'edit suggestions in a specific format with search and replace blocks', which adds useful context beyond basic functionality. However, it doesn't cover potential limitations like rate limits, authentication needs, or error conditions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and appropriately sized, with key information front-loaded. However, the second paragraph could be more concise, and the 'IMPORTANT' section repeats path information already stated elsewhere, slightly reducing efficiency.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (code review with file processing) and lack of annotations/output schema, the description is moderately complete. It explains the core behavior and format of suggestions but doesn't detail what happens with invalid paths, how large files are handled, or the structure of the returned edit blocks, leaving some gaps for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters thoroughly. The description reinforces that paths 'must be absolute paths' and mentions directory recursion, but this is already covered in the schema. It adds minimal value beyond what the structured schema provides, meeting the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('send code', 'get specific edit suggestions') and resources ('sage model', 'SEARCH/REPLACE blocks'). It distinguishes from sibling tools by specifying this is for 'expert review' with edit suggestions, unlike 'sage-opinion' or 'sage-plan' which likely serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidelines: 'Use this tool any time the user asks for a "sage review" or "code review" or "expert review"'. It also includes alternative handling when paths aren't specified ('use as many paths... as you're aware of'), giving clear context for when and how to invoke the tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sage-planA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behavioral traits: the multi-model debate process (generation, critique, refinement, consensus), the creation of detailed plans, and file-saving behavior when outputPath is provided. It also notes the expense of the debate, which is useful context. However, it lacks details on error handling or performance expectations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, starting with the core purpose. Most sentences add value, such as explaining the debate process and file-saving behavior. However, some redundancy exists (e.g., reiterating absolute paths), and the structure could be slightly tighter by integrating the IMPORTANT note more seamlessly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a 5-parameter tool with no annotations and no output schema, the description does a good job of covering the tool's behavior and key usage aspects. It explains the debate process and file outputs, but it could be more complete by detailing the format of the output (e.g., Markdown structure) or potential limitations, which would help set clearer expectations for the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds some value by emphasizing the importance of absolute paths for the 'paths' parameter and explaining the file-saving behavior for 'outputPath', but it does not provide additional semantic context beyond what the schema offers, such as typical use cases for parameters like 'maxTokens' or 'rounds'.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Generate an implementation plan via multi-model debate.' It specifies the verb ('generate') and resource ('implementation plan'), and distinguishes it from siblings by detailing the unique multi-model debate process, which is not implied by the sibling names 'sage-opinion' and 'sage-review'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: for creating detailed, well-thought-out implementation plans through iterative debate. However, it does not explicitly state when not to use it or mention alternatives like the sibling tools, which could help differentiate use cases more precisely.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.