Server Quality Checklist

Tool Scores

• group_todos_by_topicC
  2.7/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It does not state whether the operation is read-only, what the output format is, or any side effects. The phrase 'group by various criteria' gives no insight into behavior beyond the basic function.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no fluff. It efficiently communicates the core action and examples of grouping dimensions. Perfectly concise for the information provided.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool groups TODOs, which is a complex operation, but there is no output schema and no description of the return value or grouping output structure. The description omits critical details like how criteria are applied, what the grouped result looks like, and whether it affects data. Incomplete for an agent to confidently invoke.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters, which would normally score a baseline of 4. However, the description mentions criteria (file paths, priority, ownership) that seem to imply selectable parameters, yet none exist. This creates confusion about how grouping is configured, lowering the score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool groups TODOs and lists criteria such as file paths, priority, and ownership. This distinguishes it from siblings like scan_todos (scanning) and explain_todo (explaining). However, the tool name mentions 'topic' but the description's criteria do not explicitly include topic, creating slight ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like scan_todos or explain_todo. The description only states what the tool does, with no context about suitable scenarios or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_workspaceA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It states the action (set) and context (for scanning TODOs), but does not explain side effects like overwriting existing settings or whether the change persists for subsequent calls.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, with the primary action front-loaded. Every word contributes, with no fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with one parameter, and the description covers purpose and usage condition. However, it omits any mention of validation, side effects, or how the set workspace affects subsequent operations, leaving some gaps for a mutation tool without annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides full coverage with a description for the 'root' parameter. The tool description adds no additional semantic detail beyond restating that it sets the root directory, so it matches the schema baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states 'Set the workspace root directory for scanning TODOs,' using a specific verb and resource. This clearly distinguishes it from sibling tools like get_workspace and scan_todos.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The second sentence provides an explicit condition: 'Call this first if the default workspace detection doesn't work.' This tells the agent when to use it, though it doesn't mention alternatives or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• explain_todoA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It states the tool retrieves context and code lines, implying a read-only operation. However, it does not disclose behavior on missing IDs, exact return structure, or any side effects. The description is not misleading, but lacks depth expected without annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that immediately states the action and what is included. There is no filler or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with two well-described parameters and no output schema, the description adequately indicates the return value (context and surrounding code lines). It could specify what 'context' entails, but it is complete enough for the tool's straightforward scope.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already documents both parameters (id and contextLines) with full descriptions. The description's mention of 'surrounding code lines' reinforces contextLines but adds no new semantic detail beyond what the schema provides, matching the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb (Get) and identifies the resource (specific TODO item) and the output (surrounding code lines). It clearly distinguishes from siblings like scan_todos, which likely lists items, by focusing on a single item's context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies when to use it: when you need more context for a specific TODO item. It does not explicitly mention when not to use it or alternatives, but the distinct purpose among siblings provides sufficient guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scan_todosA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden. It states that it returns all TODOs with locations and metadata, but doesn't explicitly confirm it's read-only or explain what 'metadata' includes. It adds some value but lacks depth on side effects or edge cases.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two succinct sentences, front-loaded with the action and resource. No wasted words, and the structure is immediately scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description provides enough context for a simple scan tool: states the action and the output (TODOs with locations and metadata). While it lacks details on the return structure or performance implications, the overall complexity is low, and the provided info is reasonable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, with each parameter ('root', 'exclude', 'include') well described in the input schema. The description adds no additional parameter-specific meaning, so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Scan') and resource ('the codebase for TODO comments'), clearly distinguishing this tool from siblings like 'group_todos_by_topic' and 'explain_todo'. It unambiguously states what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The context is clear: this is the tool for scanning TODO comments, while siblings serve different purposes (workspace management, grouping, explaining). It doesn't explicitly name alternatives or exclusions, but the purpose itself strongly implies the appropriate use case.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_workspaceA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It states what is returned (the workspace root) but does not disclose behavior in edge cases such as what happens if no workspace is set (e.g., returns null, throws error). The description is accurate but lacks depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that is front-loaded with the verb and resource. Every word contributes meaning with no waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a low-complexity tool with no parameters and no output schema. The description explains the purpose and what it returns. It could add edge-case behavior for full completeness, but for a simple getter it is largely complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so per the rubric the baseline is 4. The description adds no parameter info, but none is needed since the schema shows zero properties.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Get' and names the resource 'current workspace root directory' and its purpose 'will be used for scanning'. This clearly distinguishes it from sibling tools like set_workspace and scan_todos.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context: it retrieves the workspace root for scanning purposes. While it doesn't explicitly mention alternatives or exclusions, the context is enough for an agent to know when to call it, especially since the only related sibling is set_workspace.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.