Server Quality Checklist

Tool Scores

• add_taskA
  3.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full behavioral burden. It only states that the tool registers a task but does not disclose side effects, permissions, return values, or validation of depends_on. For a mutating operation, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with purpose, and the second sentence is a useful alternative pointer. No redundant content and every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple and the schema covers all parameters, but the description lacks behavioral details such as return value or task state after registration. There is no output schema or annotations to compensate, so while the usage context is helpful, the description is minimally adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all parameters are already documented in the schema. The description does not add any parameter-specific semantics beyond what the schema provides, so the baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Register' with resource 'a single follow-up task' and adds context 'discovered mid-investigation'. It also distinguishes from sibling tool by naming add_tasks_bulk(), making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly states when to use this tool ('discovered mid-investigation') and when to use the alternative ('Use add_tasks_bulk() instead if defining the full plan upfront'). This clearly routes the agent to the correct tool based on the planning stage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reopen_taskA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses important behavioral traits: resetting to pending, preserving history, recalculating dependency state, and requiring a reason. This is substantive, though it could mention error cases or idempotency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each earning its place. The opening sentence states the main action and condition, the second details side effects, and the third instructs on the required reason. No filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given a simple two-parameter tool with no output schema and no annotations, the description covers the core use case, behavioral side effects, and parameter expectations. It could mention return value or failure behavior, but for this complexity it is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already covers both parameters with helpful descriptions. The description reinforces the purpose of 'reason' but does not add new semantic detail beyond the schema. Baseline 3 is appropriate given full schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb ('Reopen') and resource ('blocked task'), clearly distinguishing this from siblings like update_task or reset. It also states the key operations: resets to pending, preserves finding, recalculates dependency state.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'after its blocker has been resolved' provides explicit timing context for when to use the tool. It does not explicitly name alternatives or exclusions, but the context is clear enough for an agent to select it appropriately.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_taskA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing side effects. It discloses that completing a task automatically unblocks dependents and that dependency rewiring only applies to pending tasks. This goes beyond a generic 'update' description, though it could mention potential status transition restrictions or idempotency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the primary purpose, followed by critical constraints and side effects. Every clause earns its place; no redundant wording.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the main use cases: resolving, updating finding alone, and rewiring dependencies. It doesn't explicitly explain status transition behavior (e.g., moving back to in_progress), but given the rich input schema, it is sufficiently complete for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents each parameter's purpose, including the pending-only constraint for depends_on and the required finding. The description adds behavioral context (auto-unblock) but adds little new parameter-specific semantics. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description begins with 'Resolve a task after completing work on it,' which clearly identifies the verb (resolve) and resource (task). It also outlines multiple distinct capabilities (updating finding, rewiring dependencies) that distinguish it from sibling tools like add_task or get_all_tasks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool: after completing work, and it specifies when a finding is required (completed/skipped/blocked). It does not explicitly name alternative tools, but the context is strong enough that an agent would know when to select this over siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_tasks_bulkA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden. It discloses atomicity ('one atomic call'), validation ('All tasks are validated before any are inserted'), and return behavior ('Returns a prioritised execution plan'). These are valuable behavioral traits beyond basic write semantics, though it does not cover error handling or idempotency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no filler. The opening 'FIRST CALL' is a strong front-loaded signal. Every word adds value: atomicity, validation, DAG, prioritised plan. Exemplary conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has one parameter with nested structure and no output schema. The description explains the core behavior (bulk DAG registration) and return value (execution plan), which is sufficient for an AI agent to invoke it correctly. It could mention idempotency or response format, but overall it is complete for the described use case.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so parameters are structurally documented. The description adds semantic meaning by explaining that tasks form a DAG (clarifying depends_on) and that a prioritised execution plan is returned (relating to priority). It turns the schema's raw field list into a coherent concept of a task graph.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool's function: 'Register your entire investigation plan as a DAG in one atomic call.' This distinguishes it from the singular 'add_task' sibling by emphasizing bulk registration of the whole plan. The phrase 'FIRST CALL' further reinforces its specific role.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description conveys when to use it ('FIRST CALL') and implies it is the entry point for registering a full plan. It does not explicitly name alternatives or exclusions, but the 'FIRST CALL' directive and mention of the entire investigation plan provide clear usage context compared to siblings like add_task or get_ready_tasks.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_all_tasksA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description discloses the full scope of the snapshot and implies read-only nature, but does not explicitly state safety traits or performance characteristics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the main action and immediately followed by useful details and usage context—no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Adequately covers what the tool does and when to use it for a simple read-only operation, though it omits any explicit mention of return format or scale limitations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, so schema coverage is trivially 100%; baseline of 4 applies without needing description compensation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it retrieves a full snapshot of the entire DAG with specific attributes (tasks, status, findings, dependencies, progress), distinguishing it from filtered siblings like get_ready_tasks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit context for when to use ('review the full picture or check what is blocked'), but does not mention exclusions or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_ready_tasksA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing behavior. It explains the filtering condition (dependencies fully resolved), sort order (by priority), and the meaning of an empty result (all done or blocked). This exceeds basic expectations, though it doesn't mention potential errors or side effects (likely read-only).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each adding value: the trigger and core function, the practical use, and the interpretation of empty results. No redundant information is present, making it highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no params, no output schema), the description is fully complete. It covers when to call it, what it returns, how results are sorted, and what an empty list means. There are no gaps for a user to guess.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters, so schema coverage is trivially 100%. For 0-parameter tools, the baseline is 4, and the description adds meaning by explaining what the tool returns and how to interpret the result. No parameter documentation is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Returns all tasks whose dependencies are fully resolved, sorted by priority'. It uses a specific verb and resource, and distinguishes itself from get_all_tasks by focusing only on ready tasks. The added context about working on these tasks next reinforces its purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs to 'CALL AFTER EVERY update_task', providing a clear trigger for usage. It also explains what to do with the output ('tasks you should work on next'). However, it does not explicitly mention when not to use it or name alternative tools, so it lacks full when/when-not guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• conclude_analysisA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It discloses that the tool blocks and returns instructions if tasks are pending, and returns a full grouped summary when resolved. However, it does not detail side effects like whether the investigation is permanently closed or tasks become locked, leaving some ambiguity about mutating behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise at three sentences, front-loaded with 'FINAL CALL' to signal importance. Every sentence provides necessary context: purpose, blocking behavior, return value, and usage condition. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool without an output schema, the description covers the essential aspects: what it does, when to call it, and what it returns (with grouping details). It omits specifics about the summary structure or irreversible consequences, but given the tool's simplicity and no schema, this is largely sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the schema is an empty object, making schema coverage 100%. The baseline for 0 params is 4, and no additional parameter description is needed since there is nothing to describe.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies a specific verb and resource: 'Close out the investigation.' It distinguishes itself from sibling task-management tools by being the finalization step, with additional detail about blocking on pending tasks and returning a summary. This makes its unique purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use it: 'Call this when you believe all tasks are done.' It also implies the when-not by explaining that it blocks and returns actionable instructions if tasks are pending, effectively warning against premature invocation. This is clear usage guidance with implied exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• resetA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description takes full responsibility for disclosing behavior. It clearly states the irreversible and destructive nature: 'Irreversible — all tasks and findings are permanently deleted.' This is critical safety information for an agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is composed of three short, information-dense sentences covering the action, irreversibility, and usage context. Every word earns its place with no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool (no parameters, no output schema, no annotations), the description fully covers what the tool does, its consequences, and when to use it. There are no significant information gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has no parameters, and the schema is empty. Per the rules, a baseline of 4 applies for zero parameters. The description adds no parameter information, which is acceptable since there are none.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the action ('Clear all tasks') and the resource ('all tasks and findings'), and establishes the context ('start a new investigation session'). This distinguishes it from sibling task-management tools like add_task or update_task.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool: 'Only use when starting a completely new investigation.' This provides a clear condition and implies it should not be used for other scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.