Server Quality Checklist

Tool Scores

• task_infoC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states this is a read operation ('Returns'), which is helpful, but doesn't cover aspects like whether it requires authentication, rate limits, error handling, or what 'full details' includes (e.g., format, fields). This leaves significant gaps for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no wasted words, making it easy to parse. However, it could be more front-loaded with key distinctions from siblings to improve structure, but it's appropriately sized for its purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a read operation with no annotations and no output schema, the description is incomplete. It doesn't explain what 'full details' means in terms of return values, error cases, or behavioral traits, leaving the agent with insufficient context to use the tool effectively beyond basic parameter input.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with the parameter 'taskIDs' well-documented in the schema as 'A list of task IDs to retrieve information for'. The description adds no additional meaning beyond this, such as ID format examples or constraints, so it meets the baseline for high schema coverage without compensating value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Returns full details for requested tasks' clearly states the verb ('Returns') and resource ('full details for requested tasks'), but it's somewhat vague about what 'full details' entails and doesn't differentiate from sibling tools like 'current_task' or 'decompose_task'. It's better than a tautology but lacks specificity about what makes this tool distinct.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'current_task' (which might get current task info) or 'decompose_task' (which might break down tasks). There's no mention of prerequisites, context, or exclusions, leaving the agent to infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• current_taskB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool returns a list of in-progress tasks, which implies a read-only operation, but doesn't cover aspects like authentication needs, rate limits, error handling, or the format of the returned list. For a tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that directly states the tool's function without any unnecessary words. It is front-loaded with the core action ('Returns'), making it efficient and easy to parse. Every part of the sentence earns its place by conveying essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no output schema, no annotations), the description is minimal. It states what the tool does but lacks context on usage, behavioral traits, or output details. For a tool that returns a list, without an output schema, the description should ideally hint at the return format or structure, but it doesn't, leaving gaps in completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description doesn't add parameter details, which is appropriate here. A baseline of 4 is applied for zero parameters, as the schema fully handles the parameter semantics without requiring description input.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Returns') and resource ('list of tasks that are currently in progress'), making the purpose evident. It distinguishes from siblings like 'create_task' (creation) and 'update_task' (modification) by focusing on retrieval of in-progress tasks. However, it doesn't explicitly differentiate from 'task_info', which might also retrieve task details, leaving some ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when to choose 'current_task' over 'task_info' (which might get task details) or other siblings, nor does it specify prerequisites or contexts for usage. This lack of explicit when/when-not instructions limits its utility for an AI agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_taskA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that tasks start in 'todo status' and mentions prerequisites (decompose_task for complexity, update_task before/after execution), which adds useful context. However, it doesn't cover behavioral aspects like error handling, permissions, or what happens on creation (e.g., does it return an ID?). The description adds some value but leaves gaps for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is 4 sentences but could be more front-loaded. The first sentence states the purpose, but the subsequent sentences about decomposition and update_task usage, while important, might be better structured. It's not overly verbose, but the flow could be improved for clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description provides good usage guidelines and some behavioral context (starting status, prerequisites). However, for a mutation tool with 7 required parameters and complex nested objects, it lacks details on what happens after creation (e.g., success response, error cases). The description is adequate but has clear gaps in completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 7 parameters thoroughly. The description adds no parameter-specific information beyond what's in the schema. According to the rules, with high schema coverage (>80%), the baseline is 3 even with no param info in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Creates a new task that must be executed.' It specifies the verb ('creates') and resource ('task'), but doesn't explicitly differentiate from siblings like 'update_task' or 'decompose_task' beyond mentioning them. The purpose is clear but lacks direct sibling comparison.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidelines: 'If decomposing a complex task is required, must use 'decompose_task' first before executing it' and 'Must use 'update_task' before executing this task, and when executing this task has finished.' It clearly states when to use alternatives (decompose_task for complex tasks) and prerequisites (update_task before and after execution).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_taskA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the tool is for updates (implying mutation) and includes operational rules (e.g., include lessons learned, batch updates), but doesn't cover permissions, error handling, or response format. It adds some context but lacks comprehensive behavioral traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, followed by usage guidelines. Each sentence adds value (purpose, timing, content, efficiency), but the second sentence could be more concise (e.g., 'Use before and after execution'). Overall, it's efficient with minimal waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and a complex input schema (multiple nested objects), the description is moderately complete. It covers purpose and usage well but lacks details on behavioral aspects like side effects, error conditions, or return values, which are important for a mutation tool with rich parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents the 'tasks' parameter and its nested properties. The description adds no parameter-specific details beyond implying updates to 'status and/or other properties', which is already covered in the schema. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Updates') and resource ('status and/or other properties of one or more tasks'), making the purpose specific. It distinguishes from siblings like 'create_task' (creation) and 'task_info' (read-only), but doesn't explicitly differentiate from 'decompose_task' (which modifies tasks differently).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: 'Must use this tool before executing tasks, and when finished executing tasks' (timing), 'Should always include lessons learned' (content requirement), and 'Always update multiple tasks in a single call if dependencies allow it' (efficiency rule). This gives clear when-to-use directives without naming alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• decompose_taskA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behavioral traits: the tool creates subtasks (implying mutation), specifies prerequisites (complexity > low, todo status), explains parallel execution logic ('Subtasks with the same sequence order may be executed in parallel'), and notes recursion ('Created subtasks may be decomposed later if needed'). It lacks details on permissions or error handling, but covers substantial operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose in the first sentence, followed by essential guidelines and behavioral notes in concise, bullet-like statements. Every sentence earns its place by providing critical operational rules without redundancy or fluff, making it highly efficient for an agent to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (mutation with multiple behavioral rules) and lack of annotations or output schema, the description does a strong job covering usage context, prerequisites, and behavioral traits. It explains the decomposition logic, parallel execution, and recursion, though it doesn't detail the response format or error scenarios, leaving minor gaps for a mutation tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters thoroughly. The description adds no specific parameter semantics beyond what the schema provides (e.g., it doesn't clarify format or constraints for taskID, decompositionReason, or subtasks). The baseline score of 3 reflects adequate coverage by the schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Decomposes an existing complex task') and resource ('into smaller, more manageable subtasks'), distinguishing it from sibling tools like create_task, update_task, or task_info which handle different task operations. The verb 'decomposes' precisely indicates breaking down rather than creating or modifying.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use criteria: 'All tasks with complexity higher than low must always be decomposed before execution' and 'Tasks MUST be in todo status to be decomposed.' It also implies alternatives by specifying prerequisites, guiding the agent away from using this tool for simple tasks or tasks not in todo status.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.