Server Quality Checklist

Tool Scores

• task_addA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits. It does so by specifying the return format (compact JSON) and noting that the source is whatever the server actually stored, which informs the agent of potential variability. It also mentions the source is first-class, referencing module docstring, which adds context. However, it does not discuss side effects, error states, or idempotency, but given it's a creation endpoint, the info is reasonably transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, two sentences, and front-loaded with the core purpose. Every sentence adds value: the first states the action and endpoint, the second clarifies the response format and a behavioral nuance. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool is simple (3 required params, no nested objects), and there is an output schema (though not provided in the input), the description is fairly complete. It covers the endpoint, source handling, and return format. It could mention potential errors or idempotency, but for a POST creation endpoint, it covers the essentials. The absent annotation makes it carry more weight, but it handles that well.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description must compensate. It explains that the 'source' parameter (implied by the endpoint) is set to 'mcp' and describes the response, but it does not provide specific details on 'title', 'description', or 'repo_path' beyond what's in the schema. However, it adds value by clarifying the 'source' semantics and response structure, which is useful for the agent. Since all parameters are required and self-explanatory from their names, the description's addition is reasonably helpful.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: creating a no_human task via a specific endpoint and source. It clearly distinguishes from the sibling tool 'task_status' which presumably handles status queries, by focusing on task creation. It uses a specific verb ('Create') and specifies the resource ('task'), making it clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool (to create a no_human task) but does not explicitly contrast with alternatives. Since there is only one sibling and no elaboration on when not to use it, it only provides moderate guidance. It does not mention any prerequisites or conditions for use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• task_statusA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully carries the burden and discloses important non-obvious behavior: resolution by task id or unique prefix, fallback to a client-side external_id scan, and the reason for that fallback. This gives the agent accurate expectations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences with no filler. Each sentence adds functional value: the endpoint, the lookup strategy, and the return format.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the single parameter, presence of an output schema, and absence of annotations, the description is sufficiently complete. It covers the lookup method, fallback behavior, and return representation without needing to describe output schema fields.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides only the parameter name and type with no description, and schema description coverage is 0%. The description compensates fully by explaining that the parameter accepts a task id, unique id prefix, or external_id and by detailing the resolution order.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool fetches a task's full current state via a specific endpoint. It uses a precise verb and resource, and the read-oriented purpose distinguishes it from the sibling task_add.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear context for when to use the tool: whenever a task's current state is needed. It does not explicitly name alternatives or exclusions, but the intended use is evident.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.