Server Quality Checklist

Tool Scores

• create_taskC
  2.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only says 'create a new task,' which implies a mutation but does not disclose side effects, success/failure indicators, permissions required, or whether existing tasks are affected. The description is minimal and does not go beyond the obvious semantics of creation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise (one sentence), but it is also under-specified. It sacrifices essential contextual information for brevity, resulting in an incomplete tool definition. The front-loading of 'create a new task' is clear, but the rest of the necessary details are absent, making the conciseness ineffective.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of annotations, no output schema, and 4 parameters with zero schema descriptions, the description must provide comprehensive guidance, but it does not. It omits required-field information, parameter semantics, return values, and any operational constraints. For a tool with this level of complexity, the description is severely inadequate, leaving an agent with insufficient information to call it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, so the description must compensate by explaining parameter meanings. However, the description does not mention any of the four parameters (title, assignee, priority, tags) or their required status. It provides no value in helping the agent understand what each parameter does or what values are expected, leaving the agent to rely solely on the schema's names and types.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb ('create') and resource ('task in the tracker'), which distinguishes it from siblings like list_tasks and get_task. It is unambiguous that this tool creates a new task, but it does not elaborate on any unique capabilities or constraints beyond the basic action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives, nor does it mention any prerequisites or exclusions. It only states what the tool does, leaving the agent to infer that creation is appropriate for new tasks without any context about how it relates to listing or retrieving tasks.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_taskB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden of behavioral disclosure. It reveals no return format, error behavior, side effects, or authorization requirements; only that this is a getter. An agent is left guessing what will happen when a task is not found or the ID is malformed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no wasted words. It is appropriately sized for a one-parameter getter, though it could have contributed a little more detail in the same space.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-id getter, the description is almost sufficient, but it omits any mention of the return value, absence behavior, or how this tool relates to list_tasks/create_task. With no output schema, those gaps make the description a bit thin.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description's 'by id' merely restates the schema's id property without adding any semantics (e.g., id format, whether it's globally unique, or how it is composed). The tool name plus schema already convey most of the meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description says 'Get a single task by id' – a specific verb (get) plus a resource (task) plus a distinguishing bound (single/by id). This clearly distinguishes it from sibling tools like list_tasks (list multiple) and get_doc (different resource).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no explicit guidance on when to use this tool instead of list_tasks or search_docs, and no mention of prerequisites or exclusions. The 'by id' phrasing only implies a usage condition rather than stating it directly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_tasksA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It names the operation and filters, but does not state whether results are paginated, ordered, or limited, what task fields are returned, or any scope/authorization context. For a tool with no output schema or annotations, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single front-loaded sentence that starts with the verb and resource, then adds the filter conditions. It contains no filler or redundant information, making it an efficient definition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with two optional parameters and no nested objects, and the description covers the resource and filters. However, with no output schema and no annotations, it leaves the return shape, pagination and ordering behavior, and default result set unspecified — the main missing context an agent would need.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description compensates by explicitly explaining both parameters as optional filters ('status or assignee'). It adds the semantic role of each parameter and their optionality, which goes beyond the bare schema property definitions, though it lacks formatting examples beyond the enum values.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('List') and resource ('tasks from the team tracker'), and identifies the optional filters by status or assignee. This makes the tool's purpose unmistakable and distinguishes it from siblings like get_task (single task retrieval) and create_task (creation).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies this tool is for retrieving a collection of tasks, with optional filtering, but it does not explicitly name alternatives or state when not to use it. There is no guidance about using get_task for a specific task or search_docs for document searches, so the routing is left to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_docA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It conveys that the operation is a read (fetching full content), but it does not describe error handling, authentication requirements, rate limits, or what happens for invalid IDs. For a simple read operation, this is adequate but not exhaustive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, tightly-packed sentence with no filler. It front-loads the core purpose ('Fetch the full content of a doc by id') and then adds the source context. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with a single parameter and no output schema, the description is complete enough. It specifies the resource, the parameter's provenance, and the action. Given the low complexity, nothing critical is missing for an agent to call it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema only defines 'id' as a required string with no description. The tool description adds critical semantic context by specifying that the id comes from search_docs results, which tells the agent how to obtain a valid id. This goes beyond what the schema alone provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: fetching the full content of a doc by id. It explicitly ties the id source to search_docs results, which distinguishes it from other sibling tools like get_task or search_docs itself. The verb 'Fetch' and resource 'doc' are specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a usage context by saying 'from search_docs results,' suggesting it is a follow-up operation. However, it does not explicitly state when to use this tool vs. alternatives, nor does it mention any exclusions. The guidance is implicit rather than explicit, leaving the agent to infer the intended workflow.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• review_checkA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the behavioral burden. It explains the action and the heuristic categories, and implies a read-only analysis. However, it does not disclose what the tool returns, whether it can modify anything, or how results are reported. This is a moderate gap for a tool with no output schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single dense sentence with no filler. The main action is front-loaded, the check categories are compactly listed, and the purpose and reference to the team checklist are included without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple, with one parameter and no annotations or output schema. The description covers input scope and intended timing well. However, because there is no output schema, the description should have said at least something about what the caller receives after the checks run. That omission keeps it from being fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema only declares 'text' with no description, so the description must compensate. It does by clarifying that the text can be generated content or a diff, and by linking the checks to that input. It does not specify formatting or size limits, but for a single parameter this is meaningful added context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Run heuristic checks' over 'generated text or a diff', and lists concrete check categories (secrets, TODOs, placeholder text, size). It clearly distinguishes itself from the unrelated task and document sibling tools by focusing on pre-review quality validation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly defines when to use the tool: 'before it goes to a human reviewer'. It also scopes the input to generated text or a diff, which gives clear context. It does not name alternatives, but the sibling tools are unrelated, so no exclusion is necessary.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_docsA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the transparency burden. It discloses that this is a stand-in for a real Confluence-style API and what it searches, but it does not describe result format, query syntax, or limitations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences with no filler. The core action and scope are front-loaded, and the stand-in note adds useful context without bloating the text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter search tool, the description is mostly adequate, but with no output schema it should mention what the tool returns (e.g., matching document IDs or snippets). The lack of any return-value information leaves a meaningful gap for an agent deciding how to use the result.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% and the only parameter is 'query', which is just a string type. The description adds meaning by explaining this is full-text search over specific document categories, clarifying how the query parameter is used.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool does full-text search over internal docs/wiki, with concrete content examples (style guide, runbooks, checklists). This distinguishes it from siblings like get_doc, which implies direct retrieval rather than search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: this is for searching internal documentation rather than managing tasks or products. It does not explicitly name alternatives or exclusions, but the search-over-documents framing makes the intended use obvious enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.