Server Quality Checklist

Tool Scores

• get_build_statusA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description bears full responsibility for behavioral disclosure. It does state that the tool 'returns' status information and lists specific return fields, implying a read-only operation. However, it omits details about potential delays, caching, authentication, or error conditions that might be relevant for an agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that immediately states the purpose and the included output fields. Every word contributes meaning, and there is no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one optional parameter and no output schema, the description adequately conveys what the tool does and what the response contains (last build result, timestamp, duration). It could slightly benefit from mentioning possible build result values or status meanings, but overall it is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema fully documents the single optional 'job' parameter with a clear default and description. The tool description adds no extra parameter semantics, so the baseline 3 applies without any need for compensation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Returns' and clearly identifies the resource as 'the documentation CI/CD build pipeline in Jenkins'. It also lists the key output fields (build result, timestamp, duration), which makes the tool's function unambiguous and distinctly different from the sibling content tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context (checking build status) but provides no explicit guidance on when to use this tool versus alternatives, nor any exclusions or prerequisites. Sibling tools are clearly different in domain, so the lack of explicit guidance is not critical, but it is still a gap.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_topicsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full behavioral disclosure burden. It states the operation and filtering options, but does not mention result format, ordering, pagination, or any side effects. The basic read-only nature is implied, yet additional behavioral context is absent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that conveys both purpose and filtering parameters without any redundant or vague wording. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a low-complexity list tool with two optional enum parameters and no output schema, the description is almost complete. It clearly states what the tool returns and which filters apply. A minor gap is lacking explicit mention of the default behavior when no filters are provided, though the schema's 'default: all' partially covers this.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no parameter descriptions, so the description must compensate. It does name both parameters as optional filters ('optionally filtered by product or topic type'), adding some meaning beyond the raw property names. However, it does not explain the enum values or the meaning of the 'all' default, leaving part of the semantics to inference.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and resource ('documentation topics'), and clearly defines the scope ('all available'). It distinguishes itself from sibling tools like search_docs and get_topic by focusing on listing topics rather than searching or retrieving a single topic.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description conveys when to use the tool: to enumerate all documentation topics, with optional filtering by product or topic type. It does not explicitly name alternative tools or exclusions, but the context is clear and no misleading guidance is present.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_content_gapsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It discloses the core behavior (returns zero/low-result queries) and the interpretative aspect (gaps in documentation). It doesn't mention read-only or data freshness, but for a simple analytics getter, this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core action and followed by a clear use case. Every phrase earns its place; no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with optional params and no output schema, the description adequately explains the tool's purpose and value. It doesn't describe the return format, but the output (search queries) is inferable from the description and schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers all three parameters with descriptions and defaults, so baseline is 3. The tool description adds no extra parameter context, but none is needed; parameters like 'days' and 'min_searches' are self-explanatory in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns search queries from analytics data with zero or low results, specifically identifying topics users searched but documentation doesn't cover. This specific verb+resource+outcome distinguishes it from siblings like search_docs or list_topics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use this to identify missing documentation,' giving a clear context. It doesn't name alternatives or exclusions, but the distinct purpose makes when-to-use implicit. It could have been stronger by contrasting with related tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_topicA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits. It clearly states the tool returns Markdown content, which conveys the output format. However, it lacks details about error handling (e.g., invalid topic_id), potential size limits, or any permission requirements, leaving some behavioral ambiguity for a read operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, with the primary action stated in the first sentence and supplementary usage guidance in the second. Every word contributes value, with no redundancy or unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter read tool, the description covers the essential steps: how to obtain the parameter and what the tool returns. It lacks some edge-case details like handling of missing IDs or content formatting nuances, but given the low complexity and clear sibling context, it is nearly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% as the only parameter topic_id is described as 'Topic ID returned by search_docs.' The description adds the context of searching first but does not provide additional parameter details beyond what the schema already states, so it meets the baseline without exceeding it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Retrieve the full Markdown content') on a specific resource ('a specific documentation topic by its ID'), making the purpose immediately clear. It also distinguishes itself from sibling tools by explicitly referencing search_docs for finding the topic_id, which prevents confusion.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit workflow guidance: 'Use search_docs first to find the topic_id, then call this to read the full content.' This tells the agent exactly when to use this tool and how it fits in sequence with search_docs, effectively differentiating it from alternative approaches.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_docsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of disclosing behavior. It mentions that the search is 'semantic' and that results include product, version, and topic-type metadata, giving the agent a clear expectation of output contents. It does not cover edge cases like empty results or rate limits, but the core behavior is transparently described.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exactly two sentences, front-loaded with the main purpose. The first sentence states what the tool does, and the second gives usage guidance and describes the result. There is zero redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 4 parameters and no output schema, the description provides sufficient context: the purpose, the return contents (top matching sections with metadata), and a practical usage hint ('answer questions'). The schema handles parameter details, so the description fills the necessary gaps without being exhaustive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers all 4 parameters with descriptions, so parameter semantics are already well-defined. The description adds no additional parameter-specific context, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Semantic search across Actian documentation.' It specifies the resource (Actian documentation), the action (search), and the output (top matching sections with metadata). This distinguishes it from sibling tools like get_topic and list_topics, which are for direct retrieval or listing, not semantic search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use this to answer questions about Actian products,' providing a clear use case. However, it does not mention alternatives or exclusions (e.g., when to use get_topic or list_topics instead), so it misses the 'when-not' guidance for a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.