Server Quality Checklist

Tool Scores

• search_documentsC
  2.4/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the full burden of behavioral disclosure, and it barely restates the tool's function. It offers no insight into result formats, pagination, authentication, rate limits, or any side effects. This is essentially tautological.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise (six words), but it is under-specified and essentially restates the tool name. It is not structured or front-loaded with useful information beyond the basic action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and only a minimal description, the tool is not adequately contextualized. The agent has no information about what results to expect, how many matches are returned, or any limitations. This is a significant gap for a search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides 100% coverage: both 'query' and 'searchableType' have descriptions. The tool description adds no further meaning, but since the schema already documents the parameters, a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Search for Aha! documents' clearly states a specific action (search) and a resource (Aha! documents). It is unambiguous but does not differentiate from sibling tools like get_page or get_record, which could also be used to retrieve documents.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There is no mention of what kind of documents are searched, how search results differ from direct retrieval, or any exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_pageC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, leaving the description responsible for behavioral disclosure. It implies a read operation via 'Get' but does not confirm safety, permissions, error behavior, or what 'relationships' actually entails. The vague 'optional relationships' adds little transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, short sentence that is front-loaded with the primary purpose. However, 'optional relationships' is imprecise and could be more specific, slightly reducing its effectiveness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is too thin. It does not explain what an Aha! page is, what relationships are available beyond the includeParent flag, or how the tool behaves in error cases. More context is needed for an agent to invoke it confidently.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with both parameters documented. The description adds 'by reference number' and 'optional relationships', but these are already implied or covered by the schema. It provides no additional syntax or format details beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool gets an Aha! page by reference number, identifying both the resource type and the lookup method. However, it does not explicitly differentiate from the sibling get_record, making it less distinct than ideal.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no guidance on when to use this tool versus alternatives like get_record or search_documents. It does not state any prerequisites, exclusions, or scenarios where this tool is preferred.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_featureB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden but only states 'Create a new feature in Aha!'. It discloses that this is a write operation, but nothing about required inputs, potential side effects, or return behavior. Since there is no output schema, this is inadequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence. It is minimal but not padded, achieving maximum brevity and clarity. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 17 parameters and no output schema, this one-sentence description provides almost no contextual completeness. It does not explain what a feature is, how it relates to a release, or what the tool returns. The agent is left to infer everything from the schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All 17 parameters are fully described in the input schema, so the description's lack of parameter detail is acceptable. The schema provides definitions for each field, including required ones, meeting the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Create' and names the resource 'feature' within 'Aha!', clearly distinguishing it from sibling read/search tools. The purpose is unambiguous and immediately understood.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives, nor any prerequisites like needing a release_id. The description is a bare statement with zero usage context, leaving the agent to infer when this tool should be invoked.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_recordA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full responsibility for disclosing behavior. It only states that it 'gets' a record, which is already evident from the name. It does not mention error handling (e.g., what happens if the reference is not found), return format, or any side effects, leaving the agent with limited understanding of the tool's runtime behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one concise sentence, front-loaded with the action and resource. It contains no redundant information and earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with a single, well-described parameter and no output schema, the description is mostly sufficient. It tells the agent what the tool does and how to specify the input. However, it lacks any mention of the return value or possible response structure, which would be helpful for an agent to interpret results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already includes a description for the single parameter 'reference' with examples (e.g., DEVELOP-123 or ADT-123-1), so the description adds little beyond what the schema provides. The tool description's phrase 'by reference number' reinforces the parameter's usage but does not offer new details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool gets an Aha! feature or requirement by reference number, using a specific verb and resource. It distinguishes itself from siblings like get_page (pages), search_documents (search), and create_feature (create).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool should be used when you have a reference number to fetch a specific record, but it does not explicitly mention alternatives or when not to use it. There is no comparison with sibling tools or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.