Server Quality Checklist

Tool Scores

• get_topicC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits. It does not mention pagination, rate limits, authorization needs, or what 'all documents' entails (e.g., return format, limits). Minimal transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence with no fluff. It efficiently conveys the core purpose, though it could benefit from additional context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description should explain return values. It does not mention whether it returns document IDs, full content, or metadata. Incomplete for a retrieval tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, with a descriptive parameter name and schema description. The tool description adds no extra meaning beyond the schema; baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the resource 'all documents from a specific topic', distinguishing it from siblings like get_document (single document) and list_topics (list names). However, it does not explicitly differentiate itself.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives (e.g., get_document for a single document, search for filtered results). The description only states what it does without usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• searchB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits. It only states a general search without explaining scope (all documents in what context), result limits, pagination, or whether it supports partial matches. This minimal information is insufficient for safe invocation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no extraneous information. It is front-loaded with the core action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema), the description could still be more complete. It does not indicate what the tool returns (e.g., list of documents) or any default behaviors, leaving gaps for the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a clear description for the 'query' parameter. The description adds no further meaning beyond the schema, so it achieves the baseline score. It does not compensate for any missing schema detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action: 'Search across all documents for a query.' It uses a specific verb and resource, and distinguishes from sibling tools like get_document and get_topic, which are for individual retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide any guidance on when to use this tool versus alternatives. It lacks information on appropriate contexts, prerequisites, or exclusions, leaving the agent without usage recommendations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_documentB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description must disclose behavioral traits. It only states the read-like operation but fails to mention if it is read-only, any side effects, required permissions, or response format. The agent is left to assume safe behavior without confirmation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-formed sentence with no superfluous words. It efficiently conveys the core purpose without redundancy, making it easy for the agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple structure (2 params, no output schema), the description is minimally complete. However, it lacks information about return values (e.g., document content, metadata) which would be helpful for an agent. The schema covers parameters well, but the description could enhance completeness by indicating output.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with clear descriptions for both parameters (topic as folder name, document without .md extension). The description adds no additional meaning beyond the schema, so it earns the baseline score of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (Get) and the resource (specific document from a topic). It distinguishes from sibling tools like get_topic (which gets a topic) and search (which returns results), making it easy for the agent to select this tool for retrieving a single document.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No usage guidelines are provided. The description does not specify when to use this tool versus alternatives like get_topic or search, nor does it mention prerequisites or context. The agent receives no guidance on tool selection criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_indexA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations present, so description carries full burden. It describes the tool as returning a main index, but lacks details about the format, scope, or any behavioral constraints. Adequate but not thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single, concise sentence with no extraneous information. Efficiently communicates the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters, no output schema, and no annotations, the description could be more complete by specifying what the index includes (e.g., list of topics, structure). Minimal but sufficient for a simple overview tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Tool has zero parameters, so schema coverage is 100%. Description adds no parameter details, but baseline score of 4 is appropriate for a parameterless tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states verb 'Get' and resource 'main index/overview'. It distinguishes from sibling tools like get_document or list_topics, which focus on specific items rather than an overview.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description implies usage when a broad overview is needed but does not explicitly state when to use this over alternatives. No when-not-to-use or comparison with siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_topicsA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states it lists topics and their documents, but does not disclose behavioral traits like whether it is read-only, whether documents are returned as metadata or full content, or if there are pagination limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is concise and front-loaded with the key verb and resource. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity (0 params, no output schema), the description is somewhat complete but lacks details on what exactly is returned for 'their documents' and whether any implicit constraints apply (e.g., pagination, permissions). It could be improved by mentioning that it returns document references, not full content.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has no parameters, so the schema is effectively empty. The baseline is 4, and the description does not need to add parameter details. It correctly notes the action without redundant parameter info.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('list') and the resource ('all available information topics and their documents'), which differentiates it from sibling tools like 'get_topic' (single topic) and 'get_document' (single document).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies it is used to get a broad overview of topics, but it does not explicitly state when to use it versus siblings like 'search' or 'get_index', nor does it mention when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.