Server Quality Checklist

Tool Scores

• query_hrB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing behavioral traits. It only states 'Search', which implies a read-only operation, but provides no details on side effects, authorization needs, rate limits, or output characteristics (e.g., pagination, result limits). This is insufficient for a tool with no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise with two sentences, no fluff. The first sentence states the core purpose, the second suggests filter usage. It is appropriately sized for a simple search tool, though it could be slightly more detailed without losing conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 3 parameters, no output schema, and no annotations, the description covers the basic search functionality but lacks details on return format, result limits, or any behavior beyond the search itself. It is adequate but not complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds that filters provide 'more targeted results', which provides context but does not add significant meaning beyond the schema's parameter descriptions and examples. The value is marginal.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches the 'HR Knowledge Base', specifying the verb (Search) and resource (HR Knowledge Base). It mentions optional filters, which adds specificity. However, it does not explicitly differentiate from sibling tools like query_compliance or query_legal, which have similar structures but different domains.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description advises using optional filters for targeted results, implying usage context. However, it lacks explicit guidance on when to use this tool versus alternatives, and does not mention any prerequisites or when it should not be used.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_complianceB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It only mentions searching but does not disclose whether it is read-only, how results are returned, pagination, or any side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences convey purpose and guidance without any unnecessary words. Perfectly concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and moderate parameter count, the description lacks information about return structure or behavioral details, making it incomplete for confident use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds minor context about optional filters but does not significantly expand on schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Search the Compliance Knowledge Base for relevant information,' which includes a specific verb (search) and resource (Compliance Knowledge Base). It distinguishes from sibling tools like query_hr and query_legal which target different domains.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description says 'Use optional filters for more targeted results,' implying when to use optional parameters but does not explicitly state when to use this tool versus alternatives or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_legalB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears full responsibility for behavioral disclosure. It only indicates a search (read) operation but lacks details on side effects, rate limits, authentication, or data source characteristics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that front-loads the primary action. It is concise without waste, though more context could be added without harming conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and 4 parameters, the description adequately defines the tool's purpose but does not address return format, pagination, or result behavior. It is acceptable but has gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema covers 100% of parameters with descriptions and examples. The description only says 'Use optional filters' which adds no new meaning beyond the schema. Baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Search' and the resource 'Legal Knowledge Base', distinguishing it from sibling tools like query_compliance and query_hr by domain. It is specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions using optional filters for targeted results, implying usage when legal information is needed. However, it does not explicitly state when to use tools like query_compliance or query_hr instead, nor does it provide when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_knowledge_servicesA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description accurately indicates a read-only list operation. No additional behavioral details are necessary for this simple tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that is concise and front-loaded, with no wasted words. It earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a parameterless tool with no output schema, the description fully suffices. It tells the agent exactly what to expect.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so the schema already covers all. The description adds no extra parameter information, but the baseline of 4 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb (list) and resource (knowledge base services), distinguishing it from sibling query tools. It is specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for listing services before querying, but does not provide explicit when or when-not scenarios. Siblings are query tools, but no direct comparison is made.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.