Server Quality Checklist

Tool Scores

• escalate_to_humanC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It only states 'Queue a request' without detailing what happens after queuing, whether it is synchronous, or if any confirmation is returned. The term 'Queue' hints at asynchronous handling but lacks specificity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, front-loaded with the action, and contains no filler words. While under-specified, the efficient structure earns a high score for conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (2 params, no enums) and has an output schema, but the description omits critical usage context such as when to escalate, how priority affects queuing, and what happens to the request. The output schema may cover return values, but the description alone is incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description does not mention the summary or priority parameters. With 0% schema description coverage, the agent receives no guidance on what content to include in summary or what priority values are valid.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the verb 'Queue' and identifies the resource as 'a request for a human staff member', clearly indicating the action. It is distinct from siblings like search_service_knowledge and collect_intake, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to escalate versus using search_service_knowledge or collect_intake. No prerequisites, scenarios, or exclusion criteria are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_service_knowledgeB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It does mention returning citations, which is useful, but lacks context about authentication, result limits, error behavior, or the meaning of 'approved' knowledge. The read-only nature of a search is implied but not stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is immediately front-loaded with the verb and purpose. Every word earns its place, and there is no redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While an output schema exists, the description lacks essential context such as when to use this tool versus alternatives, and what the locale parameter does. For a search tool with two parameters, the description is under-specified and leaves the agent to guess about locale semantics and search scope.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does not explain the 'query' or 'locale' parameters beyond their names. 'Query' is somewhat self-evident, but 'locale' is unexplained and could significantly affect results. The description adds no value over the bare parameter names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Search') and clearly identifies the resource ('approved service-business knowledge') and outcome ('return citations'). It distinguishes itself from sibling tools like collect_intake and escalate_to_human, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for searching knowledge, but does not explicitly state when to use this tool versus the sibling tools, nor any exclusions or prerequisites. There is no mention of 'use this before escalating' or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• collect_intakeA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the transparency burden. It discloses a key behavioral trait: the tool does not claim appointment confirmation. This is valuable, but it omits other potential behaviors like data persistence, return format, or side effects. Still, it adds meaningful beyond-schema context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence that front-loads the core action and includes a key nuance. No wasted words, though it omits potentially useful details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with four self-explanatory parameters and an output schema (not shown). The description is enough to understand the basic operation but lacks details about what happens after collection, any constraints on input values, or how the returned data is structured. Sibling tools indicate there are other pathways, but the description does not connect to them.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% and the description does not explain any of the four parameters (name, callback_number, service, preferred_time). The phrase 'scheduling details' vaguely maps to these fields but adds no specific semantics beyond what the parameter names already convey. The description fails to compensate for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Collect scheduling details' with a specific verb and resource. It also adds a distinguishing constraint ('without claiming that an appointment is confirmed'), which helps differentiate it from other tools like escalate_to_human or search_service_knowledge.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool is used when gathering scheduling information and explicitly notes it does not confirm appointments, providing some usage context. However, it does not mention alternatives like search_service_knowledge or escalate_to_human, nor does it state when not to use this tool, so the guidance is only implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.