Server Quality Checklist

Tool Scores

• list_callsA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It indicates this is a read operation ('List') and specifies default and max values for 'limit', which adds useful context. However, it lacks details on permissions, rate limits, pagination, or error handling, leaving gaps in behavioral understanding for a tool with filtering capabilities.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with the core purpose, followed by clear sections for arguments and returns. Every sentence adds value without redundancy, and the bullet-point style in the text enhances readability. It efficiently conveys necessary information in a compact format.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (3 parameters, filtering functionality) and the presence of an output schema (implied by 'Returns: JSON array'), the description is reasonably complete. It covers input semantics and output format, but lacks behavioral context like error cases or usage comparisons with siblings, which holds it back from a perfect score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must compensate. It adds meaningful semantics beyond the schema: it explains that 'company' uses partial match and is case-insensitive, 'deal_stage' includes example values, and 'limit' has a default and max. This clarifies usage significantly, though it doesn't cover all potential nuances like format constraints for parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'List available call transcripts with optional filtering.' It specifies the verb ('List') and resource ('call transcripts'), making the function unambiguous. However, it doesn't explicitly differentiate from sibling tools like 'get_call' (likely for single call) or 'search_calls' (possibly more advanced search), so it doesn't reach the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'search_calls' or 'get_call'. It mentions optional filtering but doesn't specify scenarios where this tool is preferred over siblings, leaving the agent to guess based on tool names alone. This lack of comparative context reduces its effectiveness.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_callsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds some context: the search is case-insensitive and covers all transcript content, and it specifies default and max values for 'limit'. However, it lacks details on permissions, rate limits, or error handling, which are important for a search operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with the purpose, followed by clear sections for 'Args' and 'Returns'. Each sentence adds value without waste, making it easy for an agent to parse and understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no annotations), the description is fairly complete. It covers purpose, parameter semantics, and return format. However, it lacks usage guidelines and some behavioral details like error handling. The presence of an output schema reduces the need to explain return values, but gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds significant meaning beyond the input schema, which has 0% description coverage. It explains that 'query' is case-insensitive and searches all transcript content, and specifies 'limit' as maximum excerpts with default and max values. This fully compensates for the schema's lack of documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Search for keywords across all call transcripts.' It specifies the verb ('search') and resource ('call transcripts') with scope ('all'). However, it doesn't explicitly differentiate from sibling tools like 'get_call' or 'list_calls', which prevents a score of 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_call' or 'list_calls'. It mentions searching 'all call transcripts' but doesn't clarify use cases, prerequisites, or exclusions, leaving the agent without contextual direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_callA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states what the tool returns ('Full transcript content including metadata and transcript text'), which is helpful, but doesn't cover other important aspects like whether this is a read-only operation, potential rate limits, authentication needs, error conditions, or what happens if the call_id is invalid. For a tool with zero annotation coverage, this leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with the core purpose in the first sentence, followed by clear sections for Args and Returns. Every sentence earns its place by providing essential information without redundancy, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no nested objects) and the presence of an output schema (which handles return values), the description is mostly complete. It covers the purpose, parameter meaning, and return content. However, it lacks behavioral details like error handling or operational constraints, which would be beneficial even with an output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful context for the single parameter by explaining that call_id is 'The unique identifier for the call', which clarifies its purpose beyond what the schema provides (schema coverage is 0%, so the schema only gives the title 'Call Id' and type). This adequately compensates for the low schema coverage, though it doesn't detail format or constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Get the full transcript content') and resource ('for a specific call'), distinguishing it from sibling tools like list_calls and search_calls which handle multiple calls or search operations rather than retrieving detailed content for a single call.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by specifying it's for 'a specific call' with a call_id, suggesting it should be used when you have the identifier for a particular call. However, it doesn't explicitly state when to use this versus alternatives like list_calls or search_calls, nor does it mention prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.