Server Quality Checklist

Tool Scores

• get_spanC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states this is a read operation ('Get'), which implies it's non-destructive, but doesn't address other aspects like authentication needs, rate limits, error handling, or what 'information and metrics' specifically includes. This leaves significant gaps for a tool that likely interacts with tracing data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's function without unnecessary words. It's front-loaded with the core purpose, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It doesn't explain what 'span information and metrics' entails, how results are structured, or any behavioral traits like safety or performance. For a tool in a tracing context with siblings, more detail is needed to guide effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with the single parameter 'span_id' clearly documented in the schema. The description adds minimal value beyond the schema by mentioning 'by span_id', but doesn't provide additional context like format examples or constraints. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get') and target resource ('span information and metrics'), making the purpose understandable. However, it doesn't distinguish this tool from its sibling 'get_trace' or 'get_complete_trace', which likely retrieve related but different data, so it doesn't achieve full differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_trace' or 'get_complete_trace'. It mentions 'by span_id', which implies a prerequisite but doesn't explain how this differs from sibling tools or when other tools might be more appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_traceC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states it 'gets' information, implying a read operation, but doesn't specify whether this requires authentication, has rate limits, returns structured data, or has any side effects. The description is minimal and lacks important operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise - a single sentence that directly states the tool's purpose without any wasted words. It's front-loaded with the core functionality and appropriately sized for a simple retrieval tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations and no output schema, the description is insufficiently complete. It doesn't explain what 'trace information and metrics' includes, the format of the return data, error conditions, or how this differs from sibling tools. The agent would need to guess about the tool's behavior and output.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the single parameter 'trace_id' clearly documented in the schema. The description adds no additional parameter semantics beyond what's already in the schema, so it meets the baseline for adequate but unenhanced parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get') and resource ('trace information and metrics'), making it easy to understand what the tool does. However, it doesn't explicitly differentiate from sibling tools like 'get_complete_trace' or 'get_span', which appear to be related trace/span retrieval operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_complete_trace' or 'get_span'. It doesn't mention prerequisites, constraints, or comparative use cases, leaving the agent to infer usage from tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_complete_traceA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions retrieving 'complete trace information and metrics,' which implies a read-only operation, but doesn't disclose behavioral traits like authentication requirements, rate limits, response format, or potential performance implications of fetching 'ALL data.' The description adds minimal behavioral context beyond the basic purpose.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with two concise sentences that directly state the tool's purpose and usage context. Every sentence earns its place by providing essential information without redundancy or unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (simple single-parameter query), lack of annotations, and no output schema, the description is minimally complete. It covers purpose and usage but lacks details on behavior, response format, or error handling. For a tool with no structured safety or output information, more contextual detail would be beneficial.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with the single parameter 'trace_id' fully documented in the schema. The description adds no additional meaning beyond what the schema provides, as it only repeats 'by trace_id' without explaining format, constraints, or examples. With high schema coverage, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Get complete trace information and metrics') and identifies the resource ('by trace_id'). It distinguishes from siblings by emphasizing 'COMPLETE or ALL data' versus likely partial data from get_span or get_trace. However, it doesn't explicitly name the siblings for comparison.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: 'Reserved for explicit requests for COMPLETE or ALL data.' This implies it should be used when full trace data is needed rather than partial information. It doesn't explicitly name alternatives or state when not to use it, but the context is sufficiently clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• authA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool handles authorization with an API key, including fallback to user input if missing, which is useful behavioral context. However, it doesn't mention what happens after authorization (e.g., does it return a token, set a session, or enable other tools?), rate limits, or error handling, leaving gaps in transparency for a security-related tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences that are front-loaded and waste-free. The first sentence states the core purpose, and the second provides critical usage guidance. Every word contributes to understanding the tool's role and invocation logic, making it highly efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (authorization with security implications), lack of annotations, and no output schema, the description is moderately complete. It covers the basic purpose and usage but omits details like what authorization enables, return values, or error cases. For a tool named 'auth' with siblings focused on data retrieval, more context on its role in the workflow would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with the parameter 'api_key' well-documented in the schema as optional if an environment variable is set. The description adds value by explaining the fallback behavior ('ask the user for the API key') and implying a directory lookup, which provides context beyond the schema's technical details. With only one parameter and high schema coverage, this earns a strong score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool's purpose is to 'Authorize using the AGENTOPS_API_KEY,' which is a clear verb+resource combination. However, it doesn't distinguish this from potential sibling tools (like get_complete_trace, get_span, get_trace) or explain what authorization enables. The purpose is understandable but lacks context about what system or capabilities authorization unlocks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool: 'If the API key is not provided and cannot be found in the directory, ask the user for the API key.' This clearly outlines the fallback behavior and conditions for invocation, making it easy for an agent to decide when this tool is necessary versus relying on environment variables or other methods.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.