Server Quality Checklist

Tool Scores

• btcpay_requestC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool executes API operations but doesn't disclose critical traits like authentication requirements, rate limits, error handling, or whether it performs read/write operations. This leaves significant gaps in understanding how the tool behaves in practice.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that clearly states the tool's function without unnecessary words. It is appropriately sized and front-loaded, though it could be slightly more informative without losing conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a generic API tool with three parameters, no annotations, and no output schema, the description is incomplete. It fails to explain what the tool returns, how errors are handled, or provide context on the service-based approach, making it inadequate for effective agent use without additional documentation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the input schema already documents the three parameters (serviceName, methodName, parameters) with descriptions. The description adds no additional meaning beyond what the schema provides, such as examples of common use cases or parameter interactions, resulting in a baseline score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool executes BTCPayServer API operations using a service-based approach, which provides a general purpose but lacks specificity about what resources or actions it handles. It doesn't distinguish from sibling tools like get_method_info or get_service_info, making it somewhat vague about its exact function beyond being a generic API caller.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives. The description mentions a service-based approach but doesn't explain when to choose this over other methods or tools, leaving the agent without clear usage context or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_method_infoC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions retrieving 'detailed parameter requirements and examples,' implying a read-only operation, but fails to specify if this requires authentication, rate limits, or what the output format looks like (e.g., structured data or examples). This leaves gaps in understanding the tool's behavior and constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It is front-loaded and appropriately sized, making it easy to parse quickly. Every part of the sentence contributes to understanding the tool's function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (2 parameters, no output schema, and no annotations), the description is incomplete. It doesn't explain what the output includes (e.g., parameter types, examples, or error cases), nor does it address behavioral aspects like authentication needs. For a tool that retrieves detailed information, more context is needed to guide effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, clearly documenting both required parameters ('serviceName' and 'methodName') with examples. The description adds minimal value beyond this, as it only reiterates the general purpose without providing additional syntax, format details, or context for parameter usage. Baseline 3 is appropriate since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get') and resource ('detailed parameter requirements and examples for a specific service method'), making it easy to understand what it does. However, it doesn't explicitly differentiate from sibling tools like 'get_service_info' or 'btcpay_request', which might have overlapping or related functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It lacks context on prerequisites, such as needing prior knowledge of service names, or comparisons to sibling tools like 'get_service_info', which might offer broader service information. This omission leaves the agent without clear usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_service_infoB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the tool 'discovers' and 'explores', which suggests a read-only operation, but doesn't explicitly state whether it's safe, whether it requires authentication, what format the output returns, or any rate limits. This leaves significant behavioral gaps for an agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise with two clear sentences that directly state the purpose and usage without any wasted words. It's front-loaded with the core functionality and efficiently communicates essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 optional parameter, no output schema, no annotations), the description is reasonably complete for a discovery tool. However, it lacks details about output format and doesn't fully compensate for the absence of annotations, leaving some contextual gaps for an agent trying to use this tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, so the schema already fully documents the single optional parameter. The description doesn't add any additional meaning about parameters beyond what's in the schema, but since schema coverage is high, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with the verb 'discover' and resource 'available BTCPayServer services and their methods', making it easy to understand what the tool does. However, it doesn't explicitly distinguish this from its sibling tools (btcpay_request, get_method_info), which would be needed for a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides some usage guidance with 'Use this to explore what operations are available', which implies this is for discovery purposes. However, it doesn't explicitly state when to use this tool versus its siblings (btcpay_request, get_method_info) or provide any exclusion criteria, leaving room for ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.