Server Quality Checklist

Tool Scores

• execute_capabilityA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations provided, so the description must carry the full burden of behavioral disclosure. 'Execute' implies an action that may have side effects, but the description does not mention what the execution does, whether it returns a result, requires authentication, or is irreversible. It provides no behavioral context beyond the action itself.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence of 12 words. It starts with the verb 'Execute' and immediately conveys the primary purpose. There is no waste, redundant phrasing, or irrelevant detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations, no output schema, and a generic execution operation, the description is too sparse. It does not explain the execution behavior, what a successful execution looks like, potential errors, or how this fits into the workflow with sibling tools (e.g., obtaining the schema first). It is minimally sufficient for a basic understanding but lacks critical context for safe and correct usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With a schema description coverage of 0%, the description compensates by explaining the roles of the parameters: 'key' identifies the capability and 'input' must match the capability's schema. This adds meaningful constraint beyond the raw schema types, particularly the requirement that the input object conforms to a specific schema. It does not elaborate on the input structure, but the key relationship is clearly conveyed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action: 'Execute a capability by key' with the input schema constraint. It uses a specific verb (execute) and resource (capability), and the 'by key' mechanism distinguishes it from sibling tools like search_capabilities (searching) and get_capability_schema (retrieving schemas).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by saying 'with input matching its schema,' suggesting that users may need to obtain the schema first (via get_capability_schema). However, it does not explicitly state when to use this tool versus alternatives, nor any prerequisites or exclusions. Usage guidance is minimal and only implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_capability_schemaA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden of behavioral disclosure. It only states 'Get the input schema', which adds no context about read-only guarantees, error behavior, or return format. The output schema exists but is not described in the text, leaving the tool's behavior under-specified.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one concise sentence that is front-loaded with the action and resource. It contains no filler and every word contributes to understanding the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter getter tool, the description adequately covers purpose and key source. An output schema exists, which presumably documents the return structure. It lacks details on failure scenarios, but overall the description is complete enough for the tool's low complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It clarifies that 'key' is a capability key returned by search_capabilities(), giving meaningful semantics. However, it does not describe the key's format or provide examples, though for a single string parameter this is somewhat acceptable.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Get' and names the resource 'input schema for a capability key', clearly distinguishing this from sibling tools search_capabilities (which searches) and execute_capability (which executes). It tells the user exactly what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states the key is 'returned by search_capabilities()', implying the user should first search and then retrieve the schema for a resulting key. This gives clear context on when to use the tool, though it does not explicitly mention when not to use it or how it relates to execute_capability.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_capabilitiesA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that results are short summaries, that this is the only discovery method, and that capability keys are not guessable. It implies a read-only search operation. It does not mention error handling or pagination, but the core behavioral traits are well covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with the primary verb and resource. Every sentence adds essential information: what it does, how to use it, and the follow-up workflow. There is no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that an output schema exists (so return values need not be described), the description is complete for a search/discovery tool. It explains the critical usage context, the ordering of calls, and the nature of results, making the tool self-contained enough for an agent to execute correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It implicitly explains the 'query' parameter through 'keyword or natural language', but the 'limit' parameter is not addressed at all. The schema only shows a default of 10, leaving the agent to guess its meaning and effect.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches the capability catalog by keyword or natural language. It distinguishes itself from siblings by indicating it returns short summaries and is the sole discovery mechanism for capability keys, setting up the workflow for get_capability_schema and execute_capability.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance is given: 'Always call this first' and explains that capability keys are not guessable and not listed elsewhere, making this the only entry point. It also tells the agent to call get_capability_schema before execute_capability, providing a clear when-to-use and sequence.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.