Server Quality Checklist

Tool Scores

• run_delegationC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses return values ('summary, prompt_path, transcript_path'), which adds context beyond the input schema, but fails to describe behavioral traits like whether it's read-only, destructive, requires authentication, has side effects, or rate limits. For a tool named 'run', this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the main action and resource, followed by return values. It's concise with two clauses, but the second clause could be more integrated; overall, it's efficient with minimal waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given complexity (2 parameters, nested object, no annotations) and an output schema exists, the description is moderately complete. It covers the purpose and return values, but lacks behavioral context and full parameter details. The output schema reduces the need to explain returns, but gaps in usage and transparency remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaning by specifying the 'name' parameter as 'library:name', clarifying its format, but doesn't explain 'inputs' (a nested object with no details). This partial compensation meets the baseline for low coverage, but leaves 'inputs' undocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the action ('Run a delegation') and resource ('by its library:name'), but it's vague about what 'run' entails—does it execute, simulate, or test? It distinguishes from siblings 'get_delegation' and 'list_delegations' by implying execution rather than retrieval, but the purpose lacks specificity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives like 'get_delegation' or 'list_delegations'. It mentions 'by its library:name', hinting at a prerequisite (a delegation must exist), but no explicit when/when-not rules or context for selection are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_delegationC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states this is a 'Get' operation, implying it's likely read-only, but doesn't confirm safety aspects like whether it requires authentication, has rate limits, or what happens on errors. For a tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's front-loaded with the core purpose and includes the key parameter detail. Every word earns its place, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no nested objects) and the presence of an output schema (which likely covers return values), the description is somewhat complete but has gaps. It lacks behavioral context due to no annotations and minimal parameter semantics. For a simple lookup tool, it's adequate but not fully helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, meaning the input schema provides no descriptions for the single parameter 'name'. The description adds some meaning by specifying it's a 'library:name' identifier, but this is minimal—it doesn't explain the format, examples, or constraints of the 'name' parameter. With low coverage, the description doesn't adequately compensate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get') and resource ('full details for a delegation'), and it specifies the lookup method ('by its library:name'). However, it doesn't explicitly distinguish this from its sibling 'list_delegations' (which likely lists multiple delegations) or 'run_delegation' (which likely executes a delegation), so it misses full sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus its siblings. It doesn't mention alternatives like 'list_delegations' for listing multiple delegations or 'run_delegation' for execution, nor does it specify prerequisites or contexts for usage. This leaves the agent without clear usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_delegationsB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses a behavioral trait: 'refreshes the registry from disk', indicating a side effect that might impact performance or data freshness. However, it lacks details on permissions, rate limits, or what 'refreshes' entails (e.g., overwrites cache).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two clear sentences. However, the second sentence about refreshing the registry could be better integrated or explained, slightly reducing efficiency. It's front-loaded with the primary purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 0 parameters, 100% schema coverage, and an output schema exists, the description is reasonably complete. It covers the main action and a side effect. However, for a tool with behavioral implications (refreshing), more context on when and why to use it would enhance completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are 0 parameters, and schema description coverage is 100%, so no parameter documentation is needed. The description doesn't add param info, but with no params, this is acceptable. Baseline is 4 as per rules for 0 parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'list' and resource 'delegations', making the purpose specific. However, it doesn't explicitly distinguish this from sibling tools like 'get_delegation' (which likely retrieves a single delegation) or 'run_delegation' (which likely executes one).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'get_delegation' or 'run_delegation'. The description mentions 'refreshes the registry from disk', which hints at a side effect but doesn't clarify if this is a primary use case or when it's appropriate compared to other listing methods.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.