Server Quality Checklist

Tool Scores

• keyring_checkout_historyB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must carry the full behavioral disclosure burden. It states it is a 'query' but does not explicitly declare read-only behavior, nor does it describe return format, pagination, rate limits, or any side effects. The phrasing 'who accessed what, when, and why' suggests it returns log entries, but the agent is left without concrete behavioral expectations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured. The opening line conveys the core purpose immediately, followed by a compact, scannable parameter list. No redundant filler exists; each sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While the parameter descriptions are helpful, the tool lacks an output schema and the description does not explain the return type or structure. For a query tool, an agent would benefit from knowing whether it returns a list, a count, or a summary, and whether results are paginated. This missing context leaves the definition partially incomplete for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does so thoroughly: each parameter (agent_id, service, secret_name, days) is given a clear meaning and default behavior. This adds substantial value beyond the bare schema types and defaults, enabling correct filter usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action and resource: 'Query the checkout ledger — who accessed what, when, and why.' This distinguishes it from the sibling tools listed (which suggest listing, requests, and affinity), though it does not explicitly contrast with them. The purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus the siblings. The description does not mention contexts where this should be preferred, nor does it list any exclusions or alternatives. An agent would have to infer usage from the tool's name and purpose alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• keyring_agent_affinityA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full behavioral burden. It indicates a read-only analysis operation through 'Returns,' and states the data source, but it does not explicitly confirm lack of side effects, permissions, or any output limits/pagination behavior. This is acceptable but not thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: two purposeful sentences followed by an args line. The core purpose is front-loaded, with no filler or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter analysis tool with no output schema and no annotations, the description covers the input, source, and high-level output. Minor details like exact output shape or aggregation semantics are missing, but these are not essential for a correct call.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter 'days' is clearly explained as 'How far back to analyze (default 30)' despite 0% schema description coverage. This fully compensates for the schema gap by giving the parameter meaning and default context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Analyze') and a clear resource ('agent-service affinity patterns from the checkout ledger'), and explains the output: 'which agents use which services most.' It implicitly distinguishes from siblings like list_available and checkout_history by focusing on affinity analysis, but it does not explicitly name alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states intended use cases: 'workstation specialization analysis and context overload detection.' However, it does not explicitly say when to prefer this tool over its siblings or provide exclusion conditions, leaving usage guidance partly implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• keyring_list_availableA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and adds important behavioral context: it is a listing operation, and it explicitly guarantees 'never values,' which is critical for security-sensitive tool use. It also notes that the parameters are used for identity verification, clarifying why an agent would provide them. It does not describe response format or error behavior, but the main behavioral guarantee is clear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The main behavior is front-loaded in the first sentence, followed by a compact two-parameter list with examples. There is no filler; every sentence contributes either to purpose, safety, or parameter semantics.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list operation with two optional parameters and no output schema or annotations, the description covers the key points: what it lists, what it returns, and the identity parameters. It is missing explicit usage routing relative to siblings, error conditions, and a clear statement of whether at least one identity argument is expected, leaving some gaps in an otherwise adequate definition.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Scheema description coverage is 0%, so the description must add paramter meaning, and it does. did is explained as 'Agent's DID (did:key:...) for identity verification,' and agentmail_inbox as 'Agent's AgentMail address as fallback identity,' giving the agent a basis for choosing between them. It stops short of specifying whether one is required or how precedence works.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action—list secrets the calling agent can access—and specifies the resource and a key distinction: it returns names and descriptions only, never values. This naturally separates it from sibling tools like keyring_authenticated_request, which imply performing some other keyring operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool: when an agent needs to discover which secrets it can access. However, it does not explicitly name alternatives or state when not to use it, leaving the routing decision to the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• keyring_authenticated_requestA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the full burden and meets it: it discloses that the key never appears in the response, that only response data is returned, that every call is logged, and that scope/identity fallback behaviors exist. This is far beyond a generic 'makes a request' statement.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: a concise summary, security/logging notes, then a complete Args list. It is long because it must document 11 parameters without schema help, but a few sentences ('The keyring retrieves the secret...') restate the opening mechanic.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 11 parameters, no annotations, and no output schema, the description covers invocation and core behavior thoroughly. It does not specify response/error shape beyond 'response data,' but that is largely service-dependent and does not prevent correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description compensates fully by documenting all 11 parameters with examples, defaults, required conditions (base_url for unregistered services), auth_type options, and fallback semantics. An agent can correctly populate every parameter from this text alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description opens with a specific verb+resource: 'Make an authenticated API call using a managed secret.' It clearly describes the operation and references sibling-adjacent concepts (keyring_list_available, checkout ledger), making it easy for an agent to distinguish this request tool from listing/history/affinity siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context: the agent is told to obtain secret_name from keyring_list_available, that calls are logged to the checkout ledger, and that unregistered services need base_url in params. It lacks explicit 'use X instead' exclusions, but the intended selection is unmistakable given tool names and content.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.