Server Quality Checklist

Tool Scores

• get_account_summaryB
  3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the full burden. It discloses a key behavioral trait: 'this tool never exposes direct PII', which is valuable privacy context. However, it does not mention other behaviors such as whether it requires specific permissions, handles missing accounts, or returns errors. The single disclosed trait is important but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that front-loads the primary action and object, then adds a meaningful privacy qualifier. It is appropriately short and contains no filler, though it could include a bit more context without becoming verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with one parameter and no output schema, so the description need not be extensive. Still, it leaves gaps: no mention of what the summary contains, how to interpret the response, or typical usage scenarios. While not critical for a trivial read operation, the lack of any usage guidance and return format means an agent may not fully understand the tool's output.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameter descriptions (coverage 0%), and the description does not mention the 'account_id' parameter at all. The parameter name is self-explanatory, which partially compensates, but the description adds no semantic detail about its format, scope, or expected values. Since schema coverage is low, the description should have compensated but did not.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Return') and the resource ('account summary'), and adds the qualifier 'minimized' to distinguish it from a full account detail. While it does not explicitly name sibling tools, the siblings (get_case, request_account_freeze) are sufficiently different in name and purpose that an agent can distinguish without explicit differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool vs alternatives, nor any exclusions or prerequisites. It only says it returns a summary, implying it is for summary needs, but does not mention when to prefer it over get_case or request_account_freeze, or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• request_account_freezeB
  3.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It does reveal a key trait: human approval is always required, implying the request is not immediately executed. However, it does not disclose what happens after approval, whether it is asynchronous, what the response will be, or any side effects. Partial disclosure is better than nothing, but gaps remain.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences with no filler. The first sentence states the action and impact; the second adds the critical policy constraint. Every word earns its place and the structure is front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given three required parameters, no annotations, no output schema, and a high-impact action, the description is insufficiently complete. It does not explain parameter semantics, expected outcomes, or how to handle approval delays. An agent would need additional documentation or inference to use this tool correctly in a real flow.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage and the description provides no parameter explanations. The parameter names (account_id, reason, idempotency_key) are somewhat self-explanatory, but the description adds zero meaning beyond the raw names. For a tool with three required parameters, the description should at least hint at their purpose, especially the idempotency key's role in preventing duplicate requests.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb ('request') and resource ('account freeze') and adds 'high-impact' to convey significance. It inherently distinguishes itself from the read-only sibling tools (get_case, get_account_summary) by being an action that modifies state.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives, nor any exclusions or prerequisites. It mentions human approval is required, but that is a policy constraint, not a usage guide. An agent must infer that this tool is for freezing accounts, but there is no explicit comparison to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_caseA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the full behavioral burden. It does disclose a meaningful constraint—the tenant-boundary scoping and read-only retrieval semantics of 'Retrieve.' However, it does not disclose error behavior for missing/non-accessible cases, auth requirements, or return expectations, which are meaningful gaps for an unannotated tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence with the verb and resource front-loaded and zero filler. The tenant-boundary qualifier earns its place as a scope constraint.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple retrieve-by-ID tool with a single parameter and no output schema, the description covers the essentials: what it retrieves and under what scope boundary. Missing output-format and error details are minor for this complexity tier.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description should compensate. It connects case_id to 'support case' and the tenant-boundary constraint, adding some meaning beyond the bare name. Still, it provides no format, example, or uniqueness detail for case_id—though the single self-explanatory parameter name reduces the need for heavy elaboration.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  'Retrieve a support case' states a specific verb and resource, and the tenant-boundary clause adds a precise scope constraint. The resource (case) is clearly distinct from the sibling tools (account summary, account freeze), though the description stops short of explicitly naming those alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied through the verb and resource—call this when you have a case_id and need case details. The tenant-boundary scoping hints at context, but there is no explicit when-not-to-use guidance, no mention of alternatives, and no condition that routes the agent to get_account_summary or request_account_freeze.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.