Server Quality Checklist

Tool Scores

• verify_agent_scopeA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses key checks performed but omits return format or behavior (e.g., error handling, whether it blocks or just returns a status). Without output schema, agent lacks full behavioral insights.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two efficient sentences: first states core purpose, second enumerates checks and gives a usage directive. No redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is reasonably complete for an authorization check but lacks return value specification. Agent must infer whether response is boolean or detailed object. Missing error scenarios.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers all 3 parameters with descriptions, so baseline is 3. The description adds minimal extra meaning beyond schema, e.g., clarifying the 'action' parameter's role. No significant enhancement.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool checks authorization for a specific action or endpoint, and lists specific verifications (X.509 certificate, scope, financial limits, revocation). It is distinct from sibling tools (audit_log_append for logging, check_risk_score for risk scoring).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs 'Call this BEFORE executing any action that might exceed scope — not after.', providing strong usage guidance. However, it does not explicitly mention when not to use or compare to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• audit_log_appendA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description must disclose behavior. It reveals immutability of the log, return values (risk score and transaction ID), and automatic certificate revocation for high-risk events (score >= 0.85). This is good context beyond basic logging.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with the primary action, no unnecessary words. Every sentence adds value: purpose, use cases, return values, and an important behavioral note.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema, but the description correctly states return values (risk score and transaction ID) and a key side effect (certificate revocation). With 3 parameters (including a nested object) and good schema coverage, the description is sufficient for an agent to invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds concrete examples for metadata (amount, currency, venue for transactions; resource_type, resource_id for data ops) and elaborates on event_type usage (e.g., 'transaction_initiated' for financial ops). This provides practical guidance beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'append' and the resource 'agent's immutable audit log'. It distinguishes the tool from siblings (check_risk_score, verify_agent_scope) by focusing on recording events for traceability.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says when to use (voluntarily record actions like transactions, data ops, API calls) and provides specific event_type guidance. However, it does not mention when not to use or alternatives, though siblings are distinct.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_risk_scoreA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. Describes return values (score, band, trend, guidance) and implies read-only operation. Could mention lack of side effects, but sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: first describes what is retrieved, second gives usage guidance. No wasted words, front-loaded with key information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Returns risk score, band, trend, and guidance. No output schema, so description covers return values adequately. Could specify response format or prerequisites, but sufficient for a simple read tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has zero parameters (100% coverage). With 0 params, baseline score is 4 as per guidelines. No additional parameter info needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description states specific verb 'Retrieve' and resource 'rolling 30-day risk score, risk band, drift trend'. Clearly distinguishes from sibling tools (audit_log_append, verify_agent_scope) which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states use case: 'decide whether to self-throttle, escalate to a human, or proceed normally'. Does not explicitly exclude alternatives, but context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.