Server Quality Checklist

Tool Scores

• check_budgetA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, description carries full behavioral disclosure burden. Excellent coverage: specifies reset schedule ('resets on schedule'), calculation logic ('clamped to >= 0'), rate limiting enforcement status, and precise return structure including ISO 8601 timestamp format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with clear visual separation between purpose statement and return value specification. Front-loaded with the core action. Detailed return dictionary is verbose but justified given budget monitoring requires precise field definitions.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Comprehensive coverage for a zero-parameter read operation. Despite existence of output schema (per context signals), description ensures completeness by documenting all return fields, calculations, and temporal behaviors (reset schedules) that might not be visible in schema alone.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Zero parameters present; baseline score of 4 applies per rubric. Description appropriately does not fabricate parameter semantics where none exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb ('Check') and resource ('safety budget utilization'). Implies read-only status check distinct from sibling 'certify_action' (which performs mutations) by framing itself as state retrieval and referencing certifications as metadata rather than actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use guidance or comparison to siblings. Does not advise calling this before 'certify_action' to validate budget availability, though the return structure implies this use case.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• certify_actionA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure, successfully detailing the decision categories (Certified, CertifiedWithWarning, EscalateToHuman, Blocked), zone classifications, and constraint margin logic. It explains the 'drift_budget' return value implying budget consumption, though it could more explicitly state that this operation consumes budget or creates a persistent certificate record.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The docstring structure is logical and front-loaded with purpose, though the Returns section lists 9 output fields that likely duplicate the existing output schema (context signals indicate has_output_schema=true), creating minor redundancy. The example usage is valuable and appropriately placed at the end.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a high-complexity safety tool, the description provides robust coverage including parameter semantics, return structure, decision logic, and executable examples. Minor gaps remain regarding inter-tool workflow (e.g., whether to call `check_budget` first) and explicit mutation/side-effect statements.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Given the schema has 0% description coverage, the Args section provides comprehensive compensation by documenting all 6 parameters with semantic ranges (e.g., scope '[0, 1], where 0=narrow, 1=global') and illustrative examples (e.g., 'act_123', 'claude_v3'), fully clarifying the dimension scales and identifiers.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with the specific verb 'Evaluate' and clear resource 'safety of an action', elaborating that it uses the 'QAE safety kernel' to assess proposed actions. It effectively distinguishes from siblings: unlike `check_budget` (resource checking) or `get_certification_history` (retrieval), this performs active multi-dimensional safety evaluation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  While the phrase 'assess whether a proposed action is safe to execute' implies usage prior to action execution, there is no explicit guidance on when to use this versus `check_budget` (e.g., whether to check budget first) or how it relates to `get_certification_history`. The description lacks explicit 'when-not-to-use' or prerequisite guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_certification_historyA
  4.2/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full behavioral burden excellently. It discloses critical data lifecycle constraints ('Up to 50 certifications are stored; older ones are discarded'), ordering guarantees ('most recent first'), and rich return structure details that annotations would typically cover.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  While well-structured with Args/Returns sections, the description is verbose; the extensive Returns field enumeration duplicates information that should ideally live in the output schema (which exists per context signals). The retention limit sentence earns its place, but the field-by-field return documentation could be trimmed.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero annotations and minimal schema coverage, the description provides comprehensive context including data retention policies, return format, and parameter constraints. The output schema exists, so the detailed return documentation in the description is somewhat redundant, though the behavioral metadata (50 item limit, discarding) is essential.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the Args section compensates effectively by documenting the 'limit' parameter's purpose, default value (10), and maximum constraint (50) - crucial information not present in the bare JSON schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verb 'Retrieve' and resource 'certification history', clearly distinguishing it from sibling 'certify_action' (which implies writing/creating) and 'check_budget' (unrelated domain). It establishes the scope as 'recent' history with explicit ordering ('most recent first').

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by detailing what the tool returns (history vs new certifications), but lacks explicit selection guidance like 'Use this to audit past decisions; to certify new actions use certify_action instead.' No when-not-to-use or prerequisite guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.