Server Quality Checklist

Tool Scores

• run_actionB
  3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds the 'tier-gated' concept and the modes (DRY_RUN, approve, execute), which go beyond the annotations. However, it does not explain the gating mechanism, what each mode entails, or the nature of destructiveness beyond what annotations already indicate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence (17 words) that front-loads the verb and key concept. While very brief, it avoids fluff. However, it could include structured information without losing conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (9 parameters, destructive, no output schema, tier-gating), the description is woefully incomplete. It lacks details about how gating works, when to set dry_run, what approval_token is, and the specific adapters. It needs much more context to be usable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage and 9 complex parameters, the description fails to add any meaning to the parameters. It only mentions 'adapter' indirectly and does not explain host, host_type, action, approval_token, etc. This is a critical gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with a clear verb 'Run' and specifies the resource 'tier-gated actuation via an adapter'. The parenthetical '(DRY_RUN, approve, execute)' further clarifies the modes. This clearly distinguishes it from sibling tools which are about scanning and fact queries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide any guidance on when to use this tool vs alternatives. It does not mention prerequisites, conditions, or warning about destructive impacts, which is needed given the destructiveHint annotation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• drift_scanB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false. The description adds minimal behavioral context beyond the concept of drift. It does not disclose how the baseline is determined, performance implications, or any side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence of 10 words, front-loading the key action ('Compute drift findings'). No wasted words; every part earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description should explain what a drift finding consists of or how results are presented. It does not. The simple structure (1 optional param, no nested objects) partially mitigates the need, but the lack of output context and no comparison to siblings leaves the agent underinformed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for the single 'subject' parameter, which is already described as 'Restrict the scan to one subject.' The tool description does not add any additional meaning or clarification beyond the schema, so baseline score of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'compute', the resource 'drift findings', and explains what drift findings are ('observed facts versus the known-good baseline'). It distinguishes from sibling tools like query_facts and fact_history by focusing on drift analysis.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives (e.g., query_facts, fact_history). It does not mention prerequisites, exclusions, or context for usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ingest_observationA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate the tool is not read-only (readOnlyHint=false) and not destructive (destructiveHint=false), so the description adds the context that it parses telemetry into facts. However, it does not disclose whether previous facts are overwritten, any authentication requirements, or error handling. The added value is moderate given the annotation baseline.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that starts with the action verb. It contains no filler or redundancy, making it efficient for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 4 parameters, 3 required, and no output schema, the description is too minimal. It does not explain what 'observed facts' are, the format of the raw input, the effect of multiple ingests, or error conditions. This leaves significant gaps for an agent to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 25% (only the 'raw' parameter has a brief description). The tool description provides no parameter-specific information, leaving 3 of 4 parameters (collector, subject, predicate) without semantic guidance. This is insufficient, especially for a tool with required parameters and an enum.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Parse') and resource ('captured host telemetry') and clearly states the output ('observed facts'). It distinguishes from sibling tools like query_facts (query existing facts) and run_action (perform actions), making the ingestion purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for ingesting telemetry but does not explicitly state when to use this tool versus alternatives (e.g., when to use drift_scan or query_facts). No exclusions or prerequisites are given, so the agent must infer usage context from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_factsA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and destructiveHint. Description adds only 'active' context, which is minimal beyond annotations. No mention of pagination, limits, or other behaviors.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, front-loaded with action and resource, no wasted words. Perfectly concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Simple tool with no output schema; description covers purpose and filter options adequately. Could be improved by explicitly distinguishing from 'fact_history' or noting only active facts.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so description adds little beyond schema descriptions. It reinforces the two filter dimensions but no extra semantic depth.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states action ('list') and resource ('active fleet-state facts') with filtering options. While it implies a distinction from 'fact_history' via 'active', it does not explicitly differentiate from siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description implies use for listing facts with optional filters, but provides no explicit guidance on when to use this tool versus alternatives like 'fact_history' or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fact_historyA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read-only and non-destructive behavior. The description adds the 'bitemporal' aspect, clarifying it returns full history, which is useful context beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, 10 words, front-loaded with key information. No redundant or unnecessary text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Adequate for a simple retrieval tool with annotations covering safety, but lacks detail on return format, behavior for missing data, and how the predicate parameter affects results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description only explains the 'subject' parameter implicitly ('for a subject') but does not explain the optional 'predicate' parameter, leaving its filtering role unclear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Return', the resource 'full recorded (bitemporal) history of facts', and specifies the subject. It distinguishes from sibling tools like query_facts (current facts) and ingest_observation (adding data).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives such as query_facts for current data. The description implies use for historical data but does not state exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.