Server Quality Checklist

Tool Scores

• get_latest_build_sessionB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and idempotentHint, so the description's safety transparency is adequate. However, the claim 'Get all ... log lines' contradicts the tail_lines parameter which only retrieves the last N lines. The description does not disclose this limitation, misleading the agent about the scope of data retrieved.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is reasonably concise at five sentences, covering purpose, output format, and content. It front-loads the primary action and follows with return structure. Some redundancy (e.g., 'raw VerseBuild output lines' twice) could be trimmed, but overall efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read tool with one optional parameter and strong annotations, the description adequately conveys return fields and general use. However, it omits the tail_lines parameter's role, the fact that 'all' is misleading, and does not mention when the log might not exist (though the output provides log_exists). Given the negative impact of the missing parameter info, completeness is adequate but not thorough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% (context signal), meaning the description must document the parameter. The parameter 'tail_lines' is not mentioned anywhere in the description; no explanation of its function, defaults, or constraints is provided. The agent gains no behavioral understanding beyond the schema's own minimal label.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves 'all VerseBuild log lines from the most recent build session' with a specific verb ('Get') and resource (latest build session logs). It implicitly distinguishes from siblings by focusing on build session content, not freshness or diagnostics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description indicates the tool is 'useful for understanding the full build context', implying when to use it, but does not explicitly contrast with siblings like 'check_verse_log_freshness' or provide when-not-to-use guidance. Usage is clear but not exclusive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_verse_diagnosticsA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate readOnlyHint, idempotentHint, and non-destructive. The description adds significant behavioral detail: parsing method, deduplication, return format (structured JSON with fields), and source log path. This goes well beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-organized with a summary line, then details, then return structure. It is mostly concise, though the return value list is slightly lengthy. Front-loaded with the purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description fully explains the tool's operation, output format, and parameter effects. Combined with schema descriptions, the agent has all necessary information to use the tool correctly. No gaps identified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides descriptions for all parameters (tail_lines, file_filter, max_results, severity_filter), so the description adds no parameter-specific insight. Baseline 3 is appropriate given schema coverage is high despite context signaling 0% (likely a discrepancy).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it retrieves Verse compile diagnostics from logs. It uses a specific verb ('Get') and resource ('verse diagnostics'), distinguishing it from sibling tools like check_verse_log_freshness (log recency) and get_latest_build_session (session info).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for diagnostics but does not explicitly state when to use this tool versus alternatives. No exclusions or when-not-to-use guidance are provided, leaving the agent to infer context from the sibling names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_verse_log_freshnessA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint, idempotentHint, destructiveHint. The description adds the return structure (log_path, exists, modified_at, age_seconds) beyond the annotations. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences: first states the action, second explains usefulness, third lists return fields. No wasted words, front-loaded with key information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, safe operation) and that annotations cover safety, the description fully explains the tool's purpose and output. Output schema exists but description lists return fields explicitly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has no actual parameters (input schema is an empty object). Since there are zero parameters, baseline is 4. Description does not need to add parameter information and correctly focuses on behavior.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it checks when the UEFN log was last modified, and explicitly connects it to determining build completion. It is distinct from sibling tools get_latest_build_session and get_verse_diagnostics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear use case ('useful for determining if a build has completed'), but does not explicitly state when not to use or mention alternatives. Context from siblings implies usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.