Server Quality Checklist

Tool Scores

• diffA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly, idempotent, and non-destructive behavior. The description adds valuable behavioral context: newest-first ordering, dotted-prefix matching scope, and the derived `superseded_by` id with empty-string semantics for the latest event. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded: the first sentence gives the core purpose, and the second provides high-value examples of prefix matching and derived data. Every sentence earns its place with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema and safety annotations, the description sufficiently covers the essential behavior: ordering, prefix semantics, and the derived superseded_by trail. It does not discuss sibling-tool selection, but the core functionality is thoroughly described.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers both parameters fully (100% coverage), so the baseline is 3. The description restates prefix matching with an example but does not add new parameter-level semantics beyond what the schema already documents.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns change history for a codebase entity, newest first, and highlights unique behaviors like prefix matching and the derived `superseded_by` field. However, it does not explicitly differentiate from the similarly-named sibling tool `history`, so it stops short of full sibling distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The intended use is implied: call this when you need a chronological change history for an entity, especially with prefix matching. But the description does not compare this tool to alternatives like `history` or `blame`, nor does it mention exclusions or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• searchA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly, idempotent, and non-destructive behavior. The description adds search semantics: full-text across four fields and substring matching with escaped wildcards (from schema). This contextualizes behavior beyond annotations, though pagination/ordering are not mentioned (but output schema covers returns).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: one declarative purpose statement and one illustrative set of examples. No redundant content and the main intent is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with two parameters, full schema coverage, output schema, and clear annotations, the description adequately conveys what it searches. The example questions help the agent map natural language to tool invocation; however, it does not mention result ordering or the limit parameter behavior (though schema covers limit).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, including the case-insensitive substring behavior and wildcard escaping. The tool description adds only usage examples, not new parameter semantics, so it meets baseline but does not exceed schema detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'search' and names the exact resources (entity paths, diffs, reasoning, agents). Example queries like 'what changes were made for the billing feature?' clarify the scope and distinguish it from sibling tools like diff or blame.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit example questions that signal appropriate use cases, such as cross-cutting search across multiple entities. It does not directly name alternatives or state when not to use this tool, but the examples imply broad search rather than targeted diffs or history queries.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• changesetA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly, idempotent, non-destructive. Description adds ordering (oldest first) and specific error format, going beyond annotations without contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, each with clear purpose. No wasted words. First sentence states what the tool does, second gives usage context and error handling.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple single-parameter tool with full schema coverage and an output schema, the description sufficiently covers ordering, error condition, and intended use. No gaps identified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% and fully describes the changeset_id parameter. Description adds no new parameter semantics beyond what the schema provides, so baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states 'All events that share a changeset_id, oldest first.' It specifies the resource (events) and ordering, distinguishing it from siblings like 'history' (likely broader) and 'search' (different target).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use to reconstruct the full scope of a feature or task across multiple entities,' providing clear context. Also describes error behavior for empty changesets. Lacks explicit when-not or alternative comparisons.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• stale_decisionsA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare read-only, idempotent, and non-destructive behavior. The description adds significant transparency: it details the two deterministic rules, explains that pure age alone never surfaces, specifies the live-entity evidence checks, and states that the tool is templated/deterministic with no LLM call.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is moderately long but well-structured, with the core purpose front-loaded. It uses three paragraphs for rules, output, and behavior; every sentence contributes information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers purpose, rules, output fields, ordering, filters, determinism, and next steps. Combined with the existing output schema and annotations, the agent has everything it needs to invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already describes all four parameters at 100% coverage, including entity_path's prefix semantics. The description only restates that filtering by entity_path, project, or agent is possible, adding no new parameter-level information; the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens by defining the tool as returning decisions due for revisit, either past their date or with a triggered stale condition. It then details two deterministic rules, clearly distinguishing this from history/search/log_change.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool (decisions due for revisit) and states 'Surfacing only' with a follow-up recommendation to supersede if triggered. It does not explicitly name alternative tools or exclusion rules, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• blameA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate safe read-only idempotent operation. The description adds value by detailing return fields (status, superseded_by) and error handling behavior (returns error object with isError: false). No contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short paragraphs, no fluff. The first sentence immediately states the core purpose. Every sentence adds necessary context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given a single parameter, existing output schema, and comprehensive annotations, the description covers the tool's functionality, return data, and error case fully and clearly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Only one parameter with 100% schema coverage. The description adds the constraint 'exact entity path (no prefix matching)' and provides examples, enhancing the schema's description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves the most recent change to an entity, likening it to git blame for semantic entities. It distinguishes from siblings like history or diff by focusing on the latest change and including decision state.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains what the tool does and notes error behavior when no history exists. It lacks explicit guidance on when not to use or alternatives, but the purpose is clear enough for correct selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• historyA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnly=true, idempotent=true, and destructive=false, so safety is covered. The description goes beyond by disclosing the error behavior for unparseable 'since' input, returning a JSON error array instead of silently returning empty results. This is valuable behavioral context not in the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences: the first states purpose and ordering, the second gives usage guidance and error handling. It is front-loaded, with no wasted words, and every sentence contributes meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists and the description covers purpose, filtering, ordering, and error behavior, the tool is fully specified for an agent. The description is complete for this 5-parameter optional-input tool without needing to explain return values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with detailed descriptions for each parameter, so the baseline is 3. The description adds minor value by explicitly stating these parameters can be combined, but it does not explain syntax or semantics beyond what the schema already provides. No compensation needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Filtered change history across all entities, newest first.' It uses a specific verb ('browse' implicitly via 'history') and resource ('all entities'), and the 'newest first' ordering adds precision. This distinguishes it from siblings like log_change, diff, and search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit guidance on how to combine filter parameters ('since', 'entity_path', 'project', 'changeset_id') to scope results. It does not explicitly mention when not to use this tool or name alternatives, but the usage context is clear enough for an agent to know when to invoke it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• log_changeA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations (readOnlyHint=false, destructiveHint=false) are present, but the description adds extensive behavioral context: how events are stored, validation failure behavior (error vs warnings), the dual-event rename pattern, supersede linking, and quality validation. It significantly exceeds the annotation information.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: summary, timing, special cases (rename, supersede), validation failures. It is comprehensive but slightly lengthy; however, every section adds necessary information. Could be tightened slightly, but overall effective.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (15 params, 2 required, output schema), the description covers all important aspects: what the tool does, when to use, edge cases, validation rules, and output format hints (id, timestamp, warnings, error). It complements sibling query tools well and leaves no major gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All 15 parameters have schema descriptions (100% coverage), and the description adds substantial extra meaning beyond them. For example, it explains the rename dual-event pattern, provides reasoning examples and quality validation warnings, lists all change_type values in prose, and gives entity path examples. Interdependencies between parameters (e.g., rename_from + change_type) are clearly explained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool records a change to a codebase entity, specifies to call it immediately after making a meaningful change, and explains the output (id and timestamp). It distinguishes itself from sibling tools (query tools) by focusing on logging.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to call ('immediately after any meaningful change'), provides specific use cases (renames, supersedes), and explains what not to do (do not delete/edit history). However, it does not explicitly compare to sibling tools, though it's clear this is the logging tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• prior_attemptsA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark the tool as read-only and idempotent; the description adds that it is templated, deterministic, pull-only, and conservative by design, warning that an empty list is the preferred answer over speculative false positives. It also discloses fallback substring matching behavior and that a leading note row indicates this fallback.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but well-structured: summary, usage context, output field interpretation, and tuning parameters. The output field explanation is dense but earns its place by explaining the meaning of 'reverted' and 'reopened'. It is slightly verbose but appropriate for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With an output schema present, the description need not restate return values, but it explains the conceptual model (outcome trail, confidence, supersede, current_status) essential for correct interpretation. It gives the agent complete context for when and how to use the tool, including edge cases like empty results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Even with 100% schema coverage, the description adds crucial semantics: the precedence rule between entity_path and description, the fuzzy matching fallback when the semantic extra is unavailable, and the interpretation of min_confidence values and window_minutes. This goes well beyond the schema's field descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns prior change attempts with inferred outcomes, and the imperative 'Call this BEFORE editing an entity' makes the action and resource explicit. It is distinct from sibling tools like history or diff by focusing on rejected/reopened attempts with outcome reasoning.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a strong directive to call before editing and explains how the outcome field can prevent repeating a rejected approach. It also provides explicit tuning guidance via min_confidence and explains when to prefer the default 'proximity_high' versus widening recall with 'proximity_low'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.