Server Quality Checklist

Tool Scores

• expand_call_graphC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavior. It says 'Walk callers/callees' but does not mention output format, whether it is read-only, performance implications, or any side effects. The description is minimal and does not compensate for the lack of annotations. Also, the schema has no parameters despite mentioning 'depth', which could confuse the agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, which is concise. However, it may be over-simplified, leaving out important details. It is front-loaded with the action, so it is efficient, but it could benefit from a bit more context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has zero parameters and no output schema, so the description carries the full burden. For a tool that walks a call graph, it does not explain what the output looks like (e.g., a list of paths, a tree), how deep the walk goes by default, or any limitations. Given the complexity of a call graph expansion, the description is incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has zero parameters, but the description mentions 'up to a given depth', implying a depth parameter exists. Since schema coverage is 100% (no params), the description adds misleading information. The agent might expect a depth parameter that is not in the schema, or the tool might use a default depth. The description does not clarify how depth is specified.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool walks callers/callees from a symbol up to a given depth, which is a clear verb+resource. However, it does not differentiate from sibling tools like analyze_impact or symbol_search, which may also involve traversing relationships. The phrase 'up to a given depth' hints at a limiting parameter, but the schema has no parameters, suggesting the description may be inaccurate.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. It does not specify when a user would want to expand a call graph versus other analysis tools. Context is implied (e.g., exploring dependencies), but no explicit exclusions or alternatives are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_impactC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It implies a read-only operation by using 'list', but does not explicitly confirm side-effect freedom, performance implications, or whether any special context is required. Missing details about how the symbol is specified also obscure behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence with no redundancy or fluff. It conveys the core concept efficiently and is easily parsed.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations, no output schema, an empty input schema, and a non-trivial analysis task, this description is far from complete. It lacks information on input format, output structure, potential limitations, and usage context, making it insufficient for an agent to invoke correctly without additional external knowledge.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema is empty (0 parameters, additionalProperties true), yet the description refers to 'a given symbol' implying an input. The description fails to specify how to supply that symbol, leaving the agent without necessary invocation information. Even though 0 formal params typically get a baseline of 4, the mismatch between description and schema makes this score appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (listing symbols and files) and the resource (reverse dependencies of a given symbol). It is specific enough to distinguish from sibling analysis tools like expand_call_graph or detect_cycles, though it does not explicitly name alternatives. The phrase 'reverse dependencies' adds precision.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like symbol_search or graph_diff. The description only states what it does without indicating prerequisites, typical use cases, or situations where other tools would be more appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_patternC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the full burden of behavioral disclosure. 'Locate' suggests a read-only operation, but it does not state whether it scans the entire repository, whether it requires specific permissions, or what it returns. It does not disclose any limitations or side effects, leaving significant ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with no filler or repetition. It is front-loaded with the core action. However, its brevity comes at the cost of substance, making it terse rather than effectively concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema and no annotations, the description should clarify what 'structural code patterns' means and ideally give examples or limitations. It does not explain the tool's scope, what filters exist, or how results are presented. This is insufficient for an agent to decide whether to invoke it over sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the schema coverage is trivially 100%, so the baseline for 0 parameters is 4. The description does not need to explain parameter semantics since there are none, and it adds nothing missing in this dimension.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a verb and resource ('Locate structural code patterns') but is vague about what constitutes a 'structural code pattern.' It does not differentiate from siblings like symbol_search or get_definition, which also involve locating code elements. It is not a tautology, but it lacks the specificity needed to understand the tool's exact scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like symbol_search or analyze_impact. There are no exclusions, prerequisites, or context about scenarios where this tool is preferred. The single sentence implies a generic 'find' operation without any usage hints.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• detect_cyclesB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full responsibility for disclosing behavior. It implies a read-only operation via 'detect', but doesn't explicitly state side effects, permissions, or whether it returns a list vs. a boolean. No mention of performance, error conditions, or impact on the system. This is a significant gap for a tool with zero annotation support.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that states the core purpose without fluff. It is appropriately sized for a tool with no parameters, but it could have added one clause about output or usage without becoming verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no annotations, output schema, or parameter details, the description carries full responsibility for disclosure. It omits what the tool returns (e.g., list of cycles, graph structure), any side effects (though likely read-only, it's not stated), and performance considerations. For a simple tool this might be acceptable, but the lack of any return-type or behavioral note leaves significant gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters)Skip the schema is empty)Skip the description need not describe parameters. With 0 parameters, the baseline is 4, and the description doesn't mislead or omit anything related to parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool's function: finding dependency cycles between symbols and files. The verb 'find' is specific, and the resource 'dependency cycles between symbols and files' distinguishes it from sibling tools like analyze_impact and check_boundaries, though it doesn't explicitly name alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus siblings like analyze_impact or check_boundaries. The description doesn't mention context, prerequisites, or scenarios where this tool is preferred, leaving the agent to infer usage from the name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_boundariesB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits, but it fails to do so. It does not explain what verification entails, whether it reads or mutates state, what side effects may occur, or what the output format is. The verb 'Verify' gives a hint of read-only behavior, but that is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no filler or redundant information. It communicates the core purpose efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite the tool's apparent simplicity, the description lacks essential context: what criteria define boundary violations, how results are returned, and how this tool relates to adjacent tools. With no output schema and no annotations, an agent would not know what to expect when invoking this tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema covers all parameter information implicitly. The baseline for 0 parameters is 4, and since there is nothing to document, the description does not need to add further parameter detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Verify') and identifies a clear resource ('architectural boundaries between modules and layers'). It is distinct from sibling tools like detect_cycles or analyze_impact, though the notion of 'boundaries' could be more precise.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. It does not mention typical scenarios, prerequisites, or exclusions. The description is a bare statement of purpose with no usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_contextB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description must carry the full burden. It states that context is assembled but does not disclose whether the operation is read-only, what inputs are expected (given zero schema parameters), what the assembled context includes, or any limitations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One short sentence, front-loaded with the core action and no filler. It earns a high score for conciseness, though brevity contributes to vagueness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no annotations and no output schema, the description should explain what context is returned and how it is assembled. It only provides a high-level phrase, leaving major gaps for an apparently complex tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero declared parameters and the schema is empty, so the baseline of 4 applies. The description adds no parameter semantics, but there are no parameters to document; the phrase 'natural-language question' hints at an implicit input.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('assemble') and identifies the resource ('task-relevant code context') and trigger ('natural-language question'). It is understandable but does not explicitly distinguish itself from sibling context tools like symbol_search or summarize_file.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided about when to choose this tool over siblings; there are no exclusions or alternative references. The only usage hint is the phrase 'for a natural-language question,' which implies a broad query but does not say when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• graph_diffB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses what the diff produces but omits side effects, mutation risks, auth requirements, output format, or error handling. Minimal behavioral detail beyond the core operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One precise sentence that delivers the primary purpose without fluff. Front-loaded with the core action and outcome.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no annotations, no output schema, and an empty schema with dynamic params, the description does not compensate. It lacks return value details, parameter specification, and usage context, making it insufficient for a complex graph diffing operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has zero properties but allows additionalProperties, implying dynamic input. The description says 'between two points in time' but does not specify how to indicate those points (e.g., commit hashes, timestamps) leaving the agent without essential input guidance. Baseline 3 for 0 params is not met because the tool clearly needs arguments.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action ('Diff the code graph') and its specific outcome ('added/removed symbols and edges'), making it distinct from siblings like semantic_changelog or search_history.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus the sibling tools (e.g., semantic_changelog for human-readable changes). It does not clarify prerequisites like repository state or which time points are acceptable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• semantic_changelogB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does communicate that the output is human-readable and derived from graph changes, but it omits important behavioral context such as output format, side effects, whether a sync must have occurred first, and how this differs from graph_diff.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence with no filler. Every word contributes meaning, including 'human-readable' and 'between syncs,' which add useful context beyond the name.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool, minimalism is less risky, but the absence of annotations and an output schema means the description should explain what the returned changelog looks like and when it is available. The current text is minimally viable but leaves those details unspecified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  This tool has zero parameters, so no parameter documentation is needed. The empty schema makes coverage trivially complete, and the description introduces no conflicting parameter expectations, so the baseline score applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool produces a human-readable changelog derived from graph changes between syncs, giving both the output form and the data source. It is distinguishable from the likely raw graph_diff sibling via the 'human-readable' qualifier, though it lacks an explicit action verb like 'generate' or 'return.'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like graph_diff, search_history, or sync_repository. The phrase 'between syncs' implies a relationship to synchronization but does not explain prerequisites or exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_historyB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits. It only says 'search', which implies read-only, but does not explicitly state whether it modifies anything, what data it returns, or any limitations (e.g., date range, scope). This is insufficient for a tool with no annotation support.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with no fluff, front-loading the core action and subject. It is appropriately sized for the tool's simplicity, exhibiting no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no annotations, no output schema, and no parameters, the description carries the full burden of explaining what it does and what to expect. It only says 'past ingestion/sync history', which is vague. It lacks details about the nature of the history, the output format, any filtering capabilities, or limitations. This is inadequate even for a simple tool because the agent has no idea what 'history' entails or how to leverage the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero declared parameters, and the schema's additionalProperties set to true allows arbitrary inputs. The baseline for 0 parameters is 4, and the description does not need to elaborate on parameters since there are none. However, it does not clarify what optional arguments might be accepted, but given the schema coverage is 100% (empty properties), the description adds no additional semantic value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Search past ingestion/sync history of the repository.' It uses a specific verb ('search') and resource ('history of ingestion/sync'), and it is distinct from sibling tools like 'symbol_search' or 'get_definition' which focus on code structure. The purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives, nor does it mention exclusions or prerequisites. It merely states what it does, leaving the agent to infer usage context. There is no mention of alternatives or when not to use it, so the guidance is minimal.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• symbol_searchB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, but the description explains the hybrid semantic and keyword search nature, which is a behavioral trait. However, it does not disclose details like result ranking, limits, or side effects. Without annotations, the description carries the burden but is somewhat thin.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that effectively captures the core functionality without waste. It is front-loaded and concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool searches over all symbols, one would expect details on how to formulate queries (e.g., what parameters to provide), result format, and maybe limitations. The description and schema are minimal, and with no output schema, the agent lacks important context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no defined parameters, and description doesn't explain how to specify search queries. Since the schema is effectively empty (additionalProperties true but no defined properties), the description doesn't compensate for the lack of parameter semantics. The agent would need to infer how to pass the search term, which is a gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs a hybrid semantic + keyword search over all symbols in the repository, which is specific. It distinguishes from siblings like get_definition, get_context, find_pattern, which are more targeted, though it doesn't explicitly name alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for general symbol discovery, but does not provide explicit when-to-use vs alternatives or exclusions. Given the context of many sibling tools, more guidance would help but it's acceptable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• health_scoreA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must fully disclose behavior. It mentions the analysis factors but omits whether the operation is read-only, if it requires special permissions, or what the output format is. This leaves critical behavioral traits undisclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence that front-loads the action ('Score') and immediately specifies the resource and key metrics. Every phrase contributes meaning with zero wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema), the description covers the core function but lacks details about the return value or how the score is presented. While adequate for basic invocation, it could specify the output shape to be fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has zero parameters, so the baseline is 4. The description adds semantic value by listing the metrics evaluated, clarifying what the tool computes even though no parameters are needed. It does not introduce parameter-specific details because none exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: to score repository health based on four specific metrics (dead code, cycles, coupling, coverage). The verb 'Score' and resource 'repository health' are specific, and the metrics differentiate it from sibling tools like detect_cycles or check_boundaries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus alternatives. Sibling tools suggest specific analyses (e.g., detect_cycles, check_boundaries), but this description does not explicitly contrast or recommend usage scenarios, leaving the agent to infer.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sync_repositoryA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only says 'Trigger re-indexing', which implies a side effect but does not explain consequences (e.g., changes to the index, potential downtime, whether it is asynchronous, permission requirements, or whether it is safe to call repeatedly). This is a significant gap for a mutation-like operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no unnecessary words. It is front-loaded and directly states the action. There is no fluff, making it highly concise and well structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool (no parameters, no output schema) and the absence of annotations, the description provides a basic understanding but lacks contextual depth. It does not explain what happens after the trigger, whether the operation is immediate or queued, or any side effects. For a tool that performs a significant action like re-indexing, more context would aid the agent in deciding when to invoke it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema defines zero parameters, and the schema coverage is 100% (vacuously). The description does not need to explain parameters since there are none. According to the rubric, a zero-parameter tool gets a baseline of 4. The description adds nothing about parameters, but nothing is missing either. The additionalProperties: true allows arbitrary extra arguments, but the description does not mention this, which might be a minor gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Trigger re-indexing') and the resource ('the connected repository'). It is specific in that it denotes a refresh operation, which is distinct from the sibling tools that focus on searching and analysis. The verb+resource pattern provides unambiguous intent.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention prerequisites (e.g., repository must be connected) or situations where re-indexing would be appropriate. The only implicit hint is 'connected repository', but no explicit exclusions or alternative tool recommendations are given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_definitionA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full responsibility. It only states 'Fetch' which implies a read operation, but it does not explicitly disclose side effects, access requirements, or performance implications. This leaves significant behavioral ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence with no filler. Every word contributes meaning, and it is front-loaded with the primary action and resource.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, no annotations, and an empty parameter schema, the description is the sole source of information. It explains the purpose but leaves critical gaps: how to specify 'name' or 'id' (parameter names), what exactly is returned, and any limitations or edge cases. This is insufficient for confident invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has zero parameters, so the baseline is 4 per rubric. The description adds meaning by indicating that the tool accepts either a 'name' or 'id', which is more than the empty schema provides. However, it does not specify the exact parameter keys or whether both are required, though the baseline is already at 4 and the description adds value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Fetch') and the resource ('full source and metadata of a symbol'), and distinguishes it from siblings like 'symbol_search' (search vs. retrieval) and 'get_context' (context vs. full source/metadata). The phrase 'by name or id' further specifies the identification method.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used when a symbol's source or metadata is needed, but it does not provide explicit guidance on when to prefer this over alternatives like 'symbol_search' or 'get_context'. There is no mention of preconditions or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• rate_contextA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are available, so the description needs to disclose side effects and behavioral requirements. It only says 'rate' without saying whether feedback is stored, whether it requires authentication, whether it is one-way, or whether it returns any acknowledgement. This leaves material ambiguity about the tool's effect on the environment.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one short, front-loaded sentence that is easy to skim. The phrase 'Feedback loop:' is slightly redundant because 'rate the usefulness' already conveys the loop idea. It could be tightened, but every sentence earns its place and there is no major bloat.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the empty schema, no output schema, and no annotations, a very short description can be minimally viable. However, this one does not explain how to invoke feedback, whether a context identifier is needed, or what a successful rating entails. That leaves enough ambiguity that the agent may send wrong or incomplete payloads.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema declares zero named parameters, and the 0-param baseline is 4. The description contributes by identifying 'usefulness' as the core semantic input. It could still add more detail about what free-form properties are accepted, since additionalProperties is true, but it provides adequate baseline help.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('rate'), the object ('context'), and the specific dimension being assessed ('usefulness'). It scope to 'previously returned context', which separates it from sibling tools that analyze or search or transform code. The core sentence is not a tautology and conveys the tool's unique purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no explicit when/when-not guidance or named alternatives. However, the phrase 'previously returned context' and 'Feedback loop' imply the tool should be used after context has been fetched and the agent wants to score its quality. The guideline is implied rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• summarize_fileA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses more than the tool name by mentioning symbols, relations, and codebase role, giving the agent a sense of what the summary will contain. However, there are no annotations, so the description carries the full behavioral burden; it omits whether the operation is read-only, how the target file is selected, and what output shape the agent should expect.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one compact sentence that front-loads the action and names the semantic content. There is no filler, boilerplate, or repetition of already-visible schema information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of annotations and output schema, the description is a workable minimal explanation but leaves key contextual details out: return format, how the file is chosen, and what exactly 'role in the codebase' covers. It is sufficiently clear for the obvious use case but not complete enough for edge cases or alternatives.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With zero declared parameters and a schema that only allows additional properties, there is no parameter meaning for the description to expand on. The baseline for zero parameters is 4, and the description provides enough conceptual context. It could mention how to specify the file, but that is a context gap rather than a parameter-semantic one.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Summarize' and names the resource clearly: 'a file's symbols, relations and role in the codebase.' This differentiates it from sibling tools like symbol_search, get_context, and analyze_impact, which focus on different retrieval or analysis tasks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no explicit when-to-use guidance or exclusions. It does not contrast summarize_file with sibling tools such as get_context or analyze_impact, so the agent gets no help deciding when this tool is preferable over those alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.