Server Quality Checklist

Tool Scores

• get_stack_healthC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears full responsibility for behavioral transparency. It does not disclose any side effects, access requirements, performance implications, or what 'health diagnostics' entails (e.g., whether it mutates state or is read-only).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that lists the types of results. It is front-loaded and efficient, though it could be improved by adding brief usage context without sacrificing brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema and the complexity of a diagnostic tool, the description lacks sufficient context. It does not explain the tool's role relative to siblings, what 'stack health' includes, or how to interpret the results beyond what the output schema provides.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain the purpose or default behavior of the 'repo_path' parameter. The schema shows it is optional with a null default, but without context, the agent cannot infer what happens when it is omitted or provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states what the tool does (returns diagnostics, statistics, metrics), but it is vague about the specific resource or domain. 'Stack health' is not clearly defined, and the description does not distinguish this tool from siblings like 'trace_fullstack_path' or 'verify_breakage'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus its siblings. For example, it does not clarify whether this is a general health check or a debugging step, and no exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_schema_changeC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of disclosing behavioral traits. The description does not mention what happens upon failure (e.g., error messages, warnings), whether the tool modifies any state, or if it requires network access or specific permissions. The behavior is presented too abstractly for safe agent invocation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence of 10 words, which is concise. It front-loads the key verbs (runs compiler and schema verification). However, it omits essential details, crossing the line from concise to underspecified. Still, brevity is maintained.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero annotations, no output schema explanation (but an output schema exists), and 3 parameters with no description, the tool description fails to provide enough context. The agent needs to know the output format (what success/failure looks like), the expected data format for parameters, and how this relates to sibling tools. The description is incomplete for a tool of moderate complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 3 parameters with 0% description coverage, meaning the schema itself provides no descriptions. The tool description does not clarify the parameters either—'repo_path', 'modified_files', and 'schema_changes' are not explained in terms of format or semantics. Since there are no enums, the agent cannot guess valid values. A score of 3 is generous because the schema's structure hints at purpose, but the lack of any explanation makes selection difficult.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool runs compiler and schema verification across impacted files, which gives a clear verb+resource combination. However, it does not distinguish this tool from its siblings like 'verify_breakage' or 'trace_fullstack_path', which might have overlapping purposes. The purpose is adequate but lacks differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives like 'verify_breakage' or 'get_route_contract'. The description does not indicate prerequisites, such as needing a git diff or pre-identified list of modified files. Without any usage context, an AI agent may misuse or underuse the tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_route_contractB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the transparency burden. It discloses the output contents and implies a read-only extraction operation, but it does not mention edge cases, external dependencies, or any side effects beyond the output.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single focused sentence that front-loads the primary action and lists the key output categories without any wasted words. It is appropriately concise and scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description names the main outputs and an output schema exists, but parameter semantics are severely under-documented (0% schema coverage). The tool needs at least a brief note on how route_path should be specified and what repo_path controls to be fully actionable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description provides no parameter-level meaning. It does not explain the expected format of route_path or the purpose of repo_path, leaving both parameters semantically under-specified.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Extracts' and identifies the resource as 'API contract for a route'. It lists concrete deliverables (HTTP method, status codes, response model, callers with confidence scores), which clearly distinguishes it from sibling tools like trace_fullstack_path and get_stack_health.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool should be used when one needs a route's API contract, but it does not explicitly state when to use it versus alternatives. No exclusions or contrast with sibling tools are provided, leaving some selection inference to the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• trace_fullstack_pathB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavioral traits. It explains what the tool returns but does not mention whether it modifies state, requires authentication, handles large repos, or what happens if targets are not found.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded with the core purpose. Every sentence adds value without repetition or unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the basic return structure but lacks guidance on parameter usage, error handling, or performance implications. Given the complexity of tracing fullstack dependencies and having zero annotation coverage, more detail is needed to be complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description should clarify parameter meanings. The description mentions a 'target' concept but does not explain the roles of target, repo_path, or symbol_or_path, nor how they interact. With multiple optional parameters, the semantics are ambiguous.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool traces a fullstack dependency chain across Frontend, API Routes, and SQLAlchemy ORM models, specifying the resources involved and the return path format. This distinguishes it well from siblings like get_route_contract or get_stack_health.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives such as get_route_contract or verify_schema_change. The description does not indicate what input is needed or prerequisites like a valid repo path.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_breakageA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It explains what the tool does (runs compiler and schema verification) and the scope (across all files impacted by a change), but it does not disclose behavioral traits such as whether it modifies files, requires specific permissions, has side effects, or what the output schema represents. This is adequate but not exceptional.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that efficiently conveys the tool's purpose without unnecessary words. It is front-loaded and earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has two optional parameters and an output schema, the description provides a solid overview of the tool's function. It could benefit from noting that both parameters are optional, but the context signals help there. The description is complete enough for an agent to understand the tool's core purpose and scope.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for the two parameters. It does not explain the meaning of 'repo_path' or 'modified_files' beyond their names in the schema. However, the description of the tool's action (running verification across impacted files) gives implicit context that 'modified_files' likely lists changed files and 'repo_path' is the repository root. This is minimal value added.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Runs' and the resource 'compiler and schema verification across all files impacted by a change', which is specific and distinguishes it from siblings like 'verify_schema_change' (which focuses only on schema) and 'get_stack_health' (which checks overall health).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a change has been made and needs verification, but it does not explicitly state when to use this tool versus alternatives like 'verify_schema_change' or 'trace_fullstack_path', nor does it mention when not to use it or any prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.