Server Quality Checklist

Tool Scores

• inspect_stackA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full behavioral burden. It does disclose that the result is a JSON string containing StackInfo with resolved versions and evidence sources, and notes the default project directory. It does not mention failure modes, read-only guarantees, or whether inspection is limited to local files, but it is adequate for a clearly inspection-oriented tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-structured with a single verb-first sentence followed by Args and Returns sections. Every line earns its place and there is no redundant prose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one optional parameter and an output schema, the description covers the input semantics, default behavior, and return shape. It is largely complete; the only gap is that it does not clarify when to use this tool over its dependency-focused siblings.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no property descriptions (0% coverage), but the Args section fully compensates: project_dir is defined as the root directory of the Python project to inspect and defaults to the current directory. This is exactly the semantic meaning an agent needs beyond the raw type and default.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description states a specific action—detect—and the exact subject: Python version, package manager, resolved dependencies, and supported libraries. This is clear about what the tool does, but it does not explicitly differentiate it from the sibling tools resolve_constraints and freshness_audit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given about when to use inspect_stack instead of resolve_constraints or freshness_audit. The description explains what the tool does but not the conditions or prerequisites that would select this tool over its alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• freshness_auditA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the analysis method (static AST, deterministic), implies read-only behavior, and describes the return format (JSON string with violations, severity, replacements). This is solid transparency for an analysis tool, though it could mention limitations or prerequisites.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-structured: a front-loaded purpose sentence, a methodological note, then Args and Returns sections. Every sentence earns its place with no redundant text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with two parameters and an output schema, the description covers purpose, method, parameter semantics, and return value. It is nearly complete, but could add a note about what makes documentation 'authoritative' or any system prerequisites. Overall, an agent has enough to call it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It fully explains both parameters: 'code' is the Python source snippet or module to audit, and 'project_dir' is the root directory for grounding version context. This adds meaning beyond the schema's bare property titles.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Analyze Python code to detect deprecated APIs, version mismatches, and outdated patterns.' This clearly distinguishes it from siblings like inspect_stack and resolve_constraints, which address different concerns.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on what the tool does and its approach ('deterministic static AST analysis grounded in authoritative documentation'), implying when it should be used for freshness auditing. However, it does not explicitly name alternatives or state when not to use it, so it falls short of the highest bar.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• resolve_constraintsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral disclosure burden. It transparently states what the tool determines, identifies, and returns, including the JSON response shape. It does not explicitly say whether it is read-only or may make network calls, but the verbs 'determine' and 'retrieve' strongly imply a non-mutating analysis operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a summary paragraph, an Args section, and a Returns section. Every sentence contributes meaningful information without redundancy or padding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 3-parameter tool with no annotations, the description is mostly complete: it covers purpose, parameters, and return type. It could further strengthen sibling differentiation and clarify expected behavior with external resources, but the core invocation context is sufficiently specified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must fully compensate. It does: task_description is explained as the coding task description, libraries as an optional list or mapping with an example, and project_dir as the Python project root with a default. This adds meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb-resource pair: 'Determine exact project dependency versions and retrieve authoritative version-specific constraints.' It also enumerates concrete outputs (APIs, deprecated/forbidden patterns, evidence sources), which clearly distinguishes it from siblings like inspect_stack and freshness_audit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives practical usage context: it takes a task description, allows optional library filtering, and targets a Python project directory. It does not explicitly name alternatives or exclusion criteria, but the purpose is clear enough for an agent to infer when this tool applies.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.