Server Quality Checklist

Tool Scores

• pingA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations available, the description carries the behavioral disclosure burden. It clearly states the exact return format and gives a concrete example, which is strong for a simple ping tool. It does not mention failure modes or side effects, but 'Health-check' implies a non-mutating liveness probe.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence plus one illustrative example, with no filler. It packs the core behavior, the return format, and parameter usage into a minimal, front-loaded structure.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple health-check tool with one required string parameter and an output schema, the description fully covers what an agent needs: what the tool does, what input it expects, and what output to expect. Nothing essential is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides no property descriptions, so the tool description must compensate. The example 'ping(message="hello") → "pong: hello"' demonstrates that the message parameter is echoed back inside the pong string, effectively conveying its semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description identifies a specific verb ('Health-check') and explicitly defines the tool's behavior: it returns 'pong: <message>'. This clearly distinguishes it from the analysis-oriented sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'Health-check' label and echo-like example make the intended use obvious, but the description does not explicitly state when to use it versus alternatives. Given that the sibling tools are clearly unrelated, the implied usage is sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_perf_outputA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden and goes beyond a one-line summary by specifying filtering semantics, default values, and the full return shape including warnings. It does not detail malformed-input failure modes, but this is a read-only parsing operation and the core behavior is transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, followed by usage, parameters, return shape, and a practical example. Every section earns its place, especially since there is no output schema and annotations are absent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a parser with no annotated safety profile and no output schema, the description supplies the essential context: when to use it, what each parameter means, what the returned object looks like, and an example combining parameters. No critical information is missing for an agent to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides no per-parameter descriptions, so the description fully compensates by defining each argument: raw text input, top_n as the maximum number of symbols, and min_overhead_pct as a filter threshold with defaults. This gives an agent everything needed to set the parameters correctly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb and resource: parse perf report --stdio text and return top hot symbols. This clearly conveys what the tool does and is distinguishable from sibling tools like ping, suggest_neon_intrinsic, and check_arm64_deps, which address unrelated tasks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says to use the tool when a user pastes perf report --stdio output and wants CPU-consuming functions. It does not mention when not to use it, but the sibling tools are in different domains, so no exclusions or alternatives are necessary.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• suggest_neon_intrinsicA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the behavioral burden. It discloses the similarity-based ranking, architecture filtering ('Filtered to armv8-a'), and the exact return shape including notes about knowledge-base counts. It stops short of describing error/empty-match behavior, but it provides substantial transparency for a read-only suggestion tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well organized into purpose, usage conditions, Args, Returns, and Example. The example return object and worked call are informative rather than redundant. Every section earns its place and supports correct invocation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description supplies a full return structure, a notes field explanation, and a ranking example. It covers parameters, defaults, and expected results comprehensively for a tool of this complexity. No critical information needed to select or use the tool is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must fully compensate, and it does. It explains operation_description with a concrete example, enumerates valid target_arch values and the default, and states top_k's default and purpose. An agent can call this tool correctly without needing additional parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a precise statement: 'Suggest NEON SIMD intrinsics for an operation description, ranked by semantic similarity.' This names the exact verb, resource, and ranking behavior, making the tool's purpose unmistakable. It is also clearly distinct from sibling tools like analyze_perf_output and check_arm64_deps.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly lists when to use the tool: for a hot loop, Arm64 vectorization, or when the user mentions intrinsic names like vmlaq_f32. It does not give explicit when-not or alternative tool guidance, but the sibling tools are topically distant enough that confusion is unlikely.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_arm64_depsA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explains return structure, defines severity levels (error/warning/info) with concrete meanings, and gives a realistic example of outputs for cupy, tensorflow, and numpy. This goes well beyond a bare 'scan' statement.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is longer than average but every section earns its place: purpose, use case, arguments, return format, severity semantics, and example. It is well-structured and front-loaded with the core purpose before any detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is no output schema and no annotations, the description provides a complete picture: what inputs are needed, what the return object looks like, how issues are categorized, and a concrete example. An agent has enough information to invoke the tool correctly and interpret results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for missing parameter documentation. It fully explains both parameters: file_content as raw manifest text, and file_type with exact allowed values ('requirements.txt', 'pyproject.toml', or 'Dockerfile'). It also demonstrates the call signature with an example.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The opening sentence names a specific action and resource: scanning a dependency manifest for arm64 compatibility issues. It also clarifies the subject (Python dependencies) and target hardware (Arm64 servers like AWS Graviton), making it easy to distinguish from the unrelated sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool: when a user wants to know whether Python dependencies install on Arm64 or why a package fails on arm64. It provides clear context, but it does not mention exclusions or alternative tools, so it stops short of a full when-not/alternative guide.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.