Server Quality Checklist

Tool Scores

• query_chainsA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It describes what the tool does (queries and returns clusters) and the scope ('across all services indexed by the local Ariadne DB'), but lacks details on permissions, rate limits, error handling, or response format. For a query tool with no annotations, this is adequate but has gaps in behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with two sentences that efficiently convey purpose and usage guidelines without unnecessary details. Every sentence adds value, making it concise and well-structured for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (querying cross-service chains) and lack of annotations and output schema, the description is moderately complete. It covers purpose and usage well but misses behavioral details like response format, pagination, or error cases. For a tool with no output schema, more information on return values would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with both parameters well-documented in the schema. The description adds minimal value beyond the schema by mentioning 'business term or endpoint name' for 'hint' and implying clustering for results, but doesn't provide additional syntax or format details. Baseline 3 is appropriate as the schema handles most parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('query', 'returns') and resources ('cross-service chains', 'candidate clusters of related GraphQL operations, HTTP endpoints, Kafka topics, and frontend queries'). It distinguishes this tool from siblings by specifying it queries by 'business term or endpoint name' and returns clusters across services, unlike expand_node, rate_result, rescan, or show_help.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool: 'Use this when you need to understand which APIs, topics, or frontend operations are involved in a business feature.' This provides clear context for usage versus alternatives, though it doesn't name specific sibling tools, the guidance is sufficient for distinguishing use cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• expand_nodeA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behaviors: 'Read-only; no writes except an implicit positive feedback row if called within 10 min of a matching query_chains. Returns up to 3 matched source nodes × up to 10 neighbours (edges with score ≥ 0.08), plus a `stale_warning` field — call `rescan` if non-null.' This covers safety (read-only with one exception), output structure, and side effects, though it could mention error handling or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and well-structured, with the first sentence stating the core purpose and subsequent sentences providing usage guidelines and behavioral details. Every sentence adds value, though it could be slightly more front-loaded by moving key behavioral traits earlier.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description does a good job of covering the tool's complexity. It explains purpose, usage, behavior, and output structure. However, it could be more complete by explicitly mentioning the tool's return format (e.g., JSON structure) or potential error cases, which would help an AI agent better handle responses.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, so the baseline is 3. The description adds valuable context beyond the schema: it explains that 'Partial, case-insensitive match against node id and raw_name; ambiguous inputs return multiple source groups,' and clarifies the relationship to 'query_chains' results. This enhances understanding of how the parameter is used, though it doesn't introduce new parameter-specific details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'One-hop neighbours of a known node... with similarity scores and file paths.' It specifies the verb ('expand') and resource ('node'), and distinguishes it from sibling 'query_chains' by noting it's for tracing 'one hop further' after having a concrete node name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: 'Use AFTER query_chains when you already have a concrete node name and want to trace one hop further. Use query_chains (not this) when starting from a business term or when you don't yet know a node name.' It clearly defines when to use this tool versus the alternative 'query_chains'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• rate_resultA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It reveals that feedback is stored locally in feedback.db and survives DB rebuilds, and explains inference behavior. No side effects or permission needs are mentioned, but overall transparent enough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, front-loaded with purpose and usage, then provides details about inference and storage. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity and no output schema, the description is complete enough. It explains behavior, parameters, and storage. Minor gap: no return value description, but not critical for a feedback tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds meaning: explains that node_ids can be inferred from hint and cluster_rank if omitted, and clarifies cluster_rank usage (0 for expand_node). This adds value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose ('Record whether Ariadne results were useful') and distinguishes it from siblings by specifying it is used after query_chains or expand_node.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says when to call this tool ('after using query_chains or expand_node') and explains inference behavior when node_ids is omitted, providing clear usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• rescanA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes what the tool does: re-scans repos from a config file, rebuilds TF-IDF edges, and invalidates caches. It mentions 'No arguments; zero configuration' which clarifies operational simplicity. However, it doesn't cover potential side effects like performance impact or error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, starting with the primary use case and then detailing the actions. Every sentence earns its place by providing essential information without redundancy, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (index refresh with multiple actions) and no annotations or output schema, the description is mostly complete. It covers purpose, usage, and behavior well, but lacks details on output (e.g., success confirmation or error messages) and potential limitations like execution time or prerequisites.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so the baseline is 4. The description adds value by explicitly stating 'No arguments; zero configuration,' which reinforces the parameterless nature beyond what the schema indicates, helping the agent understand there are no optional or hidden parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Refresh', 'Re-scans', 'rebuilds', 'invalidates') and resources ('Ariadne index', 'TF-IDF token edges', 'cached DB handles'). It distinguishes from siblings by explaining when to use it versus query_chains or expand_node, which return stale_warnings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool: 'Call this when query_chains or expand_node returned a stale_warning, or after you know the user's code has changed.' It provides clear alternatives by naming specific sibling tools and scenarios for usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• show_helpA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses key behavioral traits: it's safe (no destructive effects), requires no database, and returns guidance. However, it doesn't specify format, length, or potential rate limits, leaving some gaps in behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, followed by usage guidelines and safety note. Every sentence earns its place by providing essential information without waste, making it highly efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (0 parameters, no annotations, no output schema), the description is nearly complete. It covers purpose, usage, and safety. However, without an output schema, it could briefly mention the type of guidance returned (e.g., text or structured data), leaving a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description adds value by implying no inputs are required, aligning with the schema. Baseline is 4 for 0 parameters, as it appropriately handles the lack of parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Return a quick setup and usage guide') and resource ('for Ariadne'). It distinguishes from siblings by focusing on help/guidance rather than operations like querying, expanding, rating, or rescanning.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides explicit guidance on when to use this tool ('Call this first when you are unsure how to use Ariadne, how to index your own microservices, or why query_chains returned no results') and when it's safe ('Always safe to call — no DB required'), effectively differentiating it from alternatives like query_chains for troubleshooting.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.