Server Quality Checklist

Tool Scores

• get_detailsB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description bears full burden. It discloses that it returns structural facts and optionally a docstring, but does not detail permissions, errors, or side effects. Adequate but minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise: two sentences with no wasted words. Core purpose is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Tool is simple with one parameter and no output schema. Description covers core behavior but lacks details on output structure, errors, or edge cases. Adequate but could be more comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  While the input schema only specifies type 'string' for 'node', the description adds useful context that the node id originates from get_context. This compensates for the 0% schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states that the tool returns 'Full structural facts for one node' and optionally includes its docstring. It ties the input to get_context, which helps differentiate from siblings like get_context and get_dependencies, but lacks explicit differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. Only implies that the node id should come from get_context. No mention of when not to use or other considerations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_callersA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears full burden. It discloses the key behavior (direct/transitive callers) but omits details like recursion depth limits, performance implications, or how transitively is defined. Adequate for simple use.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, no redundant words, front-loaded with the key action. Every part is essential and informative.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given low complexity (1 param, output schema present), the description covers the core functionality. It explains parameter origin and result scope. Could mention if output is function IDs or names, but output schema likely handles that.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description adds meaning by specifying that `fn` is a node ID from get_context/get_details, giving context for parameter usage. No examples or validation constraints provided, but valuable addition beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool returns functions that call a given function `fn`, with direct or transitive calls. Differentiates from sibling tools by focusing on callers rather than dependencies or hierarchy. Could be slightly more explicit but is clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides context that `fn` comes from get_context/get_details, which guides where to obtain the input. However, it does not explicitly state when to use this tool over alternatives like get_dependencies, leaving agents to infer from purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_class_hierarchyA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It describes the core behavior (transitive ancestors/descendants) but lacks details on permissions, rate limits, or side effects, which are acceptable for a read-only query.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no redundant words. It could be slightly improved by clarifying the output structure, but it is appropriately concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity, the description covers the input and output goal but does not specify whether the output is a flat list or hierarchical, nor its exact structure, leaving some ambiguity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, but the description adds crucial context: the 'cls' parameter is a node id from get_context/get_details, which compensates for the schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it returns 'transitive ancestors and descendants' of a class, which distinguishes it from sibling tools like get_callers or query_sparql.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It specifies the prerequisite (node id from get_context/get_details) and implies use for class hierarchy queries, but does not explicitly exclude alternatives or state when not to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_sparqlA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the default behavior (use_default_graph_as_union=True) and explains graph partitioning. However, it does not state whether queries are read-only or any side effects, which is a minor gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is relatively long but well-organized, starting with prefix/IRI definitions, then classes/properties, then named graphs, and usage guidance. It could be trimmed slightly but is effectively structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema (context shows true), return values are likely covered elsewhere. The description thoroughly explains ontology and graph context needed for queries, but omits possible SPARQL construct restrictions or output structure, which is a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema coverage, the description must compensate but only provides ontology context for writing queries, not syntax or constraints for the query parameter itself. The ontology info is valuable, but the parameter semantics remain vague.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool executes SPARQL queries against a code ontology, with specific details on prefixes, node IRIs, classes, and properties. It distinguishes itself from sibling tools like get_callers or get_class_hierarchy by being a generic query tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on graph selection: plain WHERE searches all graphs, while an explicit GRAPH block targets inferred edges. It does not mention when to avoid this tool or directly compare to siblings, but the context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_dependenciesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the core behavior: transitive dependency resolution via import edges. However, it omits details like cycle handling or response shape, though the output schema exists to cover return values.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no wasted words. It is front-loaded with the key action and resource.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given one parameter, no annotations, and an existing output schema, the description adequately explains the tool's function. It could mention what the returned list contains (file paths), but the output schema likely covers that.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter 'file' has no schema description (0% coverage), so the description compensates by specifying it expects a 'workspace-relative path', which adds meaningful context beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it returns every file a given file transitively depends on via imports. It distinguishes itself from siblings like get_callers and get_class_hierarchy by focusing on import dependencies, not callers or class hierarchy.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for dependency analysis but lacks explicit guidance on when to use this tool versus its siblings (e.g., get_callers, get_context). No mention of when not to use or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_contextA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions 'token-cheap' (cost awareness) and the structural nature, but does not explicitly state read-only or side-effect-free behavior. Adequate but not fully transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core purpose, then specific details and related tool. No extraneous words. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with 2 params and no output schema, the description covers what it returns, how to use the parameters, and how it relates to a sibling tool. It is complete for the context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description must add meaning. It explains 'path' as workspace-relative and 'depth' as drilling from folder to detail levels. This adds significant value beyond the schema's names and default.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns a 'structural summary' with 'names, signatures, line ranges' for a file or folder path. It distinguishes from get_details by excluding docstrings. This is specific and differentiates from siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises using get_details for docstrings, implying that get_context is for structural info only. It does not explicitly state when not to use it, but the guidance is clear and helpful.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.