Server Quality Checklist

Tool Scores

• list_exportsA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, and the description only states the operation without disclosing traits like read-only nature, permission needs, or side effects. Listing exports is inherently non-destructive, but the description does not confirm this.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no wasted words. The first sentence explains what it does, the second provides usage guidance. Efficiently front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description usefully lists return fields. It covers the primary use case, but could be more explicit about folder scanning behavior (e.g., recursive by default).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions for all three parameters. The description does not add extra meaning beyond the schema, so baseline score of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists exported symbols (functions, classes, types, consts) from a file or folder, specifying return fields. It distinguishes from reading the whole file but does not explicitly contrast with sibling tools like file_outline.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear usage context: 'Use this instead of reading a whole file to discover what it contains.' This gives a reason to use the tool but lacks explicit when-not-to-use or alternative tool names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_referencesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions using ripgrep and returning file/line/text, but does not disclose behavior like performance, .gitignore handling, or safety (read-only is implied but not explicit). With annotations absent, more detail would be beneficial.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no fluff, each sentence earns its place: first defines action and method, second provides usage context. Perfectly concise and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description mentions 'Returns file, line, and matching text', covering the return format. It also gives a use case. Missing details like result truncation (maxResults) are minor; overall sufficient for a simple search tool with 4 parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description does not add significant meaning beyond the schema; it reiterates 'symbol' and 'path' but does not elaborate on format or constraints. Falls into the typical baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Find all references to a symbol across a directory using ripgrep', specifying verb and resource. It distinguishes from sibling tools like list_exports (exports) and file_outline (structure), leaving no ambiguity about what finds_references does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Includes explicit guidance: 'Use to trace call chains and understand usage before modifying', indicating when to use. However, it does not explicitly state when not to use or name alternatives, so a slight deduction applies.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• file_outlineA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool shows line ranges for each symbol, includes both exported and non-exported symbols, and implies a read-only, non-destructive operation. It does not mention performance or authorization, but for a structural outline tool, these are not critical.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences: the first specifies what the tool does and what it returns, the second provides usage advice. Every sentence adds value, and there is no superfluous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity (2 parameters with full schema coverage, no output schema, no nested objects), the description adequately explains the tool's purpose, parameters, and usage context. It does not detail the exact response format, but the note about 'line ranges' gives sufficient expectation for an outline tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, as both parameters are described in the schema. The description adds minimal extra meaning: it reinforces that path is absolute and that exportedOnly controls showing exported symbols only. The phrase 'Shows both exported and non-exported symbols' clarifies default behavior but is already implied by the schema description. Baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it retrieves a structural outline of a single file, listing symbols like functions, classes, interfaces, etc. with line ranges. It distinguishes itself from siblings: list_exports (which likely only shows exports), read_region (which reads a region), and find_references (which finds references). The verb 'Get' and resource 'structural outline of a single file' are specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description suggests using this tool 'to understand file structure before reading a specific region,' providing clear context. However, it does not explicitly mention when not to use it or name alternatives besides the implied sibling tools. This is a minor gap.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_regionA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses max 200 lines per call and returns numbered source lines. No contradictory annotations. Lacks details on error handling or side effects, but read operation is inherently safe.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no extra words. Front-loaded with key action and constraint, followed by usage guidance. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers purpose, constraints, return format, and usage guidance. Lacks mention of error cases or permissions, but acceptable given tool simplicity and no output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Adds value beyond schema by stating the 200-line limit and that output is numbered lines. Schema already describes parameters, so baseline is 3; description meaningfully enhances understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads a specific line range from a file, with a 200-line limit. It differentiates from siblings by framing it as a way to read only needed code after using list_exports or file_outline.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  States to use after list_exports or file_outline to read only needed code, implying when to use it. Lacks explicit negative guidance (e.g., when not to use), but context is helpful.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.