Server Quality Checklist

Tool Scores

• search_symbolsC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It does not mention whether the operation is read-only, how results are returned, matching semantics, pagination, or any other runtime behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief and front-loaded with the primary purpose. The second sentence about group adds a useful constraint without padding, so each sentence earns its place. It is concise, though sparse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 7 parameters, no annotations, and no output schema, this description is not complete enough for an agent to confidently invoke the tool. It lacks information about required query semantics, available filter values for kind and lang, result shape, and how the various filters interact.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, yet the description only explains one parameter: group. It provides no semantics for query, kind, lang, limit, path_glob, or max_tokens, leaving most of the parameter surface undocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action and resource: 'Find definitions, key paths and headings by name across the repository.' This identifies what the tool does and its scope. However, it does not explicitly differentiate from siblings like get_symbol or file_outline, which may overlap in function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no guidance about when to use search_symbols versus alternatives such as get_symbol, file_outline, or ast_query. The only usage-related hint is the group value enumeration, which is parameter guidance rather than tool-selection guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_importsC
  2.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the transparency burden. It does reveal that the result is scoped to a single file and that exports are included only when the language supports them. It does not state whether the operation is read-only, what the result shape is, or how max_tokens affects output.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one short sentence with no filler, and the core scope is front-loaded. The language-conditioned export clause is efficient, though slightly cryptic.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations and no output schema, this definition is too thin. An agent is not told what the result looks like, how language detection works, or how max_tokens influences the output, leaving the tool minimally viable at best.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for missing parameter details. It clarifies that 'path' refers to a single file, but it does not describe 'max_tokens,' its truncation behavior, or the expected path format.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description identifies the resource ('dependency edges out of one file') and adds an export edge case, which differentiates it from sibling tools like search_symbols or ast_query. It lacks an explicit verb such as 'lists' or 'returns,' but the tool name and phrasing make the action reasonably clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given for when to use this tool rather than file_outline, ast_query, or search_symbols. There are no conditions, exclusions, or alternative tool mentions, so an agent must infer the correct usage from the name and the bare description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_docstringsB
  3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full behavioral disclosure burden. It does reveal the key output trait ('without bodies') and a strict input requirement, but it does not explain what happens if both path and symbols are provided, how max_tokens affects results, or what the return structure looks like.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short, front-loaded, and free of filler, which is good. However, the brevity crosses into under-specification: two sentence fragments cannot adequately cover three parameters, the mutual-exclusion rule, and tool-selection context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given three parameters, no annotations, and no output schema, the description is incomplete. It does not explain accepted path or symbol formats, mention max_tokens at all, or describe the output beyond 'documentation,' so an agent must infer too much to invoke the tool confidently.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for missing parameter meaning. It adds the useful 'exactly one of path or symbols' constraint, but it never defines what path should point to, what symbols should contain, or what max_tokens controls. This leaves significant semantic ambiguity for three parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The phrase 'Documentation without bodies' combined with the tool name get_docstrings identifies a specific resource (docstrings) and a distinctive scope (no function bodies), which helps distinguish it from sibling tools like get_symbol or file_outline. It stops short of an explicit imperative statement such as 'Returns docstrings for a path or symbols,' so clarity is good but not maximal.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The instruction 'Pass exactly one of path or symbols' is a clear invocation rule and implies the tool is for retrieving documentation rather than symbol bodies. However, the description never names alternative tools or states explicit when-to-use versus when-not-to-use conditions, leaving sibling differentiation mostly implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_symbolB
  3.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral burden and does substantial work: it explains mode behavior ('source' returns the body, 'signature' and 'doc' omit it), ambiguity resolution (returns candidates rather than guessing), and line-based disambiguation by start_line. It does not cover errors, permissions, or max_tokens effects, but the core selection behavior is transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded with the main purpose. The mode and line notes are dense but scannable. It avoids filler, though the ambiguity discussion could be tightened slightly without losing value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a moderate-complexity tool with five parameters and no output schema, the description covers the primary ambiguity-handling behavior and mode semantics. It does not explain max_tokens, the exact meaning of path, or how the returned candidates/symbols are structured. It is adequate for basic use but incomplete for full confident invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must add parameter meaning. It explains mode values and the line parameter, and mentions lookup by name, key path, or heading slug. However, it does not map 'key path' and 'heading slug' to specific parameters, and it leaves max_tokens entirely undocumented. Partial compensation, but with real gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The first sentence states a clear action and resource: 'Fetch definitions by name, key path, or heading slug.' It conveys the tool's core function and gives some sense of the lookup keys. It does not explicitly contrast with sibling tools like search_symbols, but the specificity of fetching definitions rather than searching or outlining is enough to distinguish it.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance is given about when to prefer get_symbol over siblings such as search_symbols, get_docstrings, or ast_query. The mode and line details are usage-related, but they explain how to call the tool, not when it should be selected. The description leaves alternatives and exclusions to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ast_queryA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full behavioral disclosure burden. It only says what the tool does, not whether it is read-only, what the output looks like, whether the file can be modified, or any error/edge-case behavior. The word 'query' hints at a read operation, but this is not explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no fluff. The primary action is front-loaded, and the second sentence adds valuable positioning as an escape hatch. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, no annotations, and 0% schema description coverage, the description is insufficient for a 4-parameter tool. It does not mention return format, capture behavior, token limits, or how this low-level query relates to the typed tools beyond being a fallback. Agents would likely need extra probing to use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It indirectly explains 'path' ('against one file') and 'query' ('raw tree-sitter S-expression query'), but gives no guidance on 'captures' or 'max_tokens' semantics, defaults, or usage. With 4 parameters and no schema descriptions, this leaves significant ambiguity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific action: 'Run a raw tree-sitter S-expression query against one file.' This clearly identifies the verb, resource, and scope. It also differentiates itself from siblings by calling itself 'the escape hatch for anything the five typed tools flatten away,' signaling it is the lower-level fallback.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context by positioning the tool as an escape hatch, implying it should be used when the five typed tools (search_symbols, get_docstrings, file_outline, get_symbol, list_imports) cannot express the desired query. It does not name specific sibling alternatives or give explicit when-not-to-use conditions, but the context is actionable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• file_outlineA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden. It does disclose a key behavioral trait: bodies are elided, so the output is a structural outline rather than full content. It does not mention output limits, token truncation, or depth semantics, but for a read-only outline tool the disclosed behavior is the most important part.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences front-load the core purpose and immediately add usage context. There is no redundant or filler content; every sentence contributes to effective tool invocation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is sufficient for a basic call (path) and tells the agent why to use it, but it omits parameter semantics and does not define the output format beyond 'definitions, key paths or headings, bodies elided.' With no output schema or annotations, the description leaves moderate gaps in what an agent can expect and how to tune the call.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% and the description says nothing about max_depth, max_tokens, or include_docstrings. The parameter names and defaults are somewhat self-explanatory, but the description does not compensate for the schema's lack of textual explanation, leaving the agent to infer how these parameters affect the outline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb-resource pair ('Outline one file') and specifies what the result contains: definitions, key paths or headings, with bodies elided. It does not explicitly differentiate itself from sibling tools like search_symbols or get_symbol, but the whole-file-overview scope makes its purpose clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a clear when-to-use directive: use this tool instead of reading a file to find out what is in it. It does not, however, name sibling alternatives or state when to choose them, so the guidance is clear but not exhaustive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.