Server Quality Checklist

Tool Scores

• get_file_summaryB
  3.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full behavioral burden. It discloses the main output content (symbol table, imports, exports) and a notable exclusion (not returning file body). However, it does not mention prerequisites like requiring the file to already be indexed or what happens when the file is missing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single well-structured sentence with no filler. It front-loads the core action and contents, then adds the clarifying exclusion. Every part contributes to understanding the tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, no annotations, and zero parameter documentation, the description alone is not enough for an agent to confidently invoke this tool. It explains what is returned but not parameter semantics, indexing prerequisites, or error behavior, leaving important gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage and the description does not explain 'path' or 'repo_root' semantics. While 'single indexed file' loosely ties 'path' to a file location, 'repo_root' remains unexplained, and no format, relative/absolute path guidance, or repository boundary details are provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Return') and names a concrete resource ('symbol table... imports, and exports') for a single indexed file. It clearly conveys what the tool does and even specifies what it doesn't do ('without returning the entire file body'), though it doesn't explicitly distinguish itself from siblings like get_relevant_context or get_repo_overview.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'for a single indexed file' implies the tool is for per-file summaries, but there is no explicit guidance on when to prefer this over search_code, get_dependencies, or get_relevant_context. No exclusions or alternatives are mentioned, so usage context is only implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_dependenciesA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It states the operation is read-only ('return what a file imports...'), which is helpful, but it doesn't disclose whether the tool scans the whole repo, handles dynamic imports, or has performance implications. Given the lack of annotations, a 3 is appropriate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no fluff. The main purpose is stated first, and the use case is added compactly. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 2 params and no output schema, the description covers the primary use case, but it lacks details on repo_root semantics, return format, and any edge cases (like missing files). The complexity is moderate, so this is incomplete but not severely.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, and the description does not explain either parameter. The path parameter is obvious from the tool name, but repo_root is undocumented. With no parameter info, a lower score might be expected, but the description's mention of 'file' implies path usage; still, the gap is significant.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns a file's imports and reverse dependencies, with a specific purpose (understanding blast radius). It distinguishes itself from siblings like search_code and get_file_summary by focusing on dependency relationships.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage ('to understand blast radius before changing a file') but does not explicitly state when not to use it or mention alternatives like get_relevant_context for broader context. The context signals show siblings, but no explicit routing guidance is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_relevant_contextA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description risks the full burden of behavioral disclosure. It communicates that the tool returns a synthesized context package rather than raw files, and that it is token-limited, but it does not state whether it is strictly read-only, whether it depends on an existing repository index, or what the actual response format looks like.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured. The first sentence states the core action and output, and the second provides routing guidance. There is no filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description gives enough to orient an agent to the main purpose and broad usage, but it leaves gaps around the return structure, required prior state (e.g., indexation), and parameter behaviors. Because there is no output schema, more detail about the package shape would help.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain the three parameters. 'query' and 'token-budgeted' are lightly implied, but repo_root is entirely undocumented, meaning an agent cannot infer the expected input format or behavior for that parameter without external knowledge.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool builds a focused, token-budgeted context package of symbol tables and code excerpts for relevant files. It distinguishes itself as the 'main entry point' for find/fix requests, which differentiates it from lower-level or broader repository tools, though it does not explicitly name all sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says to use this tool for 'help me find/fix X' requests and to use it instead of reading whole files or globbing the repository. This provides a clear when and when-not, but it does not name specific alternative tools such as search_code or get_file_summary.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_codeA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It does reveal the main output behavior—ranked file hits with matching symbol/reason—and the cost advantage. However, it does not mention whether the repository must already be indexed, how repo_root is scoped, or any failure/edge behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded: it states the action, example usage, return behavior, and cost benefit in two efficient sentences. There is no fluff or redundant restatement of the tool name.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a fairly simple tool and the description covers the core purpose and return shape adequately. Still, it leaves important invocation details undocumented, especially the semantics of top_k and repo_root, plus the implied prerequisite that the repository must be indexed first.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds useful meaning for the query parameter with natural-language/keyword guidance and examples, but top_k and repo_root are not explained at all. Since schema description coverage is 0%, the description was expected to compensate for all parameters but only covers one meaningfully.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches the indexed repository for files/symbols matching a natural language or keyword query and returns ranked file hits with matching symbols/reasons. It is specific about the resource and output, though it does not explicitly distinguish it from siblings like get_relevant_context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for when to use the tool: to find relevant files/symbols via semantic or keyword search. It also notes the tool is much cheaper than scanning the whole repo, which is useful guidance, though no explicit when-not-to-use or alternative routing is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_repo_overviewA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the read-only nature by saying 'Return' and describes the output shape, but it does not mention potential prerequisites, performance implications, or what happens when repo_root is null. This is adequate but incomplete.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tight sentences with no filler. The first sentence front-loads the core behavior and output content; the second gives the usage context. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only tool with one optional parameter and no output schema, the description covers the main purpose and output well. However, the undefined parameter behavior and the relationship to index_repository leave enough ambiguity that an agent may not know how to invoke it correctly on first use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage for repo_root, and the tool description does not mention the parameter at all. The agent is left to infer that repo_root is the repository path and that null may mean the current directory, which the description does not clarify.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states exactly what the tool returns: a high-level architecture summary with top-level directories, language breakdown, and files with the most symbols. This clearly distinguishes it from sibling file-level tools like get_file_summary and search_code.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says to use this to 'orient before drilling into specific files,' which gives a clear when-to-use context. It does not name alternatives or state when not to use it, but the orientation purpose is specific enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• index_repositoryA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavior disclosure, and it does reasonably well: it declares persistence behavior ('on-disk cache for fast re-runs'), signals re-indexing with refresh=true, and describes the data that is extracted. It stops at details like what the tool returns or whether it is non-destructive, but the core behavioral traits are covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, no filler: the first defines what it does, the second gives the call-order rule applied, and the third explains caching. Important behavioral information is front-loaded, and every clause adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given a simple two-parameter schema with no output schema and no annotations, the description is close to complete: it asserts when to call it for the first time, when to re-run with refresh, what the internal behavior is (dependency graph building and caching), and what gets updated. It could specify what happens after indexing or what success looks like, but it is complete enough for this kind of setup tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does explain refresh past the schema by tying it to re-running after file changes. However, repo_root is never mentioned or defined; an agent can infer it is the repository path, but the uncertainty is left to guesswork. Since at least one parameter is semantically enriched and the other is left implicit, this is adequate but not strong.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific action and resource: 'index (or re-index) a repository' and enumerates what it extracts — files, symbols, imports, and a reverse dependency graph. It reads as an explicit foundation step, and the siblings (get_repo_overview, search_code, get_dependencies) are clearly different read/query tools rather than indexing tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description says 'Call this first, and again with refresh=true after significant file changes.' This is direct when-to-use guidance, making the invocation very and rerun conditions clear. It does not explicitly contrast with the sibling tools, but the relationship is strongly implied: this is the preliminary indexing step before querying with the get_* siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.