Server Quality Checklist

Tool Scores

• semantic_searchB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are supplied, so the description carries the behavioral disclosure burden. It accurately conveys that this is a read-only ranked search and lists concrete output fields, but it does not disclose what happens when the code has not been indexed, how rankings are determined, or any failure behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise, information-dense sentences. It front-loads the tool purpose and immediately follows with the concrete return value shape. There is no filler or repetition of schema details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The output schema likely covers return-value details, but the description still lacks important contextual guidance like 'run index_workspace first' and when to choose semantic_search over find_ast_usages. It is adequate for a straightforward search tool but not fully complete for an autonomous agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description should compensate for the input parameters, but it only explains output fields. The parameter names and defaults provide some minimal intuition, but query, limit, and directory_path receive no real semantic guidance, especially around how the directory_path interacts with the indexed workspace.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly identifies a semantic/natural-language search over indexed code and lists the enrich return fields (file, class, method, line numbers, code snippets). This makes it distinguishable from index_workspace and find_ast_usages in substance, though it never explicitly compares against those sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'Natural-language search over indexed code' implies when to use it, but there is no explicit when-to-use/when-not-to-use guidance, no direct mention of find_ast_usages for exact-symbol queries, and no note that the workspace must be indexed before this tool is useful.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_ast_usagesA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations supplied, the description carries the full behavioral burden. It does disclose a valuable filtering behavior: "Filters out string literals, comments, and non-call occurrences." However, it does not mention whether indexing is required, how missing symbols are handled, or any result behavior beyond being AST-based.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single focused sentence that communicates the mechanism, target resource, and exclusion behavior without redundancy. Every clause adds useful information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple enough that the core lookup behavior is adequately described, and an output schema exists. The main gaps are the lack of usage guidance around semantic_search/index_workspace and the absence of practical details about scoping and limit behavior, which leaves the description merely adequate rather than complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description should compensate. It adds meaning for symbol_name by stating it refers to "a function or class," but it gives no guidance on directory_path or limit, such as how the directory is searched or what the limit controls. This leaves non-obvious semantics undocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb plus precise resource: "AST-based lookup of real call sites / instantiations of a function or class." It also clarifies what is excluded, which distinguishes it from string or semantic search. This is far more informative than the bare tool name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The AST-based wording implicitly indicates this is for exact code-structure matching rather than fuzzy search, and the sibling tools suggest a semantic-search alternative. However, the description does not explicitly state when to use this tool versus semantic_search or index_workspace, nor does it mention any exclusions or preconditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• index_workspaceA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It covers that the operation walks the project, uses SHA-256 hashing for change detection, and mutates a persistent local ChromaDB store. This is meaningful behavioral context beyond 'index workspace' though it does not mention potential costs, permissions, or failure modes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence with no filler. It front-loads the core action and packs in the hashing, incrementality, and storage target without becoming verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one optional parameter and an output schema, the description covers the essential behavior: what is walked, how changes are detected, and what is updated. It is reasonably complete, though it could add an explicit note about running after source changes or before semantic_search.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There is only one parameter and schema description coverage is 0%, so the description needs to clarify the parameter's meaning. It never mentions directory_path or the default of '.', forcing the agent to rely on the schema's 'Directory Path' title and default value. The description does not compensate for the low schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific action sequence ('Walks the project', 'detects changed files', 'updates the vector store') and a concrete resource ('persistent local ChromaDB vector store'). It is clearly distinct from the sibling tools semantic_search and find_ast_usages, which do search and AST lookup rather than indexing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'incrementally updates' and 'detects changed files' implies the tool is meant to keep the index fresh after edits, but the description never explicitly says when to run it relative to semantic_search or find_ast_usages. It also does not mention exclusions or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.