Server Quality Checklist

Tool Scores

• check_index_statusA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses it checks SQLite database and is lightweight. Explains return values and interpretation. No annotations provided, so description carries full burden; it is thorough but could explicitly state it is read-only.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with sections and bullet points, front-loading the purpose. While somewhat verbose, every section adds value and aids navigability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers all relevant aspects: purpose, triggers, interpretation of results, and return values. Output schema exists and is explained. Given the tool's simplicity and low parameter count, the description is complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description adds only a brief line for the directory parameter ('Path to the project directory to check'), which adds minimal value beyond the schema's type definition. No additional constraints or examples are provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool checks if the codebase has been indexed and whether search tools will return results. It distinguishes itself from siblings like search_code and search_docs by specifying its diagnostic role.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit triggers for when to call (before search tools, after empty results, new project) and when not to use (for indexing, searching). Includes alternative tool names like index_codebase.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_index_statsA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It details the return dictionary structure and fields, indicating it is a read-only operation. It does not mention performance or side effects, but for a stats tool this is sufficient. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is front-loaded with 'USE THIS TOOL' and structured into when/why, args, returns. Every sentence adds value; no fluff or repetition. Length is appropriate.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given one parameter and stated output schema existence, the description covers all necessary aspects: purpose, usage, parameter, and return structure. It is complete for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% description coverage for parameters; the description compensates by clearly stating 'directory: Path to the project directory.' This adds meaning beyond the schema's minimal type definition. For one parameter, this is adequate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool's purpose: to get comprehensive statistics about the code index. It lists specific metrics like file counts, symbol distributions, and database size. It distinguishes itself from siblings by noting what not to use it for (check_index_status, search_code).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear when-to-use scenarios: understanding index, debugging search quality, checking freshness, monitoring database. Also gives explicit when-not-to-use examples and directs to alternative tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• index_codebaseA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It details two operations (code indexing with AST, documentation indexing with embeddings), incremental indexing, parallel processing, and advantages over file search. It could mention potential time or resource consumption for large repos, but overall provides rich behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is relatively long but well-structured with front-loaded key instruction, bullet points, and a clear 'do not use' section. Every part adds value, though minor trimming is possible.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity and the presence of an output schema, the description covers indexing behavior, advantages, parameters, and return summary (files_indexed, total_symbols, etc.). It also mentions environment variable. No gaps found.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains 'directory' as a valid path and 'cpu' with detailed behavior including default logic and env var override. This adds full meaning beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states this tool builds the searchable index for code intelligence features. It distinguishes itself from siblings like search_code and search_docs by noting they depend on this index.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'YOU MUST CALL THIS TOOL FIRST' and lists triggers for when to call it (new session, empty results, etc.). It also states what not to use it for (non-code files, single-file lookups, git history) and points to alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_dead_codeA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Since no annotations are provided, the description fully bears the burden. It explains the mechanism (cross-referencing symbol and reference tables), details heuristics applied (e.g., excluding dunder methods, test files, etc.), and discloses limitations (cannot detect reflection, dynamic dispatch, etc.). It also describes the output format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with sections (PREREQUISITE, HOW IT WORKS, TRIGGER, HEURISTICS, LIMITATIONS, Do NOT use). However, it is somewhat verbose with detailed lists of triggers and heuristics; some minor trimming could improve conciseness while retaining clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity and that an output schema is provided, the description is exceptionally complete. It covers prerequisites, mechanism, heuristics, limitations, return format, and parameter details. No gaps are apparent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description explains all five parameters: directory (required), min_confidence (default 0.5, range 0.0-1.0), kinds (default ['function','method','class'], allowed values enumerated), include_tests (default false), top_k (default 50, max 500). It adds meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'find functions, methods, and classes that look like dead code (defined but never called). It distinguishes itself from siblings like search_code by specifying what it should not be used for.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit triggers for when to call the tool (user queries about dead code) and explicitly lists when not to use it, mentioning alternative tools like search_code. It also includes a prerequisite (indexing) and a prerequisite action.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_codeA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses behavioral traits: requires indexing (prerequisite), uses hybrid retrieval, return format for each search_type, and error conditions (empty results prompt indexing). It even mentions using Reciprocal Rank Fusion.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but well-structured with sections, headings, and bullet points. It front-loads the crucial message and every sentence serves a purpose. Slightly verbose but justified by tool complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (multiple search types, output schema), the description is comprehensive. It covers prerequisites, when to use/not use, examples for each search_type, and return format details. Differentiates from siblings effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description adds detailed semantics for each parameter: query explained per search_type, search_type enum values described with use cases, and directory specified as project path. Example queries further clarify usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it performs semantic code search using hybrid retrieval, far more intelligent than grep/glob/find. It distinguishes from sibling tools like search_docs, search_history, and file readers. Each search_type is explicitly defined with use cases.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool (over grep/glob/find) and when not to (file reading, git history, documentation). It includes detailed scenarios for each search_type, with example queries and alternatives to common grep commands.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_docsA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses behavior: it uses hybrid retrieval (BM25 + dense vector + RRF), requires indexing first, and returns specific fields. It clearly states it is read-only and does not search code. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with sections (PREREQUISITE, TRIGGER, IMPORTANT, Do NOT use) and front-loads the main purpose. It is slightly long but every sentence adds value; minor redundancy could be trimmed but overall effective.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and an output schema described in the description, the description covers all parameters, prerequisites, usage differentiation, and return format. It is complete for a search tool, addressing common agent questions about when and how to use it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description adds meaning to each parameter: query (natural language question, conversational), directory (path to project), top_k (max results, default 10, max 100). This compensates fully for missing schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it searches documentation for conceptual understanding and 'how does X work?' questions. It specifies verb 'search' and resource 'markdown documentation, READMEs, and code docstrings', and distinguishes from sibling tools like search_code by explicitly stating it is not for code implementations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use scenarios (conceptual questions, setup, architecture) and when-not-to-use (function definitions, code references, git history). It names alternative tools (search_code, search_history) and includes a prerequisite (indexing) and trigger questions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_historyA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description covers behavior thoroughly: operates on local Git repo, no indexing needed, explains each search type's behavior (e.g., blame shows line-by-line attribution, file_history follows renames). It does not contradict any annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (TRIGGER, WHEN TO USE EACH, DO NOT USE) and front-loaded purpose. While somewhat lengthy, each part adds value, and the format aids readability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 4 search types and an output schema, the description covers all necessary aspects: usage scenarios, parameter roles, return variations. It is fully adequate for an agent to invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema coverage, the description adds rich meaning for each parameter per search type, including required/optional context, examples, and argument details. This far exceeds baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool is for Git history queries with specific verb and resource, and explicitly distinguishes from sibling tools (search_code, search_docs) by listing what not to use it for.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit triggers (e.g., 'Why was this code changed?') and detailed when-to-use guidance for each search_type, including when not to use and alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.