Server Quality Checklist

Tool Scores

• list_indexed_projectsA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must reveal behavioral traits itself. It only states that it returns a dictionary of indexed projects, with no mention of performance, pagination, ordering, permissions, or side effects. This is minimal for a read-only operation, though not misleading.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded, with the main purpose stated first and a brief return-type note second. No unnecessary words or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple zero-parameter list operation with an output schema, the description covers the core purpose and return type. However, it lacks usage guidance and behavioral details (e.g., what 'indexed' means, potential performance implications), making it adequate but not comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no parameters, so the description does not need to explain parameter semantics. It adds no extra info beyond the schema, but this is acceptable given the zero-parameter design.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (List) and the resource (projects that have been indexed), and the 'indexed' qualifier distinguishes it from sibling tools like search_codebase or get_files. The return type is also mentioned, adding context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool vs alternatives is provided. The name and description imply it is for enumerating indexed projects, but there is no mention of alternative tools or exclusion scenarios, leaving usage context implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_filesA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses return type ('List of file metadata dicts') and error behavior ('error dict when project is omitted'), which is helpful. However, it does not mention side effects, permissions, or pagination, and it incorrectly states project is required when the schema allows null.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with Args and Returns sections, and each line adds value. The 'required' note is slightly redundant given the schema, but overall it is concise and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool, the description covers purpose, parameter semantics, return type, and error case. The absence of an output schema is partly mitigated by describing the return as metadata dicts, though it does not detail the dict structure or distinguish from get_file_content.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It explains the project parameter meaning and suggests using list_indexed_projects() to discover valid values. Yet it claims the parameter is required, contradicting the schema's default null and non-required status, which undermines the value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description uses specific verb 'List' and resource 'files that have been indexed for a project', clearly distinguishing it from siblings like get_file_content (content retrieval) and search_codebase (search). The scope is precise and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description points to list_indexed_projects() for discovering project names, providing clear context and a prerequisite. However, it does not explicitly contrast with alternatives or state when to prefer this tool over siblings like search_codebase or get_file_content.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_codebaseA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses the return format (dictionary with results, metadata, timing) and that the search operates on an indexed codebase. However, with no annotations, it does not explicitly state read-only behavior, side effects, or any rate limits or authentication requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with dedicated Args and Returns sections, each parameter is concisely explained, and the primary purpose is front-loaded in the first sentence.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers query parameters and return structure well, but it omits the dependency on the codebase being indexed (likely via reindex_project). This is a meaningful gap given the sibling tools and the workflow context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description provides detailed explanations for every parameter, including default values and an example for file_types. Since the input schema has no descriptions (coverage 0%), this fully compensates and adds significant meaning beyond the structured schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches an indexed codebase for code snippets, a specific verb+resource combination. This distinguishes it from sibling tools like reindex_project or get_files.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like get_files or list_indexed_projects. It does not mention prerequisites such as the need to index the codebase first, nor does it offer exclusions or alternative tool suggestions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reindex_projectA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries the burden. It discloses that force deletes existing index, but does not explicitly state that reindexing modifies the shared index or any prerequisites/permissions. Some behavioral detail provided, but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise purpose line followed by structured Args/Returns sections. Every sentence adds value with no repetition of schema types.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers purpose, parameters, return value, and a usage cue. Since no annotations and no sibling differentiation beyond purpose, it could explicitly state side effects ('modifies stored index') but this is implied by the verb. Mostly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Description's Args section explains each parameter beyond the schema: project_path is an absolute path, project_name names the collection, and force deletes existing index before reindexing. Schema itself has no descriptions (0% coverage), so this fully compensates.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description uses a specific verb 'Re-index' with a clear resource 'project', and explicitly notes usefulness after major code changes. This distinguishes it from sibling read/search tools like search_codebase and get_files.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  States context for use ('useful after major code changes'), which implies when to use it. Does not explicitly mention alternatives or when not to use, but the context is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_file_contentA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It transparently describes the validation behavior, error condition for non-indexed paths, and return type as a string. It does not address edge cases like binary files or encoding, but this is a simple read tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured with Args and Returns sections. Every sentence is informative with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with two parameters and an output schema present. The description covers purpose, parameter behavior, and error handling, making it complete for an agent to select and invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no descriptions for parameters (0% coverage), so the description fully compensates. It explains path as the file to read and project as an optional validation parameter that raises an error if the path is not indexed, adding significant semantic meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns the full content of a given file path. This specific verb+resource phrasing distinguishes it from sibling tools like search_codebase or list_indexed_projects.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains how to use the project parameter but does not explicitly state when to use this tool versus alternatives. Usage is implied by the read operation, but no exclusions or alternative tool mentions are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.