Server Quality Checklist

Tool Scores

• add_memoryC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description must disclose behaviors. It mentions storing but lacks details on side effects, authentication, rate limits, or return values. Minimal transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, 14 words, front-loaded with purpose. Concise but could benefit from additional structured info.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Simple tool with 2 params and output schema, but description omits return value, idempotency, or overwrite behavior. Incomplete for reliable agent usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%. Description does not explain parameters beyond implying 'text' holds the memory. No detail on 'workspace_id' meaning or format.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description states verb 'store' and resource 'plain-text memory or note into the index', distinguishing it from sibling tools like index_workspace which likely require file paths. Clear and specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this vs alternatives. Only hints at avoiding file paths, but no direct comparison or scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• clear_workspaceC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states the tool purges data, which implies destructive behavior, but does not disclose reversibility, required permissions, or any safety mechanisms. The specifics of what is purged are listed, but deeper behavioral context is missing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that efficiently conveys the tool's action and scope. No unnecessary words or information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema (not shown), the description provides no context about return values or side effects. For a destructive tool with minimal additional info (no annotations), this leaves gaps in understanding what happens after the operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, meaning no parameter descriptions. The description does not elaborate on the 'workspace_id' parameter beyond its name, so it adds no semantic value beyond the schema. Given the low coverage, this is a significant gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Purges' and clearly lists the resources (indexed code chunks, stored memories, file hashes) that are affected. It distinguishes itself from sibling tools like index_workspace (which indexes) and search_code (which searches), making its purpose unique and clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no explicit guidance on when to use or not use this tool. No alternatives or prerequisites are mentioned. The description lacks context about when this operation is appropriate or what conditions should be met before calling it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• index_workspaceB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries burden. Discloses recursive file discovery, specific indexing pipeline (Tree-sitter, FastEmbed, Turbovec), and auto-skip behavior. Missing side-effect or permission info.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences are efficient and focused, no filler. Information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Has output schema, so return values not needed. However, no error cases, prerequisites, or success conditions described. Adequate but not thorough for 3-param tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description should compensate. It mentions force=True meaning and implies directory_path is a folder path, but workspace_id is unexplained beyond name. Minimal parameter guidance.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool indexes code files in a directory using specified technologies. It distinguishes from siblings like search_code by being the indexing step, but does not explicitly differentiate.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides guidance on skipping unchanged files and force parameter, but lacks explicit when-to-use vs alternatives like search_code or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_codeA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description reveals the internal process: embedding with FastEmbed, retrieval from Turbovec, reranking with Cross-Encoder, and returning top_k chunks. This goes beyond a simple 'search' label. However, it does not explicitly state that the tool is read-only or safe, and no annotations are provided to cover those aspects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that efficiently conveys the tool's workflow. No extraneous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the core search process but omits context about the workspace parameter (e.g., search is scoped to a workspace). Given the output schema exists, return values need not be explained, but the workspace context is important for correct usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must explain parameters. It only mentions top_k's default value. The workspace_id and query parameters are not described, leaving ambiguity about their roles and constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it searches code/memory chunks using embedding, retrieval, and reranking. The verb 'search' and resource 'code/memory chunks' are specific. The sibling tools are about indexing, adding, status, and clearing, so this tool is clearly distinguished as the search tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for searching code/memory chunks but does not explicitly state when to use it versus alternatives like index_workspace or get_status. No conditions or exclusions are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_statusA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses that the tool is a read-only operation returning real-time data, listing specific output categories. This is sufficient transparency for a status tool, though it doesn't mention authentication or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that covers the tool's purpose and output without any wasted words. It is appropriately front-loaded with the core action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity, the presence of an output schema, and the description listing key output categories, the description is fairly complete. However, it could briefly mention the optional workspace_id parameter for full context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has one optional parameter (workspace_id) with 0% description coverage in the schema. The tool description does not mention this parameter at all, failing to add meaning beyond the schema. A higher score would require explaining the parameter's role.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns real-time status and statistics about the Turbocontext vector store, specifying concrete data like database size, chunk counts, etc. This differentiates it from sibling tools (index_workspace, search_code, etc.) that perform different actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implicitly indicates when to use the tool (when needing status or stats), but provides no explicit guidance on when not to use it or alternatives. This is adequate for a simple status tool but lacks depth.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.