Server Quality Checklist

Tool Scores

• check_jobA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It merely says 'check the status' but does not clarify whether the call blocks, what statuses are possible, whether it has side effects, or what happens if the job_id is invalid. This is a significant transparency gap for a polling tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, using two sentences to convey the purpose and the parameter origin. It is well-structured with a clear 'Args' section, front-loaded with the main verb. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool, this is minimally viable but has gaps. It tells what it does and where the parameter comes from, but it does not describe the possible return statuses or how an agent should interpret the result. The presence of an output schema mitigates the need to explain return structure, but the absence of lifecycle context makes it harder to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no description for 'job_id' (0% coverage), so the description's explanation is essential. It adds meaning by specifying that the job_id comes from hire_agent, which directly helps an agent understand how to obtain it. This is adequate for a single parameter, though more detail on format or examples would elevate it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Check the status of a background agent job.' It distinguishes itself from siblings by referencing hire_agent, which is the tool that returns the job_id. This is specific and actionable, though it could more explicitly contrast with list_agents or other tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool by noting the job_id is 'returned by hire_agent for long-running agents.' This gives clear context for its use. However, it does not state exclusions or explicitly recommend it over alternatives, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• hire_agentA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It only says 'Hire... to perform a task' and notes that the agent only sees the task text, but it does not explain side effects, asynchronous behavior, or what happens after hiring. This is a significant gap for a mutation-like tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-structured: a one-sentence purpose followed by a brief parameter breakdown. Every sentence provides useful information without redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a two-parameter tool, the description covers the inputs well and includes the full agent list. However, it leaves out behavioral context such as how to check on the hired task (sibling check_job exists) or whether hiring is asynchronous. Since an output schema exists, return values need not be described, but the operational lifecycle is incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description explicitly lists all allowed agent_id values, which is not present in the schema (no enum). It also gives actionable guidance for the task parameter, emphasizing detail and the fact that the agent only sees this text. This fully compensates for the schema's 0% property description coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Hire' and identifies the resource as 'Vending Machine agent', making the action clear. It distinguishes from siblings like list_agents and check_job, but it does not explicitly contrast with general tools like 'run', leaving some ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states that the tool hires an agent to perform a task, providing clear context for when it is appropriate. However, it gives no guidance on when to prefer this over alternatives, nor does it mention companion tools like check_job for tracking the task. Thus, there is clear context but no exclusions or alternative routes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• infoA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations are absent, so the description carries the full burden. It discloses the type of data returned (description, validation report, usage stats, etc.), which implies a read-only operation. However, it does not disclose additional behavioral traits such as required permissions, potential errors, or side effects. For a simple info tool, the disclosure is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded. The first sentence states the primary purpose, and the Args section is minimal yet informative. There is no redundant fluff or repetition of schema details. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, output schema available, no annotations), the description is mostly complete. It covers the tool's function and the parameter's meaning. However, it lacks explicit usage guidance or mention of alternative tools, which would make it fully complete. The output schema exists, so the description does not need to explain return values, but the lack of usage context slightly reduces completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must compensate. It does so by explaining that vending_code is "the script's vending code" and provides an example format ("S-7K2M"). This adds meaningful context beyond the raw schema definition (just a string type) and helps the agent construct a correct call.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: "Get full details about a script" and lists the specific types of details (description, validation report, usage stats, similar scripts, creator info). This is a specific verb+resource combination. However, it does not explicitly distinguish itself from sibling tools like search_scripts or list_agents, though the scope is evident.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool (when full script details are needed) but provides no explicit guidance on when to prefer an alternative. It does not state exclusions or mention alternatives. The usage context is somewhat clear from the description itself but lacks explicit direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_agentsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. The verb 'List' implies a read-only operation, but the description does not explicitly state side effects, authentication needs, or rate limits. It is adequate for a simple list tool but lacks added behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence that states the essential information without any waste. It is perfectly concise and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with an output schema, the description is nearly complete. It could be improved by explicitly noting that this is a safe, read-only operation or mentioning that output schema details capabilities, but it is largely sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the schema is fully covered. The baseline for 0 params is 4; the description adds no parameter details, but none are needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action ('List') and its target ('available Vending Machine agents') along with what is returned ('their capabilities'). It is specific enough to distinguish from siblings like hire_agent or info, which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool instead of alternatives. There is no mention of prerequisites, typical usage scenarios, or relationships to sibling tools like hire_agent or info.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_scriptsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It does disclose the result limit ('top 5 results') and result contents ('vending codes and descriptions'), but does not mention whether the operation is read-only, authentication requirements, or any rate limits. This is minimally adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded with the primary purpose, followed by a useful return summary and a structured Args section. Every sentence earns its place without redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple search tool with one parameter, the description is mostly complete: it states what to search, how to phrase the query, and what results will contain. An output schema exists, so detailed return types are covered elsewhere. It could add a note about error conditions or sorting, but overall it provides sufficient context for an agent to use the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema only provides the parameter name and type, so the description adds essential meaning by explaining 'query' as a natural language description and providing a concrete example ('analyze CSV data'). This compensates for the 0% schema description coverage and gives the agent clear guidance for constructing the parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Search') with a clear resource ('marketplace for scripts') and a defining scope ('matching a natural language query'). It also states the return value (top 5 results with vending codes and descriptions), making it easy to distinguish from sibling tools like list_agents or vector_store_search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool should be used when you need to find scripts based on a natural language description, which is clear context. However, it does not explicitly mention alternatives or when not to use this tool, leaving the comparison with sibling tools implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• vector_store_listA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden of behavioral disclosure. It does disclose the conditional behavior—empty collection lists names/counts; non-empty lists documents—and the limit parameter's effect. However, it omits any mention of read-only guarantees, permissions, or return format, leaving some ambiguity about what 'list documents' returns.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded with the main purpose. The two-sentence overview plus a short Args list contains no filler and each line contributes meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With an output schema present, return values need not be explained. The description covers both operational modes and the limit parameter, which is complete for a straightforward listing tool. Minor gaps exist around pagination or large result sets, but the core information is present.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides only types and defaults, so the description compensates fully. The Args section explains that 'collection' switches between two listing modes and 'limit' caps rows, adding crucial semantics beyond the schema's minimal metadata.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists collections or documents, using the active verb 'List' and specific resources 'collections' and 'documents'. It distinguishes itself from sibling tools like vector_store_add and vector_store_search by focusing solely on listing behavior.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains the two listing modes based on the 'collection' parameter, providing clear context for when to use each mode. However, it does not explicitly compare this tool to alternatives like vector_store_search or provide exclusion criteria, so guidance is implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• audit_codeA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses the pipeline stages and that it returns a combined report, providing some transparency. However, it is ambiguous whether 'writes fixes' means applying changes to the source code or just reporting suggested fixes, and it does not mention side effects, data handling, or execution mode (synchronous/asynchronous). With no annotations, this gap is significant.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, with two sentences covering the main functionality and a single-line Args section for the parameter. It is front-loaded with the core purpose and contains no redundant fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the primary steps and return value, but it lacks clarity on whether the audit runs synchronously or asynchronously, and whether fixes are applied directly or only reported. The output schema may explain the report structure, but the process details remain underspecified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although the schema has no description for the 'code' parameter, the tool description explicitly explains it: 'code: The source code to audit (paste the full code).' This fully compensates for the 0% schema coverage for a single parameter, giving clear guidance on what to provide.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb ('audit') and resource ('code') and details the pipeline stages (Bug Hunter, Test Goblin, Code Gremlin), making the purpose unambiguous. It distinguishes itself from siblings like run_code by emphasizing a security audit pipeline with reported outputs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for security audits and lists the stages, giving clear context for when to use it. However, it does not explicitly mention alternatives or when not to use it, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• runA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses an asynchronous behavior via job_id and polling reference, but with no annotations, it doesn't elaborate on side effects, permissions, or execution environment. This leaves gaps for a tool that runs scripts.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded with its primary purpose, includes a useful example, and separates Args clearly. Every sentence serves a purpose with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  It covers the main flow: running a script, getting a job_id, and polling via check_job. It references search_scripts for discovery. However, it could mention error handling or what happens if the script fails, though the output schema reduces the need for return detail.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema coverage, the description fully compensates by explaining both parameters: vending_code's source and example, and user_input's purpose with a pointer to the script's input_schema. This adds significant meaning beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs a marketplace script by vending code, with a specific verb and resource. It also differentiates from siblings like run_code by specifying 'marketplace script' and referencing search_scripts for finding the vending code.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear context: use after obtaining a vending code from search_scripts and poll the result via check_job. However, it does not explicitly exclude alternatives like run_code or state when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_codeA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the secure sandbox environment, the returned stdout/stderr/errors, and the ability to install packages via the packages parameter. It provides meaningful behavioral context, though it omits potential limits like network restrictions or timeouts.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-structured, with an opening summary followed by an 'Args:' block. Every sentence adds value without waste, making it easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the core functionality, all parameters, and return values, which is sufficient given the output schema. However, it lacks safety details like timeout behavior or resource limits, which would be useful for a code execution tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description fully compensates by explaining each parameter. It specifies the 'code' source, the language with valid values and default, and the packages format with an example.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Run') and resource ('code in a secure E2B sandbox'), clearly defining what the tool does. The mention of returning stdout, stderr, and errors adds distinctness, though it doesn't explicitly differentiate from sibling 'run'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage context is implied through the description of running code in a sandbox, but no explicit when-to-use or alternatives are mentioned. Sibling tools like 'run' and 'audit_code' are not referenced, so the agent gets no guidance on choosing this over alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_knowledgeA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It explains the semantic search and LLM pipeline, and details provider/model/dimension overrides with auto-detection behavior. It doesn't explicitly state that the operation is read-only or address failure cases, but it implies no side effects and provides substantial implementation detail.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: a summary sentence, a compatibility note, and a parameter breakdown. It is appropriately sized given the six parameters, with no redundant text. The parameter list is necessary because the schema lacks descriptions, so it earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity and the presence of an output schema, the description adequately covers purpose, parameter semantics, and behavioral details. It might benefit from explicit read-only clarification or alternative tool guidance, but it is essentially complete for a QA tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no descriptions for any parameters, but the tool description thoroughly explains each one: collection sources, question semantics, limit range with default, and provider/model/dimension overrides with auto-detection. This fully compensates for the 0% schema coverage, adding meaning beyond the schema's bare titles.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: ask a question, retrieve relevant chunks via semantic search, and generate an answer using an LLM. This specific verb+resource combination distinguishes it from sibling tools like vector_store_search, which likely only retrieves chunks without generating an answer.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides context on when to use the tool by stating that it works with any collection, including those created by embeddings-agent or vector_store_add. However, it does not explicitly mention when not to use it or point to alternatives like vector_store_search, so there is clear context but no direct exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• vector_store_searchA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden. It explains the auto-matching of provider/model and the ability to override, which are valuable non-obvious behaviors. It stops short of covering auth, rate limits, or error cases, but the core behavior is transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with a clear purpose, includes a brief default-behavior note, and then provides a compact Arg list. Every sentence contributes value, with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that an output schema exists, the description sufficiently covers the tool's behavior and all parameters. It provides enough context for an agent to select and invoke the tool correctly, even without explicit failure-mode details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The Args section adds detailed meaning to all six parameters, including the relationship to vector_store_add, the natural-language nature of the query, the 1–50 limit bound, valid provider values, an example model, and the string form of dimensions. This fully compensates for the schema's lack of parameter descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb ('Semantic search') and identifies the resource ('collection'), and it ties the behavior to vector_store_add's embedding route. This makes the tool's purpose clear and distinguishes it from generic search tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for collections created with vector_store_add, which gives some context for when to use it. However, it does not explicitly mention alternatives like query_knowledge or provide exclusions, so usage guidance is largely implied rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• vector_store_addA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully carries the burden and excels. It discloses the provider selection order (Voyage → OpenAI → Gemini → local hash), override mechanisms, env var config, optional doc_id UUID generation, and the metadata_json format. This goes well beyond the bare schema and gives the agent critical operational knowledge.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and appropriately sized. The opening sentence states the purpose, the provider order is a necessary complexity, and the Args list is compact and scannable. Every sentence contributes value without bloat.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 7 parameters, no annotations, and no schema descriptions, the description is remarkably complete. It covers the embedding pipeline, all parameters, provider override logic, and configuration via env vars. With an output schema present, the lack of return value detail is acceptable. The only minor omission is behavior for unknown collections, but this does not undermine overall completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description thoroughly explains every parameter in the Args section: collection's logical name, text's full text to embed, doc_id's optional stable id, metadata_json's form, and provider/model/dimensions as overrides. This adds substantial meaning beyond the schema's bare types and defaults.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Embed and store text in a named collection for later similarity search.' The verb 'embed and store' plus resource 'named collection' is specific and distinguishes it from sibling tools like vector_store_search and vector_store_list.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives strong context for when to use this tool (when adding text to a vector store for later retrieval) and implicitly contrasts with search/list siblings. However, it does not explicitly state alternatives or when-not-to-use, such as 'use vector_store_search to query.'

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.