Server Quality Checklist

Tool Scores

• searchC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'similar documents' but doesn't explain how similarity is determined (e.g., via embeddings), what the output looks like (e.g., relevance scores), or any constraints like rate limits or authentication needs. This leaves significant gaps for a search tool with multiple parameters.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's appropriately sized and front-loaded, making it easy to grasp quickly, which is ideal for conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a search tool with 4 parameters, no annotations, and no output schema, the description is incomplete. It fails to explain key aspects like the search mechanism (embedding-based), expected output format, or error conditions. This leaves the agent with insufficient context to use the tool effectively beyond basic parameter input.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents all parameters well. The description adds no additional meaning beyond what's in the schema, such as explaining the relationship between query and embeddingService or how limit affects results. This meets the baseline for high schema coverage but doesn't enhance understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool's purpose as 'Search for similar documents in a collection', which clearly indicates a search operation. However, it's vague about what 'similar' means (e.g., semantic similarity via embeddings) and doesn't distinguish from siblings like list_collections, which might also involve collections but for listing rather than searching. It's adequate but lacks specificity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. For example, it doesn't clarify if this is for semantic search (implied by embeddingService) versus other search methods, or how it relates to siblings like add_documents or list_collections. There's no mention of prerequisites, such as needing an existing collection, leaving usage context unclear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_documentsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the embedding service but doesn't cover critical aspects like whether this is a write operation (implied by 'Add'), potential side effects, error conditions, or performance characteristics. For a tool that likely modifies data, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's appropriately sized and front-loaded with the core action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of adding documents with embeddings (a write operation with processing steps), no annotations, and no output schema, the description is incomplete. It lacks details on behavior, output format, error handling, or integration context, leaving significant gaps for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all parameters. The description adds no additional meaning beyond implying that documents are processed with embeddings, which is already suggested by the 'embeddingService' parameter. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Add documents') and target resource ('to a Qdrant collection'), specifying the embedding service requirement. However, it doesn't differentiate from sibling tools like 'search' or 'delete_collection' beyond the basic verb.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'search' or 'list_collections', nor are prerequisites or typical use cases mentioned. The description only states what it does, not when it's appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_collectionC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool deletes a collection, implying a destructive mutation, but does not elaborate on critical aspects such as irreversibility, permissions required, error handling, or side effects (e.g., data loss). This leaves significant gaps for a tool with clear destructive intent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence with zero wasted words, making it highly concise and front-loaded. It immediately communicates the core action without unnecessary elaboration, which is efficient for an AI agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's destructive nature, lack of annotations, and no output schema, the description is insufficiently complete. It fails to address key contextual elements like irreversibility, error cases, or what happens post-deletion, which are critical for safe and informed usage by an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with the single parameter 'collection' clearly documented in the schema. The description does not add any additional meaning or context beyond what the schema provides (e.g., format examples or constraints), so it meets the baseline for adequate but not enhanced parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Delete') and resource ('a Qdrant collection'), making the purpose immediately understandable. However, it does not differentiate this tool from its siblings (like 'list_collections' or 'add_documents'), which would require mentioning it's a destructive operation versus read-only alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention prerequisites (e.g., that the collection must exist), exclusions, or refer to sibling tools like 'list_collections' for checking before deletion, leaving the agent with minimal context for decision-making.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_collectionsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. While 'List all available Qdrant collections' implies a read-only operation, it doesn't specify important behavioral details like whether this requires authentication, what format the results come in, if there are rate limits, or how large collections are handled. The description is minimal and lacks operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that states exactly what the tool does with no wasted words. It's front-loaded with the core functionality and appropriately sized for a simple listing operation. Every word earns its place in conveying the essential purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations, no output schema, and zero parameters, the description is insufficiently complete. While it states what the tool does, it doesn't provide enough context about the operation's behavior, result format, or integration considerations. The agent would need to guess about important operational details that aren't covered.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and schema description coverage is 100% (empty schema is fully described). The description appropriately doesn't discuss parameters since none exist. This meets the baseline expectation for a parameterless tool, though it doesn't add any parameter-related value beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('Qdrant collections'), making the purpose immediately understandable. It specifies 'all available' collections, which adds useful scope information. However, it doesn't explicitly differentiate from sibling tools like 'search' or 'delete_collection', which would require more specific comparison.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'search' or 'delete_collection'. There's no mention of prerequisites, typical use cases, or scenarios where this tool would be preferred over others. The agent must infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.