Server Quality Checklist

Tool Scores

• get_shardsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states what the tool does but doesn't describe important behavioral aspects like whether this is a read-only operation, what format the shard information returns, whether it includes cluster health status, or any performance considerations for large indices.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that communicates the core functionality without unnecessary words. It's appropriately sized for a simple tool with one optional parameter.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations and no output schema, the description is insufficient. It doesn't explain what shard information includes (primary/replica status, node assignments, size, etc.), doesn't mention error conditions, and provides no context about how this tool fits into broader index management workflows with the sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the optional 'index' parameter. The description adds minimal value by mentioning 'specific indices' which aligns with the parameter, but doesn't provide additional context about index naming conventions, wildcard support, or what happens when no index is specified.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get') and resource ('shard information'), and specifies scope options ('for all or specific indices'). It doesn't explicitly differentiate from sibling tools like get_mappings or list_indices, but the focus on shard information provides reasonable distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like list_indices or get_mappings. It mentions scope options but doesn't explain when to filter by index versus getting all indices, or how this tool relates to other index-related operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_indicesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action ('List all available Elasticsearch indices') but doesn't describe what 'available' means, whether it's a read-only operation, if there are rate limits, or what the output format looks like. This leaves significant gaps for a tool that interacts with a database system.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's appropriately sized for a simple tool and front-loaded with the core action, making it easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of interacting with Elasticsearch indices and the lack of annotations and output schema, the description is incomplete. It doesn't explain what 'available' entails, how results are structured, or potential errors, which are crucial for an agent to use the tool effectively in a real-world context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with the parameter 'indexPattern' fully documented in the schema. The description doesn't add any meaning beyond what the schema provides, such as explaining pattern syntax or examples. With high schema coverage, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('all available Elasticsearch indices'), making the purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'get_mappings' or 'get_shards' which might also involve listing indices with different scopes or details.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_mappings' or 'search'. It lacks context about use cases, prerequisites, or exclusions, leaving the agent to infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_mappingsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool retrieves field mappings but doesn't describe traits like whether it's read-only (implied by 'Get'), error handling for non-existent indices, rate limits, or response format. This leaves significant gaps for a tool with no structured safety hints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with zero waste. It's front-loaded with the core purpose and appropriately sized for a simple tool, making it highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (one parameter, no output schema, no annotations), the description is minimally adequate. It covers the basic purpose but lacks details on behavior, usage context, or output, which would be needed for higher completeness in the absence of annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with the single parameter 'index' fully documented in the schema. The description adds no additional meaning beyond what the schema provides (e.g., no extra context about index naming conventions or examples), so it meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get') and resource ('field mappings for a specific Elasticsearch index'), making the purpose understandable. However, it doesn't explicitly differentiate from sibling tools like 'list_indices' or 'search', which might also involve index operations, so it doesn't reach the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'list_indices' or 'search'. It lacks context about prerequisites (e.g., index existence) or exclusions, leaving the agent with minimal usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• searchB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds one important behavioral trait ('highlights are always enabled') that wouldn't be evident from the schema alone. However, it lacks other critical behavioral information such as pagination behavior, error handling, authentication requirements, rate limits, or what the response format looks like.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with just two sentences that both earn their place. The first sentence states the core purpose, and the second adds important behavioral context about highlights. There's no wasted language or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 4 parameters, complex nested objects in queryBody, no output schema, and no annotations, the description is insufficiently complete. It doesn't explain what the tool returns, how results are structured, error conditions, or provide any examples of proper usage. The mention of highlights is helpful but doesn't compensate for the significant gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents all 4 parameters thoroughly. The description adds no additional parameter semantics beyond what's in the schema. It doesn't explain parameter interactions, provide examples of queryBody structure, or clarify the relationship between parameters like profile and explain.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('perform') and resource ('Elasticsearch search') with the specific technology mentioned. It distinguishes from siblings like get_mappings or list_indices by focusing on search operations rather than metadata retrieval. However, it doesn't explicitly differentiate from potential search-related siblings that might exist in other contexts.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. While it mentions 'highlights are always enabled,' this is a behavioral trait rather than usage guidance. There's no mention of prerequisites, when to choose this over other search methods, or what scenarios it's best suited for.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.