Server Quality Checklist

Tool Scores

• get_mappingsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states what the tool does but doesn't describe how it behaves: no information about permissions required, rate limits, error conditions, response format, or whether it's a read-only operation. The description is minimal and lacks behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that states the core purpose without unnecessary words. It's appropriately sized for a simple tool with one parameter and is front-loaded with essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations and no output schema, the description is insufficient. It doesn't explain what 'field mappings' means in practice, what format the response will be in, or any behavioral characteristics. Given the lack of structured data, the description should provide more context about what users can expect from this operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with the single parameter 'index' well-documented in the schema. The description adds no additional parameter semantics beyond what's already in the schema, so it meets the baseline for high schema coverage without adding value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get') and resource ('field mappings for a specific Elasticsearch index'), making the purpose immediately understandable. It doesn't explicitly distinguish from sibling tools like 'list_indices' or 'search', but the specificity of 'field mappings' provides some implicit differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'list_indices' or 'search'. It mentions 'specific Elasticsearch index' which implies context but doesn't specify prerequisites, constraints, or typical use cases for retrieving mappings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_indicesB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. While 'List' implies a read operation, it doesn't specify whether this requires special permissions, returns paginated results, includes hidden/system indices, or provides any metadata beyond names. For a tool with zero annotation coverage, this leaves significant behavioral questions unanswered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that communicates the core functionality without any wasted words. It's appropriately sized for a simple listing tool and front-loads the essential information. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter read operation without output schema, the description provides the minimum viable information about what the tool does. However, given the lack of annotations and sibling tools with potentially overlapping functionality, more context about when to use this versus alternatives would be helpful. The description is complete enough for basic understanding but leaves operational context gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters with 100% schema description coverage, so the schema already fully documents the parameter situation. The description appropriately doesn't mention parameters since none exist. A baseline of 4 is appropriate for zero-parameter tools where the schema handles all parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('all available Elasticsearch indices'), making the tool's purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'get_mappings' or 'search' - it's unclear if this is a simple listing versus more detailed metadata retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_mappings' or 'search'. There's no indication of whether this is for administrative purposes, discovery, or as a prerequisite for other operations. The agent must infer usage context from tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• searchB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds value by stating 'Highlights are always enabled', which informs about a default behavior not evident from the schema. However, it lacks details on other critical aspects like permissions, rate limits, error handling, or response format, leaving gaps in transparency for a search operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with two sentences that directly convey the tool's function and a key behavioral trait. It is front-loaded with the primary purpose and avoids any unnecessary words, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a search tool with two parameters (including a nested object) and no output schema or annotations, the description is somewhat incomplete. It covers the basic action and a behavioral detail but omits information on return values, error cases, or integration with sibling tools, which could hinder the agent's ability to use it effectively in varied contexts.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents both parameters ('index' and 'queryBody') adequately. The description adds minimal semantics by implying the use of Elasticsearch query DSL but doesn't provide additional syntax, examples, or constraints beyond what the schema specifies, meeting the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Perform an Elasticsearch search') and the resource ('with the provided query DSL'), making the purpose evident. However, it doesn't explicitly differentiate from sibling tools like 'get_mappings' or 'list_indices', which would require mentioning that this tool executes actual search queries rather than retrieving metadata or listings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives such as 'get_mappings' or 'list_indices'. It mentions that 'Highlights are always enabled', which hints at a behavioral trait but doesn't clarify usage contexts, prerequisites, or exclusions, leaving the agent without explicit direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.