Server Quality Checklist

Tool Scores

• opensearch_countA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of disclosure, but it only says 'Quick' and 'Lighter than full query'. It doesn't explain the return value, potential errors, performance ceilings, or how it handles missing indexes. The behavioral trait hint is minimal but present, so a 2 is appropriate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that conveys the core action, optional filter, and comparative value. It is highly concise with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  There is no output schema and no annotations, so the description should explain the return value and any caveats. It does not mention that the result is a count or what happens when no index is specified. The completeness is lacking, especially given that the schema has zero required parameters, creating ambiguity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description repeats the 'optional query filter' concept already in the schema, adding no new semantics. Thus it doesn't elevate beyond baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs a 'document count' with an optional query filter, and it distinguishes itself from the sibling 'opensearch_query' by being 'lighter than full query'. This gives a specific verb-resource pair and scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description says 'Lighter than full query', which clearly implies using this when only a count is needed and not the actual documents. It provides clear context for when to choose this over a full query, though it doesn't explicitly name the alternative tool or mention exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• opensearch_mappingA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that the tool returns a 'compact field list,' which is useful behavioral context. However, it does not mention read-only nature, potential errors, or other operational details. For a simple read operation, this is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences, front-loaded with the primary purpose. It is concise with no filler or redundancy, earning a perfect score for efficiency.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with no output schema, the description provides the essential context: what it does and what it returns. It is complete enough for its simplicity, though it could have briefly differentiated from siblings to improve context further.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 100% coverage for the single parameter 'index' with description 'Index name.' The tool description adds no additional parameter semantics beyond what the schema already provides, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Get index mapping to understand available fields.' It uses a specific verb and resource, and the purpose is distinct from sibling tools like opensearch_count and opensearch_query, which focus on counts and queries rather than schema discovery.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when one needs to understand available fields in an index, but it does not explicitly state when to use this tool over alternatives or provide exclusions. There is no mention of context such as 'use this before querying' or 'for field discovery, not data retrieval,' so guidance is minimal.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• opensearch_queryA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosure. It does add value by explaining the proxy behavior ('returns COMPACT formatted output instead of raw JSON'), detailing output formats, and emphasizing full query flexibility. Yet it lacks information on potential side effects, permissions, error handling, or performance implications, which prevents a higher score.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: it opens with a clear purpose, then uses bulleted lists for capabilities and output formats, making it easy to scan. Length is justified by the tool's flexibility, though some redundancy exists between 'ANY' and the enumerated query types.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity and lack of output schema, the description covers essential aspects: what it does, how to build queries, and what output formats are available. It does not provide explicit examples of synthesized output or discuss handling of large result sets, but the guidance is sufficient for a competent agent to infer expected behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Since schema coverage is 100%, baseline is 3. The description enhances the output_format parameter by explaining each mode (auto, table, list, summary, compact_json) and enriches the query parameter by providing concrete examples of DSL constructs (bool, has_parent, aggregations, etc.), adding real meaning beyond the schema's brief descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Execute ANY OpenSearch DSL query and return synthesized results,' with an explicit list of supported query types and aggregations. This specific verb+resource+scope differentiates it from sibling tools like opensearch_count and opensearch_mapping, which are narrower in function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context through phrases like 'FULL FLEXIBILITY' and lists a wide range of query capabilities, effectively positioning this as the go-to tool for complex OpenSearch queries. However, it does not explicitly contrast with siblings or state when not to use it, so it falls short of full alternative guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.