Server Quality Checklist

Tool Scores

• get_index_patternsA
  3.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool returns {id, title, time_field} and gives an example of the title pattern, which adds behavioral context beyond the input schema. However, it does not explicitly state read-only behavior or any other side effects, but 'List' implies a safe read operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is only two sentences, front-loaded with the purpose, and includes a concise code snippet for the return format. Every sentence is informative with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool (list with one optional parameter and an output schema exists), the description covers the core purpose and return format adequately. It could mention that pagination is handled by the optional per_page parameter, but that is present in the schema. Overall, it is sufficiently complete for an agent to understand the tool's function.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, meaning the input schema has no descriptions for its single parameter (per_page). The tool description does not mention or explain the parameter at all, failing to add meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List' and resource 'index patterns configured in OpenSearch Dashboards.' It distinguishes itself from sibling tools like discover_search (which performs searches) and get_tenants (which retrieves tenants).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives, nor does it mention prerequisites or exclusions. It only states what it does without contextualizing its use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_saved_objectsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description must cover behavioral aspects. It mentions the compact format to avoid flooding, but doesn't disclose read-only nature, side effects, or authentication needs, leaving gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise with clear structure: purpose, API call, output format, then tenant options. Slightly verbose with the tenant explanation, but overall efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers main behavior and tenant selection but omits explanation of the per_page parameter. Output schema exists, so return details may be covered there, but the description alone is incomplete for all parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description explains type_ and tenant in detail. Per_page is not mentioned, so 2 of 3 parameters are covered, adding value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List' and resource 'saved objects', with examples of types. It distinguishes from siblings by focusing on listing by type, unlike discover_search or get_index_patterns.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides guidance on selecting tenant and suggests using get_tenants to discover tenant names. However, it does not explicitly contrast with siblings or mention when not to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_tenantsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations, but description adds details: uses specific API endpoint, flattens response to structured list with fields {name, description, reserved, hidden}. Clearly indicates read operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: first states purpose, second adds implementation detail and output shape. No wasted words, front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Zero parameters, so minimal complexity. Description fully covers what the tool does and its output structure. Output schema exists but description already provides key fields.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters, so no need for param descriptions. Description adds value by explaining output format and API source.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Explicitly states it lists multi-tenancy tenants in OpenSearch Dashboards. Differentiates from siblings like discover_search and get_index_patterns by being tenant-specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use or alternatives, but sibling tools are distinct enough that context is clear. Could be improved by noting it's for tenant management vs search/index operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• discover_searchA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully carries the burden of behavioral disclosure. It explains the internal API endpoint, parameter semantics, and return format, including that 'total may be an estimate for large indices'. It does not cover rate limits or error handling, but for a search tool, the level of detail is commendable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured: a one-line purpose, then bullet-pointed parameter details, and a clear return format specification. At ~200 words, every sentence adds value with no fluff. It is front-loaded with the primary use case.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 8 parameters (1 required) and an output schema exists, the description is still comprehensive. It explains each parameter's purpose and format, the return structure, and a usage caveat (total may be estimate). No gaps remain for effective use by an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds rich meaning for all 8 parameters: index as glob pattern, query as Lucene query_string with None meaning match all, time_from/to with examples, time_field default, size with recommendation, fields for restriction, and tenant as optional header. This goes well beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with 'Fetch documents like the Discover app', clearly stating the verb and resource. It distinguishes itself from siblings (no other search-like tool) by specifying it uses the internal search API that Dashboards' Discover uses, making its purpose unique and clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage context: 'works even when the raw OpenSearch REST API is not exposed'. It advises to keep size small and use fields to avoid flooding the context. While it does not list when not to use or explicitly name alternatives, the provided guidance is sufficient for typical use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pingA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Since no annotations are provided, the description fully carries the burden. It discloses the endpoint hit, the authentication flow (session login on 401), and the returned fields or error reason. This provides comprehensive behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (two sentences) and front-loaded with the main purpose. It could be slightly more structured, but it is efficient and without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters and an output schema exists, the description covers all necessary context: connectivity check, authentication handling, and return value structure (connected, authenticated, user, tenants, or error).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With no parameters and 100% schema coverage, the description adds no parameter details, which is appropriate. It adds context on the tool's action and output, meeting the baseline expectation for zero-parameter tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Check' and the resource 'OpenSearch Dashboards connectivity and authenticate'. It distinguishes the tool from siblings like discover_search and get_tenants, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly states when to use (to check connectivity and authenticate) but does not provide when-not-to or alternative tools. However, sibling tools have distinct functions, so the context is sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.