Server Quality Checklist

Tool Scores

• kibana_search_logsA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint and idempotentHint. The description adds specific behavioral details: the bool/must query structure, truncation hint for >20 hits, return format (total/returned/took_ms/hits), and the ES size cap. This is informative but not exhaustive (e.g., no explicit pagination guidance beyond size cap).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with main purpose, technical detail, truncation note, examples, and exclusions. Though a bit lengthy, every section serves a purpose and the information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool complexity (7 parameters, ES integration, truncation, size limits) and the richness of schema/annotations/output schema (return dict described), the description covers all critical aspects: purpose, usage guidelines with examples, behavioral traits, limits, and return structure. No major gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All 7 parameters have schema descriptions (100% coverage), so baseline is 3. The description adds value by providing real-world examples that illustrate parameter usage (e.g., query syntax, time range formats, sort_order), enhancing understanding beyond the schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches logs using ES Query String Syntax, identifies the specific operation (wrapping a POST _search with bool/must), and distinguishes from sibling tools like kibana_aggregate_logs via explicit 'Don't use when' examples.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Multiple 'Use when' examples with concrete parameter values, plus explicit 'Don't use when' scenarios directing to kibana_aggregate_logs for aggregations or scroll API for >500 docs, providing clear alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kibana_get_dashboardA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, destructiveHint=false, and idempotentHint=true. The description adds behavioral context by stating it makes a GET request, returns dashboard metadata and panel summary, and does not return log data. This goes beyond annotations by clarifying the output nature and API call, but could mention potential error cases (e.g., ID not found) for slightly higher score.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a one-line summary, the API call, return structure, and usage examples. Every sentence adds necessary information without redundancy. It is front-loaded with the key action and includes bullet-point examples for clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, output schema present), the description covers all necessary aspects: what it does, when to use it, what it returns (including specific fields like panels_count), and what it does not do (log data). It is complete for an agent to invoke correctly without additional context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides a full description for the single parameter 'dashboard_id', including format and a hint to use 'kibana_list_dashboards' for discovery. The tool description reinforces this hint with examples, adding value by showing how to obtain the ID. Since schema coverage is 100% and the description adds contextual usage, a 4 is appropriate, slightly above baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Fetch a single Kibana dashboard with panel details.' and distinguishes itself from siblings like 'kibana_list_dashboards' (which lists dashboards) and 'kibana_search_logs' (which searches logs). It specifies the verb 'fetch' and the resource 'Kibana dashboard', making the purpose explicit and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit 'Use when' examples (e.g., to get panels of a specific dashboard) and 'Don't use when' examples (e.g., when missing the ID or needing log data), directing users to alternative tools like 'kibana_list_dashboards' and 'kibana_search_logs'. This clearly differentiates when to use this tool versus its siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kibana_list_dashboardsA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds valuable context: the kbn-xsrf header requirement, pagination details (has_more, page+1), and return format. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is well-structured with clear sections (endpoint, header, pagination, examples, return format). Every sentence adds value and it is appropriately sized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity, annotations, and output schema presence, the description covers all necessary aspects: pagination, search filtering, when to use, and return format. Complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has 100% description coverage, so the description adds minimal value beyond schema. It provides example usage for search and page parameters, meeting the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description explicitly states 'List Kibana saved dashboards' and provides the HTTP endpoint. Examples distinguish this from sibling tools like kibana_get_dashboard and kibana_search_logs, making the purpose extremely clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes explicit 'Use when' and 'Don't use when' examples, referencing sibling tools by name and explaining exactly when to choose this tool versus alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kibana_list_indicesA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds significant value beyond annotations by detailing the HTTP method (GET), endpoint, and the exact structure of the return value. Annotations already indicate read-only and idempotent, but the description enriches with concrete behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with bullet points and examples, but it is somewhat lengthy and repeats some return format details. It is front-loaded with the main action and every sentence adds value, though could be slightly more concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema, the description appropriately summarizes the return keys. It covers the tool's purpose, parameters, usage context, and behavioral details, making it fully complete for an agent to select and invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has 100% coverage with detailed parameter descriptions. The description adds value through examples and usage context, such as default pattern and how to narrow down, going beyond the schema. Baseline 3 increased to 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists available Elasticsearch indices, specifies the exact API call, and distinguishes it from sibling tools like kibana_search_logs and kibana_aggregate_logs. It uses specific verbs and resources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use and when-not-to-use scenarios, including examples of when to use with specific patterns and when to skip and directly use kibana_search_logs. This provides clear guidance on tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kibana_aggregate_logsA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds that the tool wraps a POST request with size:0 (no hits), truncates output beyond 20 buckets with a hint, and returns a structured dict. It also notes efficiency for grouped stats, going beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-organized into an opening summary, technical detail, examples with when-to-use sections, and return format. It uses bullet points and concise language without redundancy. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's 9 parameters, full schema coverage, explicit annotations, and output schema description ('total_documents, took_ms, buckets'), the description is complete. It covers all necessary context for an agent to invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has 100% coverage with descriptions for all 9 parameters. The description further clarifies parameter roles through concrete examples (e.g., using time_from, group_by, metric_field), adding value beyond the schema. Baseline 3, plus 1 for enhanced semantics via examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific action: 'Aggregate logs using a terms grouping and optional metric.' It clearly identifies the resource (logs) and method (terms aggregation). The description distinguishes this tool from siblings by mentioning kibana_search_logs for raw logs and noting that date_histogram is not supported, making the purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance is provided with multiple 'Use when' and 'Don't use when' examples. It names the alternative tool (kibana_search_logs) for raw logs and explains what aggregation type is not supported (date_histogram). This helps the agent decide correctly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.