Server Quality Checklist

Tool Scores

• list_streamsA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It mentions return fields but omits safety traits (e.g., read-only nature, permission requirements). Listing operations are typically safe, but this is not stated explicitly.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: first for action, second for return fields. Front-loaded and succinct with no extraneous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list operation with one parameter and no output schema, the description covers purpose, scope, and return content. Could be enhanced with implicit behavioral context (e.g., read-only hint) but is otherwise complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and already describes the 'instances' parameter as 'Instance IDs to query'. The description adds minimal value by stating 'one or more', which is implied by the array type. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verbs ('list') and resource ('log streams'), and clarifies the scope ('on one or more OpenObserve instances'). It distinguishes from sibling tools like list_instances by specifying what is listed and the required instances parameter.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use when needing to enumerate streams for given instances but does not explicitly state when to use this tool versus siblings like search_logs or get_stream_schema. No when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• batch_queryA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses parallel execution and returning all results together, but lacks behavioral details like error handling for individual queries, rate limits, or read-only nature. It adds some context beyond the schema but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, direct, and front-loaded. Every sentence adds value: first defines the core function, second gives use cases. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (parallel queries, no output schema), the description adequately conveys purpose and input structure. It explains return behavior ('return all results together') but could specify result ordering or error aggregation. Still sufficient for selection.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with parameter descriptions in the schema itself. The description echoes the schema by stating each query has instance, SQL, and time range, but doesn't add new semantic meaning or format details beyond what's already in the structured input definition.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool executes multiple SQL log queries in parallel across instances and returns combined results. It uses specific verbs and resources ('execute multiple SQL log queries') and distinguishes itself from siblings like search_logs (single query) by emphasizing parallelism and batch capability.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit use cases: 'comparing data across instances or fetching related signals in a single round trip.' While it doesn't list exclusions or alternatives, the context makes it clear this is for multi-query scenarios, with siblings implying single-query alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_logsA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Covers automatic strategy switching, SQL limitations, and time range handling in detail. No annotations exist, so description carries full burden; does not mention auth or rate limits, but those are less critical for a search tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with main purpose upfront, followed by detailed strategy rules, SQL syntax, and examples. Slightly verbose due to SQL rule listing, but each section adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Comprehensive for an 8-parameter tool with no output schema. Covers use cases, query strategies, parameter behavior, and pagination. Missing return value specification, but output schema is absent and not required per rules.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Adds significant meaning beyond schema: explains strategy-dependent behavior for startTime/endTime, SQL syntax and capabilities, limit applicability, and performance implications of trackTotalHits. Schema coverage is 100%, but description enhances understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool searches log streams with SQL across instances, distinguishing from siblings like batch_query (batch processing) and get_logs_around (time-range retrieval) by emphasizing SQL querying and multi-instance support.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance on query strategy selection based on time range, SQL syntax rules, and specific restrictions (e.g., not adding WHERE _timestamp). Does not mention when to use alternatives like batch_query, but the context signals imply alternatives exist.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_logs_aroundA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. Discloses basic retrieval behavior (centered, up to size records) but omits details like authorization, rate limits, error handling, or exact splitting of before/after records. Adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with zero waste. First sentence covers action and result; second sentence provides use case and context. Front-loaded with essential info.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description adequately explains what the tool does and when to use it. Could hint at return format or potential errors, but for a read operation with clear parameters, it is reasonably complete. Sibling tools are mentioned indirectly via use case.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, baseline 3. Description adds meaning by explaining how parameters interact: 'centered on the anchor timestamp' and 'up to size records' clarify the selection logic beyond individual schema descriptions. Does not detail split ratio, but adds value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Describes fetching log records around a timestamp, with specific verb ('Fetch'), resource ('log records in a stream'), and scope ('immediately before and after', 'centered'). Clearly distinguishes from sibling tools like search_logs and batch_query.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states it is useful for viewing context around a known event without writing SQL, providing guidance on when to use. Does not explicitly list when not to use or alternatives beyond the SQL query hint, but context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_instancesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description carries full burden. It describes return fields and optional filtering. It implies a read-only operation without side effects, which is appropriate for listing instances.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is two sentences with no waste. Front-loaded with the core purpose: listing instances. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a list tool with no output schema, the description details what is returned. It does not mention pagination or result limits, which is a minor gap. Otherwise, it is complete for a simple listing without complex behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with parameter descriptions. The description adds that filtering by tags can be 'one or more' and filters by capability are specific values (logs, traces, metrics), which adds marginal context beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists configured OpenObserve instances and returns specific fields (ID, name, URL, capabilities, tags). This is distinct from sibling tools like search_logs or list_streams, which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description specifies it lists instances available to the server, implying use for discovery. It does not explicitly exclude when not to use or name alternatives, but in context of siblings, its purpose is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_stream_schemaA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description bears full burden. It discloses two important behaviors: parallel fetching and 10-minute caching. It does not cover error handling or auth, but for a read operation this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, 26 words, no filler. The purpose is front-loaded, and every clause adds essential information about input types, concurrency, and caching.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 required params, no output schema, no nested objects), the description covers all needed aspects: what it returns (field names, data types), input flexibility, and caching behavior, making it fully informative.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%; the description adds 'single stream name or an array' which aligns with the schema's anyOf but does not provide extra meaning beyond what the schema already conveys.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Return', the resource 'field names and data types for one or more log streams', and specifies acceptance of single or array input and parallel fetching, distinguishing it from siblings like list_streams which only list streams.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains usage details like accepting single or array streams and parallel fetching, but does not explicitly state when not to use this tool or suggest alternatives, though the differentiation from siblings is clear based on purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.