Server Quality Checklist

Tool Scores

• get_messageA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the returned document is 'full, untruncated' unlike search results, but doesn't cover error behavior, auth requirements, or exact return structure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two focused sentences, front-loaded with action and resource. No redundant wording or unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple fetch operation with fully documented schema and clear usage guidance, the description is sufficient. It explains what the tool returns ('full document') and when to use it, making it complete without an output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers all 3 parameters with descriptions. The description adds the crucial context that '_id' and '_index' are 'both returned by search', linking parameters to their source, which goes beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Fetch the full, untruncated document for a single message' with specific identifiers (_id and _index), distinguishing it from the sibling 'search' which returns concise results.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states 'Use after a concise search to inspect one hit in full', providing a clear when-to-use. It does not mention exclusions or alternatives, but none are relevant among the sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• searchA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full behavioral burden. It discloses key traits: 'merged newest-first', concise projection vs verbose mode, the expensive nature of raw lines with cost implications, and important stream semantics (Default Stream often excludes services). While it does not mention pagination or rate limits, the cost warning and stream caveat add significant value beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but efficient. Every sentence adds value: purpose, projection behavior, cost warning, and stream guidance. It is front-loaded and not redundant, though the stream caution is somewhat lengthy. No wasted words, but it could be slightly more scannable. Still, it earns a 4.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 10 parameters, no annotations, and no output schema, the description covers the critical context: when to use, how to avoid missing data, cost implications, and default behavior. It does not describe the exact return structure, but the concise-projection mention offers a hint. The absence of a return schema and the presence of sibling get_message mitigate this gap. Overall, a solid 4.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds strategic meaning for key params: verbose is tied to 'concise projection of high-signal fields', streams '*' is explained with the Default Stream caveat, and messageChars is contextualized as 'raw body vs parsed fields'. This goes beyond the schema's bare descriptions, justifying a 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource: 'Read individual matching log lines across one or more streams, merged newest-first.' It clearly distinguishes the tool from siblings by explicitly recommending 'analyze' for aggregation and implying this is for reading specific lines. The resource (log lines) and operation (search/read) are clear, and the newest-first merge adds behavioral scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use and when-not-to-use guidance: 'if you want to know WHAT is failing rather than read specific lines, use analyze first' and explains the cost trade-off. It also provides a concrete strategy for stream selection ('Pass streams:"*" to cover every readable stream') and warns about the Default Stream pitfall, directly aiding the agent in choosing this tool versus alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_fieldsA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description discloses key behavioral traits: it lists only fields that actually exist in the index, and warns about the misleading '0 matches' outcome. It also hints at the scale (thousands of fields). This is strong contextual disclosure, though it does not mention whether it is read-only or describe pagination, which would be nice.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four sentences, each serving a distinct purpose: stating the action, advising when to use, explaining the risk, and giving a usage suggestion. No fluff or redundancy. Well-structured with the primary action first.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and no annotations, the description provides sufficient context for an agent to decide when and how to invoke the tool. It covers the core use case, a critical pitfall, and parameter guidance. A small gap is the lack of detail about the return format or default behavior, but overall it is complete enough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3, but the description adds meaningful context for 'contains' by explaining its purpose and giving an example. It also reinforces the rationale for using limit. This goes beyond just restating schema definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists actual indexed fields, which is distinct from sibling tools like search or analyze. It immediately identifies the resource (message fields) and the action (list), making it unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs to use before searching on unseen fields, explains the failure mode (0 matches indistinguishable from no logs), and advises using the contains parameter to narrow results. This provides clear when-to-use guidance versus search tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_streamsA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full weight. It discloses that results are capped ('results are capped'), that the API token limits readability, and what the listing includes. It doesn't mention pagination or rate limits, but the key behavioral trait (cap) is present.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three dense sentences. The first states purpose, the second gives an alternative, the third explains the cap and how to handle it. No wasted words, front-loaded, every sentence contributes.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a list tool with no output schema, the description sufficiently explains return content, the cap, the filtering option, and the relationship to search/analyze. It covers all essential context an agent needs to decide and invoke the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds context beyond schema: explains why limit exists (thousands of streams), why titleContains is useful (capped results), and gives an example. This elevates it above baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('List the Graylog streams...') with the resource (Graylog streams) and the returned fields (id + title, and whether it removes matches from the Default Stream). It clearly distinguishes from siblings by steering users to search/analyze for searching everything.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly tells when to use an alternative: 'To search everything, pass streams:"*" to search/analyze rather than listing streams here.' Also instructs to use titleContains for finding a specific named stream, with a concrete use case.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyzeA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully carries the burden and does so admirably: it explains aggregation behavior, optional histogram, local substring filtering, and stream semantics, including the crucial caveat about the Default Stream and leading wildcard restrictions. It discloses these traits without contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but exceptionally well-structured with numbered use cases, and every sentence serves a purpose—from explaining the main function to providing operational warnings. It is front-loaded with the core purpose and remains focused without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 10-parameter tool with no output schema and no annotations, the description covers the essential operational context: what it does, when to use it, how parameters behave in practice, and caveats about streams and wildcard queries. It adequately compensates for missing structured metadata.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although schema coverage is 100%, the description significantly enriches parameter understanding by tying valueContains to a discovery use case, histogramInterval to a timing use case, and field to a failure-analysis use case. It adds contextual meaning beyond the schema entries.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool aggregates matching messages by top field values rather than returning raw lines, with a specific verb-resource relationship. It further distinguishes itself from sibling tools like search by explaining it collapses repetitions into counts, making it more informative and cheaper.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly outlines three use cases (what is failing, when, discover a value) and directly contrasts with search ('far cheaper and more informative than reading the same lines via search'). It also provides guidance on when to use streams:"*" and warns against pitfalls like the Default Stream, offering clear situational advice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.