Server Quality Checklist

Tool Scores

• syslog_grep_toolC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states 'Regex search,' which implies a read-only operation but does not explicitly confirm safety, nor does it mention behavior like result limits, sorting, or whether the search is across all entries by default. The tool's behavior beyond the search intent is opaque.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no filler. It conveys the core function and scope efficiently. Given the schema already documents parameters, this length is appropriate and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is minimal and omits crucial context: it does not explain when to prefer this tool over syslog_search_tool or syslog_tail_tool, does not mention the output format (though an output schema exists), and does not clarify that regex matching is case-insensitive (only noted in the pattern parameter schema). An agent has limited information to correctly invoke this tool in the broader workload context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, with every parameter having a description in the input schema. Therefore the baseline is 3. The description adds no extra semantic value beyond restating that the 'source' parameter restricts to one named source, which is already documented in the schema. The description does not clarify any parameter interactions or edge cases.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb (search) and resource (syslog sources), and specifies regex matching. It also notes the scope (all sources or one named source). However, it does not differentiate from sibling tools like syslog_search_tool, which likely performs a similar search but perhaps without regex, so an agent cannot easily distinguish which to use.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus alternatives. The description implies usage for regex search but does not explicitly mention exclusions or direct the agent to syslog_search_tool for non-regex searches. With several sibling search tools, this is a gap.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• syslog_search_toolC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only says 'Search collected logs' and gives no details about read-only behavior, result ordering, pagination, or what happens when no logs match. The tool has an output schema, but that is not shown and the description does not summarize return values or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no filler or redundancy. It is efficiently worded, and the key search dimensions are listed in a compact manner. It is slightly under-specified, but conciseness itself is good.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 7 parameters and an output schema, the description is sparse. It does not mention the 'program' filter, the default time window behavior (e.g., 'since' defaults to '1h'), or the case-insensitive regex pattern. It also does not clarify how this search differs from grep or tail. Given the complexity, a fuller description is needed to enable correct use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description does not add any parameter-level meaning beyond what the schema already provides; it merely names the filters without describing formats, defaults, or interactions (e.g., that severity is minimum severity). The schema explains each parameter well, so the description adds no extra value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb ('Search') and resource ('collected logs') and names key filter dimensions (source, time window, severity, pattern). It is specific, but it does not differentiate from sibling tools like syslog_grep_tool or syslog_tail_tool, so an agent cannot immediately tell which to pick.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no guidance on when to use this tool versus the sibling tools. It does not state conditions like 'use for historical searches' or exclude streaming/real-time use, nor does it mention the syslog_sources_tool for enumerating sources. An agent is left to infer usage from the vague 'Search collected logs'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• syslog_tail_toolB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears full responsibility for behavioral disclosure. It only states 'return', implying read-only, but does not explicitly state that it is non-destructive, does not require special permissions, or how it orders results. There is no mention of default behavior (e.g., pagination, limiting) beyond the schema default, and no warning about potential performance implications.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no redundant words. It is appropriately brief for a tool that likely just needs a quick factual statement. Nothing could be removed without losing meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple and has a full output schema, so the description does not need to explain return values. However, given the sibling tools (especially syslog_grep_tool and syslog_search_tool), the description lacks context on how this tool differs (e.g., no filtering, just tailing the latest entries). An agent selecting among siblings might need more guidance to pick the right one. This is a moderate gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents all three parameters. The description adds minimal semantic value – 'one source' vaguely references the source parameter but does not clarify accepted formats or relationship between parameters. This meets the baseline for full schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb ('Return') and resource ('most recent entries for one source'), which is specific enough to understand the basic function. However, it does not differentiate from sibling tools like syslog_grep_tool or syslog_search_tool – there is no mention of filtering or specific use cases, so it could be confused with other tools that also return entries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus the five siblings. It does not mention that this is for quick viewing of the latest logs without search or filtering, nor does it exclude any scenarios. An agent would have to infer usage from the name and schema.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• syslog_stats_toolA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears the full burden of behavioral disclosure. However, it only states the function without mentioning that it is read-only, what it returns beyond the output schema, or any side effects. It essentially restates the tool's name with slightly more detail, adding little beyond what is already obvious.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence with no extraneous words. The core purpose is front-loaded and directly stated. It is efficient and easy to parse, which is ideal for an agent scanning many tool definitions.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description, combined with a fully documented schema and an output schema, provides enough for basic correct invocation. However, it omits context such as common use cases, behavior when source is omitted, or time-window interpretation nuances. These are not critical given the schema, but the description alone would leave an agent without full operational context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides 100% coverage for all three parameters (top, since, source) with clear descriptions. The tool description adds no supplementary details about parameter usage or formatting. Per the baseline rule for high schema coverage, a score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Summarise') and identifies the resource ('log volume and error rate per source'), which clearly distinguishes it from siblings that search, tail, or list sources. An agent can immediately understand what this tool does and how it differs from syslog_grep_tool or syslog_tail_tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives. The name and description imply it is for aggregated statistics, but there is no direct statement like 'use this for summaries, not raw logs' or reference to sibling tools. The context is implied by the purpose, but not enforced.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• syslog_context_toolA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full responsibility for behavioral disclosure. It reveals that omitting source spans all sources, a useful trait, but does not state read-only-ness, response format, ordering, or potential limits. The presence of an output schema mitigates the missing return details, but the description alone leaves several behavioral aspects unaddressed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with zero waste. The primary purpose is front-loaded, and the usage hint follows naturally. Every word earns its place; no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with a fully documented schema and an output schema, the description provides sufficient context to call it correctly. It covers the primary use case and a helpful tip about fleet correlation. Minor gaps like time-range boundaries or default behavior are not critical given the complete schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all parameters are already documented. The description repeats the source-omission point already in the schema and adds usage context, but no new parameter-level meaning. The description contributes minimal value beyond the schema, so the baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it returns log entries around a specific moment, which is a distinct action from grep, search, or tail. However, it does not explicitly differentiate itself from sibling tools, so the purpose is clear but not maximally disjoint.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives an explicit trigger: 'Use this after an alert names a time.' It also explains a key usage pattern (omitting source for fleet-wide correlation) but does not mention which sibling to use instead in other scenarios, so it provides context without alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• syslog_sources_toolA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It implies a read-only operation ('List') but doesn't explicitly state non-destructiveness or any side effects. It also omits behavioral nuances like default filtering or how the 'include_internal' flag affects results, which are left entirely to the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence that states the action, target, and returned data without any filler. Every word contributes to understanding the tool's core function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has an output schema covering return values, and the description succinctly explains the primary purpose. However, it doesn't mention optional behavior (e.g., how the pattern filter works or that internal sources are excluded by default), though these are documented in the schema. For a simple listing tool, this is nearly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for both parameters (pattern and include_internal), so the baseline is 3. The description adds no additional meaning about these parameters—it focuses on the output fields rather than parameter usage, filters, or defaults.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and resource ('log sources'), stating exactly what the tool returns (size, last-write time). It is clearly distinct from sibling tools that search, grep, tail, or provide context, so an agent can differentiate it without opening schemas.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The purpose implies usage—when you need to enumerate available sources—but the description provides no explicit guidance on when to choose this over siblings like syslog_search_tool or syslog_grep_tool. No exclusions or alternative conditions are mentioned; usage is inferred rather than directed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.