Server Quality Checklist

Tool Scores

• graylog_search_messagesC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description must carry the full burden. It only states that a raw Lucene query is run over a time range, but does not disclose side effects (though search is likely read-only), rate limits, permissions, or pagination behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with zero filler words. It is maximally concise and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 9 parameters, a raw query language, and no annotation support, the description is too sparse. It only covers the query and time range, omitting important details about filtering, sorting, pagination, and stream selection. The output schema covers return shape but not tool-specific behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description only adds meaning to the time parameters by specifying 'absolute RFC3339'. The other parameters (query, limit, offset, fields, streams, sort, sort_order) are not explained in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'run' and resource 'Graylog Lucene query', clearly indicating a search over messages. It distinguishes from sibling tools that manage profiles or system info, though it does not explicitly say 'messages'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description does not mention sibling tools, exclusions, or context like 'use for searching log messages'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• graylog_get_profileA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It doesn't disclose side effects, error behavior, or the meaning of 'validated' beyond a vague qualifier. There's no mention of read-only status or what happens if no profile matches.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that conveys the essential action and selection criteria without any wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a simple retrieval tool, and the output schema is present, covering return structure. Yet the lack of behavioral context (e.g., validation logic, error handling) and usage guidance makes it only minimally complete for an agent to select and invoke confidently.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description compensates by clarifying that the 'name' parameter can be a name or an alias. However, it doesn't provide additional constraints or examples, leaving room for ambiguity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Return' and clearly identifies the resource as 'one validated business diagnosis profile', with selection criteria 'by name or alias'. This effectively distinguishes it from sibling tools like graylog_list_profiles.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool should be used when you have a specific name or alias, but it provides no explicit comparison to alternative tools (e.g., graylog_list_profiles for listing all profiles) or when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• graylog_diagnose_code_logsA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden for behavioral disclosure. It does reveal a two-step behavior (correlate, then expand) and notes that a business profile changes behavior ('adds stage patterns and correlation keys without changing this tool'). However, it doesn't explain what 'expand discovered trace keys' means operationally, any side effects, or how the tool interacts with Graylog. It adds some value but lacks depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded purpose, no filler. The first sentence states the core action, the second gives usage timing and input guidance, and the third adds a profile nuance. Every sentence earns its place; the description is efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 8 parameters and no annotation coverage, but does have an output schema. The description covers the tool's purpose and the most important input semantics, yet it omits guidance on time range parameters, streams, and limit, which are common but not fully self-explanatory in context. Given moderate complexity and an output schema, the description is adequate but leaves gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explicitly explains the meaning of log_literals ('stable literal text from those statements') and identifiers ('known values such as studentExamId, taskId, requestNo, or traceId'), and hints at profile. However, it leaves parameters like from_time, to_time, range_seconds, limit, and streams undocumented, relying on their names alone. The description adds value for core parameters but not for all eight.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific action ('Correlate code log literals and runtime identifiers, then expand discovered trace keys') that clearly distinguishes this tool from siblings like search_messages or list_profiles. It names the resource (code logs, trace keys) and the scope (correlate + expand), making it unmistakable what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use guidance: 'Use this after reading relevant source and finding log statements.' It also tells the caller what to pass (stable literal text, known identifiers) and mentions the profile behavior. While it doesn't explicitly name alternatives or exclusions, the context given is clear enough to guide selection among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• graylog_list_profilesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description is straightforward for a list operation, and 'List' implies a read-only behavior. However, no annotations are provided, and the description does not explicitly disclose that no data is mutated, nor does it mention any permissions, pagination, or error behavior. It adds some context with 'trigger aliases' but stops short of a fuller behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no filler. It front-loads the core action ('List') and resource, and every word contributes to understanding the tool's function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list operation with zero parameters and an output schema present, the description covers the essential purpose and output contents ('profiles and their trigger aliases'). It does not mention potential caveats like result limits or ordering, but given the simplicity of the tool, this is sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema is empty. According to the rubric, a baseline of 4 applies when there are no parameters. The description does not need to compensate for any parameter ambiguity since there are none.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('List') on a specific resource ('business diagnosis profiles') and adds the detail of 'trigger aliases' in the output. This clearly distinguishes it from sibling tools like graylog_get_profile, which targets a single profile.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied: use this tool when you need to enumerate all available business diagnosis profiles. However, there is no explicit mention of when NOT to use it or alternatives, such as graylog_get_profile for a specific profile. The sibling tool names provide context but the description itself offers no direct guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• graylog_get_system_infoA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the transparency burden. It discloses that the call is a read-only operation ('Verify', 'return') and indicates the output type, but does not elaborate on access requirements, potential side effects (likely none), or response format. This is adequate for a simple system-info getter.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that states the action and the result. Every word earns its place, with no fluff or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool (no parameters, no nested objects), the description sufficiently covers the purpose and high-level output. It could mention that this is a lightweight health-check call, but the provided information is largely complete for invoking it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, which the schema makes fully explicit (100% coverage). Per the rubric, a zero-parameter tool gets a baseline of 4 since there are no parameter semantics to explain.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool verifies Graylog connectivity and returns cluster version/node metadata. It uses a specific verb ('verify') and resource ('system info'), and is distinct from sibling tools focused on profiles, messages, and diagnostics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies this is for checking connectivity and retrieving system-level info, which is a distinct use case from the sibling tools. It doesn't explicitly state when not to use it, but the purpose is self-evident enough for an agent to choose correctly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.