Server Quality Checklist

Tool Scores

• describe_topicC
  2.8/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only enumerates the topic details and does not state that the operation is read-only, mention any side effects, response characteristics, or permission requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short and directly lists the relevant content areas, making it easy to scan. It lacks a proper sentence structure, but the word choice is efficient and there is virtually no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter describe operation, the description gives a useful preview of the returned information, and the schema fully documents the parameter. However, with no annotations or output schema, and no usage guidance, the description is minimally complete rather than fully informative.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema describes the sole topic parameter fully (100% schema description coverage). The tool description does not add any parameter semantics, but the parameter is already well defined, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description lists the specific topic metadata (partitions, replicas, in-sync replicas, non-default configs), making the resource and focus clear. It lacks an explicit verb, but the title 'Describe topic' plus the attribute list provide adequate purpose clarity and distinguish it from sibling tools like list_topics or topic_offsets.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool instead of the listed siblings, such as cluster_info, list_topics, or topic_offsets. The context is implicit at best and does not mention exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• topic_offsetsA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of conveying what happens. It does accurately convey that this is a read-only-style view of offsets per partition, which is useful. However, it does not mention per-communication behavior such as permissions, topic existence handling, whether it is read-only, or how output is structured. It is not misleading, just thin.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one concise sentence that front-loads the key purpose (earliest/latest offsets) and qualifies it with 'message backlog view'. There is no redundant wording or irrelevant details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with a single string parameter and no output schema, the description provides enough context to understand the data being fetched: per-partition earliest and latest offsets. It could mention partition considerations more explicitly, but the core meaning is complete and aligned with the sibling domains.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the parameter 'topic' is already described as a topic name. The description adds the context that the output is per partition for that topic, which is useful but not strictly necessary for filling in the parameter meaning. It does not provide format or validation details, but the simple schema already covers that.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states what the tool returns: earliest and latest offsets per partition for a topic. The 'message backlog view' qualifier gives it a distinct operational meaning. It is specific enough that the tool can be distinguished from sibling tools like describe_topic, even though it does not explicitly name them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when wanting a message backlog perspective (earliest vs. latest offsets) for a topic, but it does not explicitly state when to use this instead of siblings like describe_topic or list_topics. There are no exclusions or alternative guidance, so the usage context is only implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• describe_consumer_groupA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden, but only reveals a read-like action ('describe') plus the aggregation detail ('per-partition and total lag'). It doesn't state behavior on nonexistent groups, whether broker-side offset fetching occurs, or what 'state' fields actually appear. It is non-destructive by wording but adds little safety/permission context beyond the annotation-free baseline.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One sentence, front-loaded with the core purpose (state + lag) and entirely free of filler. It is appropriately sized for a single-parameter read tool and every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema and no annotations, the description tells the agent what it computes (state, per-partition lag, total lag), which is the surface coverage expected. However, it leaves the shape of 'state' vague, gives no field directions, and offers no routing guidance to siblings — so an agent cannot predict the full return structure or the prerequisite (existing groupId) without inference.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% since groupId is fully described as 'Consumer group id', so the baseline is 3. The description only reinforces the role by saying 'across its topics', adding no format, source, or units detail beyond the schema. It doesn't need to compensate further, but it doesn't add paramet-specific meaning either.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb-resource pair ('describes a consumer group') and adds a unique deliverable — per-partition and total lag computed across its topics — which is exactly what separates it from the sibling list_consumer_groups. The title reinforces the scope with '(with lag)'. An agent reading this knows precisely what the tool returns that no other sibling provides.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The mention of lag computation implies 'use this when you need lag or the detailed state of a specific group,' but no when/when-not guidance is given. There is no explicit statement like 'to enumerate groups first, use list_consumer_groups' or how to choose between this and topic_offsets. Usage context is visible only by implication.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_topicsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the behavioral disclosure burden. It does reveal a meaningful behavior: allowlist filtering. However, it does not state whether the operation is read-only, what happens when includeInternal is false, or any return-level behavior beyond names. It adds some transparency but not rich context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences, no filler, and the core purpose is front-loaded. Every word earns its place, and the allowlist note is a relevant scoping detail rather than noise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list operation with one optional, fully documented parameter and no output schema, the description plus schema is largely sufficient. It tells the agent what output to expect (topic names) and what filtering applies. Minor missing context includes whether results are ordered or paginated, but these are not critical for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for the single includeInternal parameter, so the schema already explains its meaning. The description does not need to add parameter detail, and it doesn't, which matches the baseline 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb and resource: 'List topic names.' It is immediately clear this is a read-only enumeration operation over topics, distinguishing it from siblings like describe_topic and topic_offsets without requiring schema inspection. The allowlist filtering clause adds precise scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage—use this when you want topic names within the allowlist—but provides no explicit when/when-not guidance or alternatives. It does not mention when to prefer describe_topic or other siblings, leaving the choice to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cluster_infoA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. 'Describe' conveys a read-only action and the listed response elements set expectations, but it does not mention authorization, connection, or failure behavior. For a simple metadata read this is a reasonable but not strong disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence states the action and immediately lists the exact returned information. Every word earns its place with no tautology or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-argument, read-only cluster metadata tool, the description covers what is fetched and is sufficient for an agent to select and invoke it correctly. No output schema or nested structure creates additional ambiguity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is no parameter ambiguity to clarify. The schema provides complete coverage, and the no-parameter baseline of 4 applies here.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Describe') with a clear resource ('the Kafka cluster') and enumerates exactly what it returns: brokers, controller, and cluster id. This clearly differentiates it from sibling tools that operate on topics and consumer groups.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The implied usage is clear: use this when you need cluster-level metadata rather than topic or consumer-group details. However, the description does not explicitly name alternatives or state when not to use it, so the routing guidance is only implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_consumer_groupsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the transparency burden. 'List' makes the read-only behavior evident, and 'and their protocol types' discloses what the output contains. It does not mention ordering, empty results, or scope, but for a simple list operation this is largely transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one short, front-loaded sentence with no filler. It includes the action, the target resource, and the relevant output detail efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is sufficiently complete for a no-arg tool, since it states both the resource and the returned information. Without an output schema, a little more detail about the exact shape or scope of the returned list would improve completeness, but the gap is minor.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema is empty, so there are no parameters to document. With zero required parameters and complete schema coverage, the description does not need to add parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb ('List') and resource ('consumer groups') and adds a useful output detail ('and their protocol types'). The plural 'consumer groups' and 'list' wording distinguish it from the sibling describe_consumer_group.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The word 'List' implies use when an overview of consumer groups is needed, but the description does not explicitly explain when to prefer this over describe_consumer_group or when not to use it. No alternatives or exclusions are given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.