Server Quality Checklist

Tool Scores

• mqtt_publishB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions the basic action (publishing) and return value (confirmation message), but lacks critical behavioral details like authentication requirements, error handling, network behavior, or what happens if the broker is unavailable. For a network operation with potential side effects, this is insufficient transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly structured and concise: a clear purpose statement followed by well-organized parameter documentation with defaults indicated. Every sentence adds value with no redundancy or unnecessary information. The Args/Returns format makes it easy to parse while maintaining readability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (network operation with 4 parameters) and complete lack of annotations/output schema, the description does an adequate job covering the basics but has significant gaps. It explains parameters well and states the return type, but misses critical behavioral context about authentication, error conditions, and MQTT-specific behaviors that would help an agent use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds significant value beyond the schema, which has 0% description coverage. It explains each parameter's purpose: 'topic' as the destination, 'message' as the payload, 'qos' with valid values (0,1,2) and default, and 'retain' with meaning and default. This compensates well for the schema's lack of descriptions, though it could provide more context about QoS implications.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Publishes a message') and target resource ('to a specific MQTT topic'), making the purpose immediately understandable. It distinguishes from the sibling 'mqtt_subscribe' by specifying the opposite operation (publish vs subscribe). However, it doesn't explicitly mention what makes this tool unique beyond the basic verb, so it falls short of a perfect 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives or in what context it should be applied. While the sibling tool 'mqtt_subscribe' exists, there's no explicit comparison or recommendation about choosing between publish and subscribe operations. This leaves the agent without usage context beyond the basic purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mqtt_subscribeA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explains the subscription behavior, message collection, and timeout mechanism, but lacks details about connection requirements, error handling, or what happens if no messages arrive. It provides basic operational context but misses important behavioral aspects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly structured with a clear opening sentence followed by well-organized Args and Returns sections. Every sentence earns its place by providing essential information without redundancy. The formatting enhances readability while maintaining brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (3 parameters, no annotations, no output schema), the description covers the basic operation and parameters well but lacks information about connection state, authentication requirements, error conditions, and the format of the returned payload. It's adequate for basic use but incomplete for robust implementation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must fully compensate. It provides detailed semantic information for all three parameters: explains topic wildcards (+ and #), clarifies num_messages as a maximum with default, and specifies timeout units (seconds) with default. This adds significant value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Subscribes to an MQTT topic') and resource ('receives a specified number of messages or waits for a timeout'), distinguishing it from the sibling tool mqtt_publish. It provides a complete picture of what the tool does beyond just the verb.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like mqtt_publish, nor does it mention prerequisites, error conditions, or typical use cases. It only describes what the tool does without contextual usage information.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.