Server Quality Checklist

Tool Scores

• list_mqtt_clientsC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool lists clients but doesn't describe how it behaves—e.g., whether it returns all clients or paginated results, if it requires authentication, what the output format is, or any rate limits. This leaves significant gaps for an agent to understand the tool's operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that efficiently conveys the core purpose without unnecessary words. It's front-loaded with the main action and resource, making it easy to parse, though its brevity contributes to gaps in other dimensions.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a listing operation with one undocumented parameter), lack of annotations, and no output schema, the description is incomplete. It doesn't cover parameter usage, behavioral details, or output expectations, making it insufficient for an agent to effectively invoke the tool without additional context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has one parameter ('request') with 0% description coverage, and the tool description provides no information about parameters. It doesn't explain what 'request' should contain (e.g., filtering criteria, pagination options) or its format, leaving the parameter entirely undocumented and unusable without external knowledge.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('List') and resource ('MQTT clients connected to your EMQX Cluster'), making it immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'get_mqtt_client' (which likely retrieves a single client) or mention the scope of listing (e.g., all clients vs. filtered).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when to prefer 'list_mqtt_clients' over 'get_mqtt_client' (e.g., for bulk retrieval vs. single client details) or 'kick_mqtt_client' (for management actions). There's also no context on prerequisites or limitations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• publish_mqtt_messageC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states the action ('Publish') which implies a write operation, but doesn't disclose behavioral traits like whether this requires authentication, what happens on failure, rate limits, or if messages are persistent. The description is minimal and lacks crucial operational context for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that states the core purpose without unnecessary words. It's appropriately sized for a simple tool, though it could be more front-loaded with critical usage information. No wasted verbiage.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations, 0% schema coverage, and no output schema, the description is incomplete. It doesn't explain what the 'request' parameter should contain, what happens after publishing, potential error conditions, or return values. The agent lacks sufficient context to use this tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It mentions publishing a message but provides no information about the 'request' parameter - what format it should be in (JSON, string), what content it should contain, or examples. The description adds minimal value beyond the schema's structural definition.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Publish') and resource ('MQTT Message') with specific deployment targets ('EMQX Cluster on EMQX Cloud or Self-Managed Deployment'). However, it doesn't explicitly distinguish this tool from its siblings (get_mqtt_client, kick_mqtt_client, list_mqtt_clients), which all operate on MQTT clients rather than messages. The purpose is clear but lacks sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing an active MQTT connection), exclusions, or how it relates to the sibling tools. The deployment context is stated but doesn't help the agent decide when this tool is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_mqtt_clientC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states this is a read operation ('Get detailed information'), which implies it's non-destructive, but doesn't disclose behavioral traits like authentication needs, rate limits, error conditions, or what 'detailed information' includes (e.g., connection status, subscriptions). This is a significant gap for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that gets straight to the point with no wasted words. It's appropriately sized and front-loaded, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (a read operation with 1 parameter), lack of annotations, and no output schema, the description is incomplete. It doesn't explain what 'detailed information' returns, error handling, or parameter details, leaving the agent with insufficient context to use the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 1 parameter ('request') with 0% description coverage, and the description doesn't add any parameter-specific information. It mentions 'by client ID', which hints that the parameter might be a client ID, but doesn't clarify the parameter name, format, or constraints, failing to compensate for the low schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get detailed information') and resource ('about a specific MQTT client by client ID'), making the purpose understandable. However, it doesn't explicitly differentiate from sibling tools like 'list_mqtt_clients' (which likely lists multiple clients vs. getting details for one), so it misses the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'list_mqtt_clients' or 'kick_mqtt_client'. It mentions 'by client ID' but doesn't specify prerequisites or contexts, leaving the agent to infer usage from the name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kick_mqtt_clientC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only states the action without behavioral details. It doesn't disclose if this is destructive (likely yes), requires specific permissions, has side effects (e.g., message loss), rate limits, or error conditions (e.g., invalid client ID).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's front-loaded with the core action and resource, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (destructive action, 1 parameter), lack of annotations, 0% schema coverage, and no output schema, the description is incomplete. It fails to address key aspects like parameter meaning, behavioral traits, or expected outcomes, leaving significant gaps for agent understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description adds no parameter semantics beyond the schema. It mentions 'by client ID' but doesn't clarify that the 'request' parameter is the client ID, its format, or constraints, leaving the single required parameter undocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Disconnect') and target resource ('a client from the MQTT broker by client ID'). It distinguishes from siblings like 'get_mqtt_client' (retrieve) and 'list_mqtt_clients' (enumerate) by focusing on termination of connections.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., client must be connected), exclusions, or compare with sibling tools like 'publish_mqtt_message' for messaging versus disconnection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.