Server Quality Checklist

Tool Scores

• release_laneB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description adds behavioral context that a conflict returns ok=False instead of a crash, but does not disclose other traits like whether the operation is destructive or requires certain permissions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two clear, focused sentences; the first states the purpose, the second adds error-handling nuance. Efficient with no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of parameter info and no annotations, the description is insufficient for agents to understand how to use the tool correctly, especially without explaining what lane and owner values are valid.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fails to explain what 'lane' and 'owner' represent, providing no additional meaning beyond the bare field names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'release' and identifies 'coordination lane' as the resource, clearly distinguishing it from siblings like claim_lane and heartbeat_lane.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by stating how conflicts are handled (409 becomes ok=False), but does not explicitly state when to use the tool vs alternatives or provide prerequisites like having claimed the lane.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• post_messageA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses that action_flag is stored but display-only (no action performed). However, no annotations are provided, and the description does not cover other behavioral aspects like idempotency, rate limits, or potential side effects beyond append-only.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two brief sentences that efficiently convey core purpose and a key nuance. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Adequately describes the tool's main function and a behavioral quirk, but lacks details on response format (though output schema exists), error conditions, and parameter constraints. Sufficient for a simple append operation but not exhaustive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must compensate, but it only explains action_flag's behavior. The other parameters (topic, sender, body) are left undefined, providing insufficient guidance for correct invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the verb 'Append one message' and the resource 'coordination-bus blackboard', and specifies append-only behavior. Distinguishes from sibling tools like read_messages, release_lane, etc., which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implied usage context from sibling tool names (e.g., read_messages for reading), but no explicit when to use or when not to use. Lacks guidance on prerequisites or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_messagesA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavior. It mentions ordering and filtering but omits details like read-only nature, pagination, maximum limit, or performance characteristics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence that front-loads key information: what (recent bus messages), how ordered (newest first), and optional filter. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With an output schema present, return values are covered externally. The description adequately communicates the core functionality for a simple read operation, though minor details like the default limit behavior could be mentioned.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%. The description adds value by mentioning 'optionally filtered by topic' for the topic parameter, but does not explain the limit parameter (default, max) or what constitutes a valid topic string.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb (read), resource (bus messages), ordering (newest first), and optional filtering (by topic). It distinguishes from sibling tools like post_message (write) or heartbeat_lane (status).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not explicitly state when to use this tool versus alternatives like post_message or get_bus_status. It implies reading messages but lacks context-specific guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• claim_laneA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description fully discloses behavior: conflict handling (409 with ok=False), lease clamping, and backward compatibility with older servers. It warns about checking the effective lease_s rather than assuming the requested value.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the main purpose and is structurally sound. It is somewhat lengthy but every sentence adds value, covering modes, conflict, lease clamping, and version info.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity, lack of annotations, and presence of an output schema, the description covers all essential aspects: behavior, edge cases (409, older servers), and guidance for checking effective lease_s. It is complete for correct selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description explains the lease_s parameter's behavior (requested vs. effective) and implies the meanings of lane and owner through context. It does not explicitly describe lane and owner, but the usage context is sufficient for understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool's purpose: 'Claim a coordination lane before starting work in it.' It lists three specific modes (claim-if-free, steal-if-lease-expired, renew-if-you-already-own-it), which distinguishes it from sibling tools like release_lane and heartbeat_lane.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use (before starting work) and references get_bus_status for checking the ceiling. However, it does not explicitly state when not to use or compare directly with siblings, though the behavior is well-defined.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_bus_statusA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully covers behavioral traits: it returns a roll-up of status data, and exposes `_meta.max_lease_seconds` which represents a silent clamp for claim/heartbeat requests. It also clarifies that absence of `max_lease_seconds` is not an error but indicates an older server. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with the main purpose upfront, followed by detailed context about `_meta.max_lease_seconds`. While slightly lengthy, every sentence adds value. It could be slightly more concise but remains efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema (indicated by context), the description does not need to detail return values. It covers the essential elements of the roll-up and provides critical context about the meta field. The tool is complex but the description is complete for an agent to use effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the description does not need to add parameter semantics beyond the schema. Baseline 4 applies as no parameter info is required.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool provides a roll-up for the command-center panel including active lanes, orphaned/stale claims, recent messages, and pending display-only action flags. It also specifies the exposure of `_meta.max_lease_seconds` for coordination-bus v1.3+. This distinguishes it from siblings like claim_lane and heartbeat_lane.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises to check this tool before claiming a lane for longer than the default to know if the request will be honored in full. It also notes behavior differences for older bus servers, providing clear guidance on when to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• heartbeat_laneA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses that renewal is subject to a server-side lease ceiling, the response's lease_s may be shorter than requested, and older bus servers (pre-v1.3) omit lease_s entirely. It also mentions the 409 error condition and references get_bus_status for the current ceiling. This is comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise yet thorough. It starts with the core action, then covers error handling, lease ceiling behavior, and older server behavior in a logical order. Every sentence adds value without redundancy. It is well-structured and information-dense.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (renewal with lease ceiling, response semantics, error handling), the description covers all necessary aspects. It explains the response's lease_s, references get_bus_status for configuration, and handles the 409 error. An output schema exists, so return values are not required in the description. It is complete for an agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%. The description clarifies the lease_s parameter as the requested duration and explains that the response's lease_s is the effective granted duration. It does not explicitly define lane or owner, but they are implied from context (lane is the coordination lane, owner is the holder). The description adds meaningful interpretation for lease_s but could be more explicit about lane and owner.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action: 'Renew the lease on a coordination lane you hold live.' It uses a specific verb (renew) and resource (coordination lane lease). It distinguishes from siblings like claim_lane and release_lane by focusing on renewal and mentioning the 409 error that tells you to re-claim instead of crashing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use: when holding a live lane to extend its lease. It provides a specific condition: a 409 status indicates you need to re-claim (using claim_lane) rather than retrying the heartbeat. It also advises checking the response's lease_s rather than assuming the request was honored, which guides correct usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.