Server Quality Checklist

Tool Scores

• room_statusA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It usefully indicates that the tool is about where the discussion stands and lists the exact data it reports, implying a read-only snapshot. However, it does not explicitly state that it has no side effects, whether it waits, or whether it consumes any state.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that front-loads the purpose and then lists three concrete, useful details: turn, round, and room closure. There is no filler or redundant content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter status tool with no output schema, the description adequately conveys the key returned information. It could be more complete by explicitly stating that the tool is non-mutating or how it differs from read_room, but the simplicity of the tool keeps this gap minor.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema already leaves nothing to document. The baseline of 4 applies because the description adds no parameter-specific detail, but none is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states what the tool reports: the current turn, budget round, and whether the room has closed. This gives a specific status resource and distinguishes it from siblings like post, write_spec, and close_room, though it lacks an explicit imperative verb such as 'get' or 'read'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It implies you should use this tool when you need a snapshot of the discussion state, but it does not explicitly say when to prefer read_room or wait_for_message instead. No exclusions or alternative conditions are given, so usage guidance is inferred rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• close_roomA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry behavioral disclosure, and it does reveal the main effect (ending the discussion) plus the prerequisite and default path behavior. However, it doesn't say whether closing is irreversible, how it affects room_status or pending messages, or what response the caller should expect.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences with no filler; the core purpose is front-loaded and the prerequisite/default follow naturally. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a two-parameter, no-output-schema tool, the description plus schema covers the essential calling sequence and the optional path behavior. It could be more complete about post-close state changes, but nothing needed for correct invocation is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3, but the description adds meaningful parameter context by revealing that spec_path defaults to the path reported by room_status. This is information not present in the input schema and directly affects how an agent should invoke the tool with zero arguments.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description uses specific verb 'End' plus resource 'discussion', clearly distinguishing close_room from siblings like post, read_room, and wait_for_message. It also frames the tool's role as the terminal action after a plan is written, which leaves no ambiguity about its purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives an explicit precondition: write the agreed plan to the spec file named in room_status first, then call close_room. It doesn't explicitly name write_spec or list when-not-to-use cases, but the sequencing is unambiguous and actionable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• postA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the budget cost ('Each post you make consumes one round of the budget'), which is a key behavioral constraint. However, it does not mention message visibility, what happens after posting, or any failure behavior, leaving some gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two focused sentences: the first states the core action, the second discloses the budget cost, and the third gives usage guidance. Every sentence earns its place with no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 2-parameter tool with no annotations or output schema, the description covers purpose, usage, and the critical budget behavior. It is slightly incomplete because it doesn't mention what happens on success/failure or explicitly cover the 'decision' kind, but overall it is quite complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds meaning beyond the schema by mapping kind values to use cases: 'set the brief' (brief), 'push back on the peer' (challenge), 'ask a narrower question' (question). It does not address the 'decision' kind, so it is not a full 5.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb and resource ('Post to the room') and explains the tool's role with concrete use cases: 'set the brief, push back on the peer, or ask a narrower question.' This clearly distinguishes it from sibling tools like read_room or write_spec.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit usage context: 'Use it to set the brief, push back on the peer, or ask a narrower question.' This tells the agent when the tool is appropriate, though it does not explicitly mention alternatives or when-not-to-use conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_roomA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden, and it does well by disclosing the stateful behavior of returning only previously unseen messages and the effect of `since`. It does not describe the output format or whether reading advances a cursor, but the core non-obvious behavior is clearly exposed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two compact sentences deliver both the core purpose and the key behavioral nuance. The most important information is front-loaded, and every sentence earns its place with no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one optional parameter, no required parameters, and no annotations or output schema, the description is sufficiently complete. It tells the agent what the tool returns, how sequence numbers affect the result, and what behavior to expect on repeated calls.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3, and the description adds value by connecting `since` to the 'continuing from where you left off' model. It clarifies that omitting `since` returns everything not already shown, which goes beyond simply restating the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb (Read) and resource (messages), and clearly explains the tool's distinctive behavior: it returns everything not already shown unless `since` is passed. This differentiates it from siblings like wait_for_message, which implies blocking for new messages, and post, which is for sending.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly establishes when to use the tool: to read messages posted so far and catch up on anything not yet seen. It does not explicitly name alternatives or state when not to use it, but the context is strong enough that an agent can infer the appropriate scenario.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• wait_for_messageA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden, and it delivers: it discloses blocking behavior, timeout semantics, the two possible non-message return states, and the required follow-up action. This goes well beyond the tool name and schema by explaining exactly how to behave in the conversation loop.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four short sentences, each earning its place: one states the core action, two define return states, and one gives the practical usage rule. The most important information is front-loaded, with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the single parameter, no output schema, and no annotations, the description covers everything an agent needs: what it returns, what to do on idle, what to do on closed, and how to stay in the conversation. No critical behavior is left unexplained.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% and the single parameter already explains timeout duration plus default and maximum in the schema. The description does not need to add further parameter detail, so the baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource: 'Block until the peer posts, then return their message.' It clearly distinguishes this from non-blocking siblings by emphasizing the blocking wait and the return contract, making the tool's unique role unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear usage guidance: call it again on idle, treat 'closed' as a stop signal, and do not end the turn while the room is open. It stops short of explicitly comparing against alternatives like read_room or room_status, so it doesn't fully earn a 5, but the intended usage pattern is well communicated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• write_specA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden of disclosing behavior. It explains that this tool is the sole output channel, writes a plan rather than code, and should be called after convergence. It does not explicitly state overwrite or repeat-call behavior, but the workflow context strongly implies a single final write.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loads the main purpose, and every sentence adds useful guidance: what to write, when to write it, and what to do afterward. There is no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given one simple parameter fully documented in the schema and the low complexity of the surrounding tool set, the description covers the tool's role, the right moment to call it, and the next step. An agent has enough information to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Only one parameter exists and the input schema already covers it fully, including required content structure and markdown format. The description adds no new parameter-level detail, but with 100% schema coverage, that is acceptable.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action and resource: 'Write the plan.' It further differentiates the tool from siblings by calling it 'the only way anything leaves this room' and 'the only file you can create,' so an agent can tell exactly what this tool is for.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit timing and sequencing: 'Call it once you have converged, then close_room.' It also tells the agent what not to do here — 'you have no file-writing tools and you are not here to implement anything' — which prevents misuse.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.