Server Quality Checklist

Tool Scores

• list_channelsA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, so the safe-read nature is covered. The description adds the case-insensitive substring behavior for the filter, which is useful. However, it doesn't describe what fields are returned per channel (id, name, type?), pagination, or whether archived channels are included. Adds some value beyond annotations but leaves behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tight sentences plus an Args section. Zero wasted words, fully front-loaded with the primary verb and scope. Highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-parameter read tool with readOnlyHint and an output schema present, the description is reasonably complete. It explains the filter semantics and scope. However, it's unclear what happens with no filter (returns all public channels), and whether results are ordered or paginated. Output schema exists so return shape isn't required, but missing details on full-list behavior leave minor gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description carries the burden for the single parameter. It does explain name_filter as a case-insensitive substring to match channel names, which adds meaning beyond the schema's bare type. However, it doesn't detail optionality semantics or default behavior when omitted. Adequate but minimal.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description states it lists PUBLIC channels with optional name substring filtering. The verb+resource (list channels) is clear and distinguishes from siblings like search_messages or read_channel_history. However, it doesn't explicitly note it excludes private/DM channels beyond the PUBLIC emphasis, which is implied.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage through the PUBLIC qualifier, suggesting private channels should be handled elsewhere. But it doesn't explicitly state when to use this vs read_channel_history (which is about a single channel's messages) or search_messages. No explicit when-not or alternative tool guidance is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_channel_historyA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations provide readOnlyHint=true, which covers the safety profile. The description adds the public-channel constraint beyond the schema. It doesn't disclose pagination behavior, rate limits, or whether the tool excludes the around_ts message or includes messages before it, but with readOnlyHint already declared, the bar is lower; the description adds modest value.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact with clear parameter listing and useful details (public-only constraint, limit bounds, around_ts semantics). Slightly verbose with the 'Args:' formatting but well-organized and front-loaded with the primary purpose. No wasted sentences.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With an output schema present, return-value documentation isn't needed. The description covers the tool's read-only nature, public-channel constraint, parameter semantics, and alternative mode (around timestamp). For a moderately complex tool with 3 parameters, this is reasonably complete, though it could mention pagination limits more explicitly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains channel (id or #name), limit (1-100, default 30), and around_ts (returns up to and including that timestamp). This adds genuine meaning beyond the bare schema, which has no descriptions. However, it doesn't elaborate on the exact format of channel ids or timestamp format, so it's adequate but not exhaustive.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb+resource: 'Read recent messages from a PUBLIC channel, or messages around a timestamp.' This clearly distinguishes it from sibling tools like send_message (write) and search_messages (query across channels). However, it doesn't explicitly differentiate from read_thread, which is a closely related read tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states channels must be public ('Must be a public channel'), which is a critical constraint. It also clarifies the around_ts behavior for reading context at a specific point. However, it doesn't explicitly say 'use search_messages instead when looking across channels,' so alternatives aren't named.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• resolve_usersA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The readOnlyHint annotation already declares this as a safe read operation. The description adds minimal behavioral context beyond this - it explains what the tool does (resolves ids to display names) but doesn't mention edge cases like unknown/invalid ids, batch limits, or format of the output. With annotations covering the read-only nature, a modest score is appropriate since the description adds only marginal value.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact - one functional sentence plus a brief parameter note. It's efficient with no filler, though putting the Args detail in the description duplicates what the schema already provides (a small redundancy). The core message is front-loaded and clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has a single simple parameter with 100% schema coverage (user_ids is the only param and it's mandatory). An output schema exists. For such a simple one-parameter, read-only utility, the description adequately covers what's needed - resolve ids to display names. There's an output schema to describe return structure, and the operation is straightforward. Not much more is needed for this simplicity level.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% for the parameter description, but there's only one parameter (user_ids) and the tool description itself explicitly explains its meaning ('List of Slack user ids to resolve') and even gives an example format ('U0123'). The description fully compensates for the 0% schema coverage on this single, self-explanatory parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb+resource pattern: 'Resolve Slack user ids to display names.' This clearly states what the tool does and what its output is (display names). Among siblings like search_messages, read_thread, and list_channels, this purpose is clearly distinct - none of the other tools resolve ids to names. It's immediately clear what this tool is for.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use it (when you have Slack user ids and need display names) but doesn't explicitly discuss when NOT to use it or how it contrasts with alternatives. The sibling tools have clearly different functions, so confusion risk is low, but there's no explicit guidance on context or exclusions. This is adequate but not richly developed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_threadA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint=true, covering the safety profile. The description adds that this requires a public channel (behavioral constraint) and returns root+all replies (scope), but doesn't disclose pagination, reply limits, or whether it fetches all reply levels recursively. With annotations covering safety, this is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is tight: one purpose sentence, then a compact Args section leading with the critical public-channel constraint. No wasted words, front-loaded with the core action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With readOnlyHint annotation and an output schema present, the burden is reduced. The tool is simple (2 params, no nesting). The description covers purpose, param semantics, and the key public-channel constraint. Minor gaps like reply-depth behavior are not documented, but the output schema likely covers the return structure, so this is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does add meaning for both params: channel accepts 'id or #name' and must be public, and thread_ts is described as 'the root timestamp (the ts of the first message)'. A 0-coverage schema with this param documentation earns credit, though format details (timestamps in float vs string, etc.) are left undocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Read a full thread (root message + all replies) from a PUBLIC channel' with a specific verb+resource+scope. It distinguishes itself from siblings like read_channel_history (which reads the channel surface) by explicitly scoping to a thread.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clarifies this works on PUBLIC channels only, which is important guidance. It doesn't explicitly name alternatives or when-not-to-use, but the public-channel constraint and the nature of the operation give clear context against siblings like search_messages and read_channel_history.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• draft_messageA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden for behavioral disclosure. It clearly discloses that the tool does NOT send, validates the target is a public channel, renders exactly what would be posted, and returns a one-time confirmation_token. It also explains the token's one-time nature. Minor gap: it doesn't describe error behavior for non-public targets, but core safety-critical behaviors are well covered. No annotation contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Mostly tight and well-structured. The 'THIS DOES NOT SEND ANYTHING' warning is bolded-in-effect and front-loaded, leading with the critical safety caveat. The Args section is compact. Minor redundancy: 'renders exactly what would be posted' and 'Always show the user this preview' slightly overlap, and 'THIS DOES NOT SEND ANYTHING' plus 'It validates...' creates a touch of repetition. Still, every sentence earns its place and the critical guidance is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having no annotations and 0% schema coverage, the description is fairly complete. It documents the return value (preview + confirmation_token), the workflow with send_message, and validates all three parameters at least at a basic level. An output schema exists which can document the preview/return structure. Gaps: behavior for invalid public-channel target is not described, and there's no mention of rate limits or auth requirements. It's not perfect but is complete enough for a drafting tool with this complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% (the JSON schema has no descriptions on properties), so the description must compensate. The description does add some semantic value: channel accepts 'Channel id or #name' and 'Must be a public channel'; text is 'the exact message text to post'; thread_ts is 'Optional thread root timestamp to reply in a thread.' However, the description partially restates what the schema already conveys through names and required-status (thread_ts optional, channel/text required). It adds moderate value but doesn't fully elaborate edge cases or formats beyond what property names suggest.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Purpose is explicit: 'Prepare a message for sending and return a preview + confirmation token.' It clearly states the verb (prepare) + resource (message) and distinguishes from siblings by declaring it is a draft/preview step, distinct from send_message. The negated purpose 'THIS DOES NOT SEND ANYTHING' sharply differentiates it from its sibling send_message.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Excellent usage guidance. It explicitly says 'To actually post, the user must review this preview and then call send_message(confirmation_token=...).' It states the workflow prerequisite, instructs 'Always show the user this preview and get their explicit approval before calling send_message', and references the sibling tool by name. This is exemplary when/when-not guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_messagesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations provide readOnlyHint=true, and the description reinforces the read-only nature by framing it as a search. The description adds the public-only scope constraint and the count cap (1-100, default 20) which the annotation doesn't cover. For a read-only search with good annotation coverage, this is solid supplemental disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded with the key differentiator ('PUBLIC channels only') in the first sentence. The Args section is brief and readable. Slightly redundant (the default 20 appears twice, in counting and in args), but otherwise tight with no wasted prose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  An output schema exists, so return-value documentation isn't required. The tool is simple (2 params, one required) and the description covers purpose, scope limitations, and parameter semantics fully. For a filtered-search tool with readOnlyHint and an output schema, this is adequately complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for parameter meaning. It does: 'query' is explained with concrete examples (ticket key, PR URL, keywords), and 'count' is documented with range and default. This fully covers the semantic burden of both parameters despite zero schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb+resource ('Search Slack messages') with a clear scope restriction ('in PUBLIC channels only'). It explicitly differentiates itself from siblings by noting DMs and private channels are excluded, distinguishing from read_channel_history and read_thread which cover private/thread content.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit usage context: 'Use this to find discussions about a Jira ticket or PR' with concrete query examples (ticket key, PR URL, keywords). It states the exclusion clearly ('Results from DMs and private channels are never returned'), guiding the agent away from using this tool for private-channel lookups. This is near-explicit when/when-not guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• send_messageA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full responsibility for behavioral disclosure. The description reveals that the token is 'one-time' (consumable on use), that the action is irreversible (posts to Slack), and that confirmation requires explicit user approval. This is meaningful behavioral context. While it doesn't mention potential errors or rate limits, the core safety-relevant traits are disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is tightly packed with zero wasted words. Every sentence earns its place: first states the action and prerequisite, second establishes exclusivity in the Slack-posting domain, third gives the approval precondition, and fourth documents the parameter. Well front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has a single required parameter with a clear origin (from draft_message), an output schema exists, and the description fully explains the workflow (draft → review → approve → send). It's a simple tool and the description covers the essential workflow context thoroughly. Minor gap: doesn't mention what happens if the token is invalid or already used.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. The single parameter (confirmation_token) is meaningfully explained as 'The one-time token returned by draft_message,' adding context about its origin and one-time nature beyond the bare schema title 'Confirmation Token.' This is sufficient for a single-parameter tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb+resource: 'Send a previously drafted message.' It explicitly distinguishes from siblings by noting 'This is the ONLY tool that posts to Slack' and references draft_message as its required precursor, making its relationship to sibling tools clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use conditions: only after draft_message has returned a token, and only after the user has reviewed the preview and explicitly approved. This effectively defines when NOT to call it (before user approval), and clearly differentiates it from read-only siblings like search_messages and read_thread.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.