Server Quality Checklist

Tool Scores

• get_recent_slack_messagesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'fetch' but doesn't specify whether this is a read-only operation, requires authentication, has rate limits, or how it handles errors. This leaves significant gaps in understanding the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no wasted words, making it highly concise and front-loaded. It efficiently communicates the core purpose without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (2 parameters, no annotations, but with an output schema), the description is minimally adequate. The output schema likely covers return values, reducing the need for description details, but the description fails to address key behavioral aspects like permissions or error handling, leaving it incomplete for safe use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the schema provides no parameter details. The description adds minimal semantics by implying 'channel_id' and 'limit' are used to fetch messages, but it doesn't explain what 'recent' means, the format of 'channel_id', or default behavior for 'limit' (though the schema shows a default of 5). This insufficiently compensates for the lack of schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('fetch') and resource ('recent messages from a Slack channel'), making the purpose understandable. However, it doesn't differentiate from sibling tools like 'get_user_messages' which might also retrieve messages, leaving room for ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives such as 'get_user_messages' or 'post_message'. The description lacks context about prerequisites, constraints, or typical use cases, offering minimal direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• post_messageC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states the action ('Post') but doesn't mention any behavioral traits like required permissions, rate limits, whether the message is editable, or what happens on success/failure. This leaves significant gaps for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's function without any wasted words. It's appropriately sized and front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given this is a mutation tool with no annotations, 0% schema coverage, but an output schema exists, the description is minimally adequate. The output schema may cover return values, but the description lacks context about permissions, error handling, or usage scenarios, leaving it incomplete for safe operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate but adds no parameter information. It doesn't explain what 'channel_id' represents (e.g., format like C123456) or what 'text' can contain (e.g., markdown, emojis, attachments). This leaves both parameters semantically unclear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Post a message') and target resource ('to a Slack channel'), making the purpose immediately understandable. However, it doesn't differentiate this from sibling tools like 'get_recent_slack_messages' or 'get_user_messages', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There are sibling tools for retrieving messages, but no indication of when posting is appropriate versus reading, or any prerequisites like channel access permissions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_user_infoB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions that the tool retrieves user info but doesn't specify what info is returned (e.g., profile details, status), whether it requires authentication, rate limits, or error handling. This leaves significant gaps in understanding the tool's behavior beyond basic functionality.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with the main purpose stated clearly in the first sentence. The additional parameter explanation is concise and adds value without redundancy. However, the formatting with quotes and line breaks slightly detracts from optimal structure.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that there's an output schema (which should cover return values), the description doesn't need to explain outputs. However, for a tool with no annotations and only basic parameter info, it lacks details on behavioral aspects like authentication or error cases. It's adequate but has clear gaps in providing a complete context for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful context for the single parameter: 'username: Slack display name or real name (not user ID).' This clarifies the parameter's purpose and constraints beyond the schema, which has 0% description coverage. Since there's only one parameter and the description compensates well, it earns a high score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get Slack user info by username or real name.' It specifies the verb ('Get'), resource ('Slack user info'), and scope ('by username or real name'), making it easy to understand what the tool does. However, it doesn't explicitly distinguish this tool from its sibling tools (e.g., get_user_messages), which slightly limits its differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by specifying 'by username or real name' and noting that it's 'not user ID,' which helps clarify when to use this tool versus alternatives that might use IDs. However, it doesn't explicitly state when to use this tool over siblings like get_user_messages or provide clear exclusions, leaving some ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_user_messagesB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden but only states basic functionality. It doesn't disclose behavioral traits like permissions needed, rate limits, pagination, error handling, or whether it's read-only (implied by 'Fetch' but not explicit). This is inadequate for a tool with potential Slack API constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, followed by a concise Args section. Every sentence adds value: the first defines the tool, and the param explanations are necessary. No redundant or verbose content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 3 parameters, no annotations, and an output schema (which handles return values), the description is minimally adequate. It covers purpose and param semantics but lacks behavioral context (e.g., Slack API nuances). For a tool interacting with external services, more completeness is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful context beyond the schema: it explains 'username' includes 'real name' (not just username), 'limit' checks messages (not fetches exactly that many), and clarifies the purpose of each param. With 0% schema description coverage, this compensates well, though 'channel_id' semantics could be more detailed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Fetch messages') and target ('by a specific user from a Slack channel'), specifying the resource (messages) and scope (user, channel). It distinguishes from siblings like 'get_recent_slack_messages' (general messages) and 'post_message' (sending), but not explicitly from 'get_user_info' (user metadata).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when needing messages from a specific user in a channel, but lacks explicit guidance on when to use this vs. 'get_recent_slack_messages' (e.g., for filtering) or 'get_user_info' (for user details). No exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.