Server Quality Checklist

Tool Scores

• comment_channel_postC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden but only says 'publish', implying creation. It does not disclose whether the operation is idempotent, what permissions are needed, error behavior, or any side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that earns its place by clearly naming the action and resource. However, it could be slightly expanded without becoming verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has two required parameters, no output schema, and no annotations, the description is too minimal. It fails to explain the return value, error conditions, or any constraints, leaving the agent under-informed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has two parameters with zero description coverage, and the tool description adds no semantic information about them (e.g., what 'msg' or 'post_id' represent beyond their names).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('publish') and the specific resource ('plain text comment to an existing Direxio channel post'). It distinguishes this from sibling tools like 'list_post_comments' (reading) and 'send_message' (likely direct messaging).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives (e.g., 'send_message'), nor are there any prerequisites mentioned (e.g., the post must exist). The description lacks contextual usage hints.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_messagesC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description must disclose behavior. It only hints at read operation but does not mention pagination, ordering, rate limits, or side effects. The description lacks essential behavioral context for an agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with key elements front-loaded. However, it is too brief for a tool with 4 parameters, missing parameter details that could be included without verbosity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, no annotations, and 4 parameters, the description is insufficient. It does not explain return values, pagination, or the purpose of limit. It barely covers the minimum to understand functionality.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so description must add value. It explains that time range parameters (to_ts, from_ts) are in Unix milliseconds and are optional, but does not mention the limit parameter. Adds some meaning but incomplete.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (list) and resource (ordinary room messages) and specifies key filters (room_id, optional time range). It distinguishes from siblings like list_channel_posts and list_post_comments by specifying 'ordinary room messages' and using room_id, but lacks explicit differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool vs alternatives like search_rooms or list_channel_posts. No mentions of when not to use it or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_channel_postsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits. It does not mention if the operation is read-only, how results are ordered, pagination behavior, or any side effects. The optional time range is noted, but nothing about limits or data size.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, no redundancy. Efficient but could benefit from slightly more structure.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 4 parameters, no output schema, and no annotations, the description is insufficient. It lacks details on return format, ordering, pagination, and any constraints beyond the parameter names.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds minimal value beyond the schema: it clarifies that room_id is a channel identifier and that from_ts/to_ts form a Unix millisecond time range. However, the limit parameter and the exact meaning of the time range (e.g., inclusive/exclusive) are not explained. With 0% schema description coverage, more detail is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List' and the resource 'channel posts', and specifies the key filtering parameters (room_id and optional Unix time range). It distinctly separates from sibling tools like comment_channel_post and list_messages.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives like list_messages or search_rooms. No exclusions or prerequisites mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_roomsB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided; the description only states 'Search or list' suggesting read-only behavior, but does not disclose any other traits like authentication, rate limits, or whether it modifies state. Minimal transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences without unnecessary words. First sentence states purpose, second adds practical guidance. Could be more structured (e.g., listing params) but efficient overall.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of output schema and annotations, the description is too minimal. It does not explain the return format beyond 'room_id', search behavior (e.g., query semantics), or parameter defaults, leaving the agent underinformed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%; the description adds no information about the three parameters (type, limit, query). The schema itself provides defaults and enums, but the description fails to explain their meaning or usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches/lists contacts, groups, and channels, and specifies the returned room_id is used with message tools. It distinguishes from sibling tools like list_channel_posts or send_message.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage via 'Use returned room_id with message tools' but does not explicitly state when to use this tool over alternatives, nor does it provide exclusions or when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_post_commentsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. Only states 'list comments' and mentions time range. Does not disclose safety (read-only), rate limits, error handling, or pagination behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence with 13 words, front-loading the purpose. Efficient but may be too brief, omitting necessary details for effective use.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 4 parameters, 0% schema coverage, and no output schema, the description is incomplete. It omits return format, pagination, error conditions, and usage context beyond basic functionality.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%. Description only mentions post_id and optional time range, failing to explain the 'limit' parameter and lacking details on timestamp format beyond 'Unix millisecond'.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists comments for a channel post, specifying the resource and action. It distinguishes from sibling tools like list_channel_posts (lists posts) and comment_channel_post (adds a comment).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use when needing comments by post_id and optional time range, but does not explicitly state when not to use it or provide alternatives. Lacks guidance on prerequisites or pagination.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• send_messageB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It specifies that only plain text is sent but does not disclose authentication requirements, idempotency, error handling, or what happens if the room_id is invalid. This is insufficient for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, front-loaded, and concise. It efficiently communicates the core action without unnecessary words, though it could include more detail without harming brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that the tool has only two simple string parameters and no output schema, the description provides adequate context for the basic operation. However, it lacks any guidance on success/error responses or retries, which marginal completeness for a simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, meaning no parameter descriptions in the schema. The description only gives context for 'room_id' (private chat, group, channel) but does not elaborate on 'msg' beyond 'plain text', nor any format constraints like minimum length is already in schema. It fails to compensate for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'send', the resource 'plain text message', and specifies the target destinations 'private chat, group, or channel room'. This distinguishes it from sibling tools like 'comment_channel_post' (which comments on a post) and 'list_messages' (which retrieves messages).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for sending plain text only, but it does not explicitly state when to use it versus alternatives like 'comment_channel_post' or when not to use it (e.g., for messages with attachments). No prerequisites or constraints are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.