Server Quality Checklist

Tool Scores

• list_chatsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false, so safety profile is provided. The description adds the cap on limit (200), offering one behavioral constraint, but does not elaborate on ordering or pagination. Barring annotations, this is sufficient but minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences convey purpose, return details, usage hint, and a limit constraint with no wasted words. The information is front-loaded and efficiently delivered.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (one optional param, read-only) and an output schema exists, so the description doesn't need to detail return formatting. It covers the key workflow (list chats -> get messages) and limit cap, making it sufficiently complete for the low complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides only the integer type and default for limit, with 0% coverage. The description compensates by noting the cap at 200, giving practical meaning beyond the schema. However, it doesn't explain the exact effect of varying limit, but this is largely evident.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'List the most recently active conversations' with a specific verb and resource. It also specifies return content (identifier, display name, last message time), distinguishing it from sibling message-retrieval tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions using the returned identifiers with get_messages or send_to_chat, which provides follow-up usage context. However, it does not explicitly state when to use this tool versus alternatives like get_recent_messages, and omits any exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_messagesA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations (readOnlyHint=true, destructiveHint=false) already signal a safe read operation, and the description adds materially useful behavioral context beyond them: the literal handling of `%` and `_`, the attributedBody gap, and the hard 200-row limit on `limit`. The security advisory about treating message text as data rather than instructions is an exemplary disclosure for an LLM-driven agent. No contradiction with the safe-read annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Five sentences, three short paragraphs, and every clause pulls its weight. Purpose, matching rules, known coverage limitation, limit cap, and a security note — all under 70 words. The structure front-loads the core definition and parcels out caveats in decreasing importance. The final warning about prompt injection is a distinctive, high-value addition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only search tool with an output schema, this description is thorough: it documents the matching algorithm, data availability, and edge-case limitations, and even warns about hostile content. One minor gap is that it never addresses case-sensitivity or namespace/regex behavior, which could plausibly matter when using a search. But given the presence of annotations and output schema, coverage is strong overall.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description carries the full burden for parameter documentation. It compensates partially by explaining the `limit` cap and the literal-match semantics of the query string, but it doesn't add much beyond what the parameter names alone suggest for a 2-parameter tool. The value it does add (wildcard literalness, cap) is meaningful, but the semantic depth is still minimal and some behavior (e.g., case sensitivity) remains unspecified.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a crisp, specific statement: 'Substring search over message bodies.' This clearly establishes verb (search) and resource (message bodies) and distinguishes it from sibling tools like get_messages and get_recent_messages. Subsequent sentences add behavioral specificity (plain text column, literal wildcards) without ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear contextual guidance about when the tool is effective versus when it falls short ('Some newer messages store their body only in attributedBody and will not be found by this search'), which implicitly tells the agent when to consider alternatives like get_messages. However, it never explicitly names sibling alternatives or says 'use X instead,' so it stops short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_recent_messagesA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnlyHint annotation, the description adds meaningful behavioral context: results include chat_identifier for attributing self-sent messages, limit is capped at 200, and message text should be treated as untrusted data rather than instructions. This is rich, non-obvious context that helps the agent handle output safely.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded: the first sentence states the core purpose, the second adds a critical attribution detail, and the third provides a security-oriented warning. Every sentence earns its place with no repetition of schema or annotation information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a simple tool with one optional parameter and an output schema, so the description does not need to explain return values. It covers scope, ordering, limit cap, attribution semantics, and untrusted-content handling, making it complete for safe and correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description compensates by clarifying the cap on limit (200). It does not restate the obvious meaning of 'limit' as a count, but the cap is an important addition. Given the parameter is self-explanatory and the default is in the schema, this is adequate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Get') and clearly identifies the resource ('most recent messages across all conversations') plus ordering ('newest first'). It distinguishes itself from sibling tools by emphasizing the cross-conversation scope, which differs from likely per-chat retrieval or search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly conveys when to use this tool: when you need recent messages across all conversations, and it notes the chat_identifier attribution caveat. It does not explicitly name alternatives like get_messages for a specific chat, but the cross-conversation phrasing provides clear context and implies the exclusion.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_messagesA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds significant behavioral detail beyond the annotations: it states tapbacks are excluded, limit is capped at 200, and includes a security note about treating message text as untrusted data. These are not implied by readOnlyHint or destructiveHint, so the description fully carries the burden of behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is structured in three tight paragraphs: purpose, parameter details, and a security reminder. Every sentence earns its place—no fluff, no repetition, and the most important info (purpose) is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 params, output schema present), the description is remarkably complete. It covers parameter formats, ordering, exclusions, limits, and even a security consideration. Nothing critical is missing for an agent to use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by explaining the exact accepted formats for 'chat' (phone/email or guid) and the pragmatic limit of 'limit' (capped at 200). It clarifies both parameters in a way the schema alone does not, adding real semantic value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Get recent messages from a conversation'—a specific verb+resource. It further specifies the chat parameter types (chat_identifier or chat guid), ordering (newest first), and exclusions (tapbacks), which fully conveys the tool's scope and differentiates it from siblings like list_chats and search_messages.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a clear usage context—how to specify the chat (phone/email for 1:1, chat guid for groups) and notes that guid comes from list_chats. However, it does not explicitly state when to choose this tool over get_recent_messages or search_messages, missing the 'when not to use' guidance present in higher-scoring examples.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.