Server Quality Checklist

Tool Scores

• read_conversationC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the output is the 'most recent messages of a conversation (newest last)' which conveys ordering, but doesn't disclose whether this is a read-only operation, whether account scoping affects results, pagination behavior beyond the limit, or what happens with invalid conversation IDs. Given zero annotations, this is a significant disclosure gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that packs in the key details: 'up to limit most recent messages', conversation scope, and ordering. It's efficient with no filler. However, it front-loads reasonably well and earns its single sentence.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  An output schema exists which reduces the need to explain return values, but the tool deals with a conversation context involving 3 params where 1 (account) is entirely unexplained, no annotations exist, and sibling overlap (recent_messages) creates ambiguity. The single-sentence description is inadequate for an agent to correctly select and invoke this tool among the 7 siblings—particularly around the account parameter and differentiation from recent_messages.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for all three parameters (conversation_id, limit, account). It explains 'limit' explicitly and conversation_id implicitly through 'of a conversation', but 'account' is entirely undocumented—the agent has no idea whether it filters, scopes, or authenticates the read. There's no mention of the conversation_id format or how account and conversation_id interact.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb (return/read) and resource (conversation messages), with 'newest last' ordering, which conveys the operation and result ordering. However, it doesn't explicitly distinguish from sibling tool 'recent_messages' or 'list_conversations'—the agent must infer that 'read_conversation' returns messages within a single conversation whereas 'list_conversations' lists conversation metadata.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There's no guidance on when to use this tool vs. alternatives like search_messages or recent_messages. The description mentions 'limit most recent messages' which partially overlaps with recent_messages, creating possible confusion. No when-not-to-use or alternative references are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_messagesB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. The description mentions it searches 'cached messages' (implying it doesn't hit live sources), but doesn't clarify whether the search is scoped to a single account, what happens with the limit default, or how results are ordered. 'Cached' is a meaningful caveat but under-specified.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, zero waste. The description states the core behavior and the one optional parameter that needs clarification. Efficiently front-loaded with the purpose in the first sentence.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with 4 params, an output schema, and sibling tools, the description is somewhat thin. It names the search semantics and the days constraint but doesn't cover limit/account parameters, and with zero schema coverage and no annotations there are gaps. The cached-messages framing hints at scope limitations but doesn't fully elaborate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for all 4 parameters. The description covers 'days' explicitly and 'query' implicitly (it's the search term), but says nothing about 'limit' or 'account'. The description adds some value for days but leaves half the parameters undocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb+resource ('Case-insensitive substring search across cached messages'), which is clear about what the tool does. However, it doesn't differentiate from siblings like recent_messages or mentions, both of which also deal with message retrieval/search, so the purpose isn't fully distinct.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains the optional days parameter restricting to last N days, which gives some context on when to use it. However, there's no explicit guidance on when to use this vs. recent_messages, unread_messages, or mentions, and the sibling names create ambiguity about which search approach to choose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_conversationsB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden for behavioral disclosure. The description indicates this is a read-only listing operation returning id, title, type. It discloses the account filter behavior (optional). However, it doesn't mention pagination (limit param), ordering, or whether results are complete across all accounts. The return shape is described by the output schema but the listed fields add some context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, front-loaded with the core purpose and return fields. Zero wasted words. Every element is functional.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has a simple 2-param schema with no required params and an output schema present, so the baseline burden is moderate. The description covers the purpose and the account filter adequately for a straightforward listing tool. However, the limit param is undocumented and there's no mention of how filtering or listing behaves across accounts, leaving some gaps for an agent relying solely on this description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for parameter meaning. The description mentions the 'account' filter explicitly and names the return fields (id, title, type). However, it doesn't explain the 'limit' parameter semantics, default behavior, or how account filtering works in practice. Minimal but some value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (list), the resource (chats/channels), and the specific fields returned (id, title, type). It distinguishes itself from siblings by indicating this returns chat/channel metadata, not messages (search_messages, recent_messages) or account list (list_accounts). Somewhat brief but clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. It mentions optional account filtering but doesn't explain how this relates to list_accounts or when a user would need this vs read_conversation. It doesn't exclude scenarios or name alternative tools for specific cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• overviewA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden of behavioral disclosure. It somewhat handles this by specifying the grouping logic ('newest first', 'per conversation') and the output fields. However, it doesn't describe the account parameter behavior, vague 'days' semantics, or any rate/limit considerations. It's a read operation implied by 'activity' but not explicitly stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise — two short sections. The first sentence states the purpose, the second outlines output fields, and the third provides usage guidance. Every sentence earns its place with zero filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  An output schema exists which handles return value explanation, and the description covers the grouping logic, ordering, and its place in the API workflow. Missing parameter semantics is the main gap, but for an overview/aggregation tool with just two params (one optional with defaults), the description is reasonably complete. Sibling differentiation is a minor gap given recent_messages and list_conversations overlap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, and the description provides zero parameter documentation. The 'days' parameter implies a time window but its defaults/format aren't mentioned, and 'account' with null default is completely unexplained. With 2 parameters and no schema descriptions, the tool description should compensate but does not.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific framing ('Who messaged you: recent activity grouped by conversation') with clear output structure (title, message count, distinct senders, last message + time). It clearly states the resource (recent activity) and the grouping (by conversation, newest first). It doesn't explicitly differentiate from siblings like recent_messages or list_conversations, though the grouping by conversation with sender stats is fairly distinct.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states this is a 'Great first call to get the lay of the land before drilling in', providing clear guidance on when to use it (as an entry point before deeper tools). It doesn't mention specific alternatives or when NOT to use it, but the suggested workflow context is actionable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_accountsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that this reads a 'local cache' and returns best-effort data ('label inferred from org names'), which is useful behavioral context about data quality. However, it doesn't discuss privacy/caching nuances, whether results might be stale, or what happens when the cache is empty.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and efficient — two short sentences. The first states the purpose clearly and the second explains the output fields. Zero waste, well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 0 parameters and a rich output schema that this description complements by explaining field semantics, this is a simple tool. The description covers purpose, output fields, and data quality caveats. It fully suffices for what is effectively a parameterless list operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are 0 parameters and schema coverage is 100% (trivially, as there are no properties). The description adds value by explaining what each field in the output means (key for filtering, tenant_id, user_id, labeled inference), which is effectively parameter/output guidance given there are no actual parameters to document.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'List the Teams accounts (tenant/user contexts) in the local cache' with a specific verb and resource. It distinguishes from siblings (list_conversations, recent_messages, etc.) by specifying accounts rather than messages/conversations. However, it could more explicitly contrast with sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context (listing cached accounts to get keys for filtering) but doesn't explicitly state when to use vs not use alternatives. It notes the key is used as an 'account' filter elsewhere, which hints at usage, but there's no explicit when/when-not guidance or named alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• unread_messagesA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses a critical limitation: the local Teams cache has NO general read marker, so true 'all unread' can't be derived. This is genuinely valuable behavioral information that prevents the agent from incorrectly interpreting results. It could add return-format details, but the core limitation disclosure is strong.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose statement and then adds a well-structured limitation warning with a clear alternative. It's moderately long due to the credential caveat, but every sentence earns its place — the limitation warning is essential and the alternative routing prevents misuse. Slightly verbose in formatting but efficient in content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is conceptually complex (subject to cache limitations, needs disambiguation from 'all unread' assumptions and from the recently-arrived interpretation). An output schema exists but its structure is not examined here. The description adequately covers the key semantic trap (per-message read state not stored on disk) and the routing decision. Given the complexity of what could go wrong, this is well-covered.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description carries all parameter burden. However, the description does not explain any of the three parameters (days, limit, account). 'days' semantics can be inferred from the recent_messages contrast, but limit and account are entirely undocumented. Mean-while, the parameter names are relatively self-explanatory, so a baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states what the tool returns: 'unread @-mentions'. The verb 'unread items needing attention' combined with the explicit mention of @-mentions gives a clear purpose. It distinguishes from sibling tools by contrasting with recent_messages, though it ties the purpose to the limitation explanation rather than stating it crisply up front.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  This is exemplary. The description explicitly explains when NOT to use this tool and names the exact alternative: 'For everything that arrived while I was away, use recent_messages(days=...) instead'. It even explains the reasoning (you read nothing while away → recent ≈ unread). This is the highest quality usage guidance possible.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• recent_messagesA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses the single-call efficiency, time-window scoping, and ordering, but doesn't mention pagination behavior, whether it counts as a read (safe) operation, rate limits, or what happens when limit is hit. Adequate but not rich for a read tool with no annotation support.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two compact paragraphs, front-loaded with the core purpose. Every sentence adds information — time window, ordering, single-call efficiency, use-case framing, and cross-reference to a sibling tool. No waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  There's an output schema present, which relieves the description of explaining return values. The tool is a straightforward filtered-list retrieval with 3 optional params, and the description covers purpose, usage, and two of three params. Could add limit/pagination semantics, but overall complete enough for this tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains `days` (time window) and `account` (with reference to list_accounts for valid values), but doesn't clarify `limit` beyond its name/schema default. Partial credit given; the `days` and `account` semantics are meaningfully expanded.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb+resource+scope: lists messages received in the last `days` days, newest first, in a single call. It clearly distinguishes itself from siblings like search_messages and mentions by specifying the temporal 'recent' scope and the 'newest first' ordering.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit use-case guidance is given ('Ideal for what did I get while I was away'), and it even references sibling tool list_accounts for how to scope by account, implicitly distinguishing itself from account-level operations. However it doesn't explicitly state when NOT to use it (e.g., vs search_messages), though the ideal-use framing is strong.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mentionsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so description carries the burden. It discloses key behaviors: returns mentions with content, ordering is newest first, and notably 'this is the only reliable read/unread signal in the local store' — a valuable caveat. It doesn't describe pagination, limit behavior, or error conditions, but core behavioral traits are disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two compact paragraphs, zero filler. First sentence establishes purpose and ordering, second paragraph covers parameter semantics efficiently. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there's an output schema (so return format doesn't need explanation), no required params, and a modest 4-parameter surface, the description covers the essential ground. It could note limit/pagination semantics and the absence of content filtering, but for a read tool with an output schema this is well-rounded.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, and the description compensates well by explaining all three optional parameters (unread_only, days, account) in plain language. It even cross-references list_accounts for the account scoping. It doesn't touch on the 'limit' parameter, but given it has a sensible default (100), that gap is minor.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description is specific: '@-mentions of you (someone @mentioned you), newest first, with content.' It names the exact verb+resource scope and includes ordering (newest first) and what to expect (content). Distinguishes clearly from siblings like search_messages, recent_messages, and unread_messages.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description explains what each option does (unread_only, days, account) in the context paragraph. It notes this is 'the only reliable read/unread signal in the local store,' which positions it against the sibling unread_messages tool. However, it doesn't explicitly state when NOT to use it or name alternative tools for when criteria differ.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.