Server Quality Checklist

Tool Scores

• get_conversation_contextA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already establish readOnlyHint, openWorldHint, idempotentHint, and non-destructive behavior. The description adds the bundling behavior and orientation use case, but it does not disclose additional operational details such as rate limits, data freshness, or response-size implications. With such robust annotations, the description's extra behavioral context is adequate but not extensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each earning its place: the first states scope, the second states intended use, the third routes to the alternative. The most important information is front-loaded and there is no wasted wording.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with only three parameters, an output schema, and strong annotations, the description is complete. It tells the agent what the tool returns, when to use it, and which sibling handles the cases this tool does not.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with each parameter already documented: group_id identifies the source, response_format explains concise vs detailed, and recent_message_count gives the range and default. The tool description adds no parameter-level meaning beyond the schema, so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action and resource: 'Get one group's metadata, member list, and recent messages in one call.' It clearly distinguishes this from siblings by emphasizing the bundled context retrieval, and it is not a tautology.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says when to use it: 'to orient yourself in a group before reading further or replying.' It also names the alternative directly: 'For direct-message chats or for paging deeper into history, use read_messages instead.' This is strong routing guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_conversationsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations (readOnlyHint, idempotentHint, etc.), the description adds meaningful behavioral detail: recency ordering, merging of groups and DMs, inclusion of 'last-message previews and member counts,' and the presence of the IDs that downstream tools require. This goes beyond simply echoing the read-only safety profile.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three tight sentences: the first states the core purpose, the second gives direct usage guidance, and the third explains the output's value for sibling tools. Every sentence earns its place with no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the rich input schema (100% parameter coverage), full annotations, and presence of an output schema, the description is complete for an agent to select and invoke the tool correctly. It explains the tool's role, output characteristics, and how the returned IDs connect to other tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters. The description adds contextual color like 'merges groups and direct-message chats' and 'last-message previews and member counts,' but it does not substantially enrich the parameter semantics beyond what the schema already states.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific action and resource: 'List the user's GroupMe conversations, most recently active first.' It also distinguishes itself from siblings by explaining that it 'merges groups and direct-message chats into one recency-sorted list' and provides the IDs that other tools need.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use it: 'Use this first, whenever you need to find a conversation or the ids the other tools take.' It references sibling tools that consume its output ('read_messages and get_conversation_context'), though it does not explicitly contrast it with search_messages or other siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_highlightsA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark the tool as read-only and idempotent, and the description adds valuable behavioral context: it wraps the likes leaderboard, limits the member summary to leaderboard messages rather than full history, and explicitly discloses that the underlying endpoint is undocumented and may fail with guidance if retired. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four focused sentences: the purpose, the use case, a key scope limitation, and a failure-mode warning. Every sentence earns its place, and the most important purpose is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only summary tool with full schema coverage, strong annotations, and an output schema, the description is complete. It explains what results look like, notes the scope limitation, and warns about the undocumented dependency.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides 100% parameter coverage with descriptions for group_id, period, and response_format. The description adds general framing around period-based top-liked messages and output content, but it does not meaningfully extend the schema's parameter-level explanations. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The opening line states a specific verb and resource: 'Summarize what mattered in a group: its most-liked recent messages.' It clearly differentiates from siblings by focusing on highlights via GroupMe's likes leaderboard rather than reading raw messages or sending messages.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for when to use the tool: 'Use this to catch up on a busy group without reading everything.' It implies a contrast with full-message reading tools like read_messages, though it does not explicitly name alternatives or state when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_messagesA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations, the description discloses that results are ordered oldest-first, sender names are resolved, attachments are normalized, and an empty page is a normal response rather than an error. These are important behavioral traits an agent would not infer from readOnlyHint, openWorldHint, idempotentHint, or destructiveHint.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four sentences with each earning its place: the first states the core action, the second gives usage context and a prerequisite, the third summarizes response normalization, and the fourth prevents a common misinterpretation. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Combined with an output schema, the rich input schema, and annotations, the description covers what is needed to select and invoke the tool: scope, ordering, pagination expectations, normal edge-case behavior, and prerequisite data. An agent can confidently call this tool with just the description plus schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all five parameters in detail, including pagination semantics, the oneOf conversation format, and the response_format enum. The description adds context about oldest-first ordering and paging but does not need to repeat param-level details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb and resource: reading messages from one GroupMe conversation, oldest first. It also implicitly distinguishes itself from list_conversations by telling the agent to obtain the conversation id from that sibling tool first, and from send_message by focusing on reading. This is a clear, usable purpose statement.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells the agent when to use this tool: to read or page through a specific group or direct-message chat once the id is known from list_conversations. It does not explicitly discuss when not to use it versus search_messages or get_conversation_context, so it stops short of full alternative routing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• send_messageA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses non-idempotence ('Each call sends a new message — calling twice posts twice'), which is critical and goes beyond the annotations' idempotentHint=false. It also reveals authenticated-user posting context and that replies render threaded in clients, adding genuine behavioral insight.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three dense sentences each earn their place: purpose, usage/prerequisite/non-idempotence, and reply mechanism. The key constraint is front-loaded and there is no redundant or vague language.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the full schema, output schema, and annotations, the description supplies the missing orchestration details: where ids come from, how replies behave, and that duplicate calls produce duplicates. An agent has everything needed to invoke this tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds value by telling the agent where parameter values come from (conversation ids from list_conversations, reply id from read_messages), which enriches the schema's per-field descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The first sentence states a specific verb and resource: 'Send a message to a GroupMe group or direct-message chat.' It also clarifies that it posts as the authenticated user, which disambiguates it from read/search siblings despite not naming them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says to use this 'once you know where to send' and points to list_conversations for ids, giving a clear prerequisite. It also explains the reply flow via read_messages. It stops short of explicitly saying when not to use it or naming react_to_message as the alternative for reactions, so it misses a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• react_to_messageA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Even though idempotentHint is already present in annotations, the description explains the behavioral consequence concretely: liking an already-liked message or unliking one never liked leaves it in the requested state. It also adds that the action is performed on the authenticated user's behalf and clarifies the meaning of conversation_id values. This goes well beyond the annotation and schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded: it states the action first, then usage context, then idempotence, then ID source. Every sentence earns its place with no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a small, three-parameter mutation tool with an output schema and strong annotations, the description covers what the tool does, how to use it correctly, where the required IDs come from, and its idempotent behavior. Nothing essential for correct invocation is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, but the description adds valuable meaning beyond the schema: conversation_id can be a group id or a composite direct-chat id like '123+456', and both IDs come from read_messages with response_format='detailed'. This is critical guidance an agent would not get from the generic schema property descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a precise verb and resource: 'Like a GroupMe message, or remove your like from one.' It clearly states the operation and distinguishes it from sending a reply ('without posting a reply'), making it easy for an agent to know what this tool does relative to send_message.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives an explicit 'Use this to' clause and frames it for acknowledgement scenarios rather than replying, which differentiates it from the sibling send_message. It also tells the agent exactly where to obtain IDs (read_messages with response_format='detailed'), though it does not explicitly name alternatives or say when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_messagesA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and idempotentHint=true, and the description adds substantial behavioral detail beyond that: it scans backwards, matches case-insensitive substrings, stops under three named conditions, and tells the agent to check oldest_message_reached and note before concluding absence. This is rich, actionable transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences with no wasted words. The main purpose is front-loaded, the usage guideline follows, and the operational caveat about scan depth is placed at the end where it is most useful as a caution.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 6-parameter search tool with 100% schema coverage, rich annotations, and an output schema, the description is complete. It covers the algorithm, stop conditions, case-insensitivity, the read_messages alternative, and the critical caveat about interpreting scan limits.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds value by explaining how the parameters interact: query matches message text, sender_name matches sender display names, both are case-insensitive substrings, and limit/max_messages_scanned act as stop conditions during the backward scan. This goes beyond the schema's individual field descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Search one conversation's message history for matching messages.' It clearly distinguishes the tool from read_messages by framing it as the targeted alternative to manual paging, so an agent can tell them apart immediately.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says when to use this tool — for finding specific messages like 'who mentioned pizza?' — and names the alternative read_messages for manual paging. This gives the agent a clear routing decision with a concrete condition.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.