Server Quality Checklist

Tool Scores

• get_conversation_contextC
  2.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits. It mentions that contact names are preferred for direct chats and group titles for groups, which gives a clue about output formatting. However, it does not state that the operation is read-only, whether it requires authentication, or what the 'context' includes. Critical behavioral information is missing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise at two sentences, with the first stating the purpose and the second adding a relevant detail (naming preference). The structure is clear and front-loaded, but it could include critical parameter information without becoming overly long.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has two parameters, no schema documentation, and no annotations, the description is insufficiently complete. It does not explain what the output contains (though an output schema exists, per rules it's not required in description). Parameter semantics and behavioral context are missing, leaving significant gaps for an agent to use the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has two parameters (chat, limit) with no descriptions (0% schema coverage). The tool description does not explain what 'chat' represents or what 'limit' controls. This leaves the agent guessing about parameter meanings, making correct invocation difficult.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Return recent conversation context for a chat,' which identifies a specific operation and resource. However, 'context' is somewhat vague; it could mean recent messages, participants, or other metadata. It distinguishes itself from sibling tools like get_unread_messages and list_chats, but the exact scope is unclear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives (e.g., get_unread_messages, list_chats). It does not mention prerequisites, limitations, or scenarios where it is preferred. The only hint is the naming preference, but that is not about usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mark_readC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description only mentions 'best-effort' which hints at unreliable behavior, but does not elaborate on failure conditions, side effects, or required permissions. With no annotations provided, the description carries full burden but falls short.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very brief (one sentence), which is concise but at the cost of missing critical details. It could be expanded slightly to cover parameter guidance and usage without becoming verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite low tool complexity (one required param, no nested objects), the description lacks completeness. It does not explain return values, behavior under failure, or how the 'best-effort' nature impacts the agent's decision. The existence of an output schema is mentioned in context signals, so return values are partially covered, but the description adds little.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter 'chat' is described only by its schema (type string, required). The description adds no information about format, source, or meaning beyond the schema. Given 0% schema description coverage, the tool definition provides no help for agents to construct valid input.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('mark read') and the resource ('chat'), and includes the scope ('through macOS Messages'). It implicitly distinguishes itself from siblings like get_unread_messages or send_message by specifying a mutation action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use or not use this tool, no mention of prerequisites or alternatives. The 'best-effort' qualifier hints at limitations but doesn't clarify when it might fail or suggest other tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• send_messageC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, description carries full burden. Mentions dry_run default for safety, but does not disclose that sending is a write/destructive operation, failure modes, or any side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no waste, but under-specified. Concise at the expense of completeness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Lacks details on chat format, output behavior, prerequisites, or error conditions. Output schema exists but not utilized in description. Inadequate for a write tool with 3 parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so description must add meaning. Only mentions dry_run default; chat and text are not explained. 'Chat' is ambiguous (ID vs name). Poor compensation for missing schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states verb 'Send', resource 'message', and context 'to a chat through macOS Messages'. Distinguishes from siblings which are read/list/mark operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. Only mentions dry_run default for safety, but does not explain when to switch dry_run off or distinguish from sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_unread_messagesB
  3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses the fallback behavior when chat is empty, which is useful. However, it omits other important behaviors such as whether the action is read-only, authentication requirements, or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short (two sentences) and front-loaded with the main action. It is concise, but at the expense of missing necessary details; however, the structure is clear and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema, the description is too brief for a tool with two parameters and sibling tools. It does not explain parameter usage or when to choose this tool over others, leaving gaps for the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain the two parameters (chat and limit) at all. It adds no meaning beyond the schema's default values, leaving the agent unaware of parameter purpose or format.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns unread incoming messages, with a specific verb and resource. It also adds a nuance about fallback behavior when chat is empty, which helps distinguish it from sibling tools like list_chats and send_message.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives (e.g., get_conversation_context for full history or mark_read for marking). The description lacks explicit usage context or conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_chatsB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. Only states it lists 'recent' chats with filtering; no disclosure of side effects, privacy, or ordering. Lacks behavioral traits like read-only assurance.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence with no redundancy. Front-loads purpose and filtering capability. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Minimal description for a simple tool. Lacks details on ordering, default limit meaning, or pagination. Output schema exists but description could still clarify 'recent' timeframe. Adequate but not rich.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% description coverage. Description explains query parameter (filter by name/phone/email/group) but does not add meaning to limit parameter (only default given). Partially compensates for one parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'list recent Messages chats' with optional filtering, distinguishing it from sibling tools like get_conversation_context which focuses on a single chat's details. Verb (list) and resource (chats) are specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus siblings. Implicitly it's for listing chats, but no explicit when-not or alternatives mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.