Server Quality Checklist

Tool Scores

• send_alertA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses that it sends push notifications and defines priority scale (1-5, default 3). However, it does not detail rate limits, authentication, or side effects beyond the basic action. This is adequate for a simple notification tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise—two sentences. The first states the core action, the second provides usage context and priority info. Every sentence earns its place with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers usage scenarios and priority, but with 6 parameters and no output schema, it could be more complete. Schema descriptions handle parameter details, so this is acceptable. However, the return value or success behavior is not mentioned, leaving some ambiguity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value by explaining priority range and default, but does not elaborate on tags, click, title, or topic beyond what the schema provides. Minimal additional insight.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool sends push notifications via Blipr and provides specific use cases (e.g., long task finished, build broke) that distinguish it from siblings like 'send_critical' or 'ask'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives concrete examples of when to use (long task finished, build broke, need approval, blocked) and mentions priority default. It does not explicitly state when not to use or name alternatives, but the examples are effective.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• send_criticalA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full disclosure burden. It explains that it bypasses silent/Focus under certain conditions and falls back to time-sensitive delivery, which is key behavioral detail. Missing are aspects like confirmation, side effects, or permissions, but core delivery behavior is transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, no fluff, and immediately conveys purpose, restrictions, and a key behavioral nuance. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool (3 params, no output schema), the description covers purpose, usage guidance, and a significant behavioral detail. It does not explain return values or side effects, but for a notification tool, these are often implicit. The presence of sibling tools provides context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and each parameter has a clear description. The tool description adds little beyond the schema (e.g., 'Short title,' 'urgently' for message). The topic parameter's fallback logic is detailed in the schema, so the description does not significantly enhance semantic meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Send a priority-5 critical page,' using a specific verb and resource. It distinguishes from sibling tools like send_alert by emphasizing urgency and criticality, and explains behavior with Apple's Critical Alerts entitlement.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use ONLY for things that genuinely cannot wait (production down, urgent approval, safety),' which frames the appropriate context clearly. It lacks explicit mention of when not to use it or alternatives, but the strong 'ONLY' provides effective guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_replyA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description fully carries the burden. Discloses non-blocking default behavior, immediate return vs long-polling via wait_seconds, return object format, and 30-minute reply retention. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single well-structured paragraph that starts with purpose, then usage, then parameter details, then return value and retention. Every sentence provides necessary information without fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations or output schema, description is thorough. Covers parameters, return format, retention, and usage. Minor gap: no mention of error cases or null fields, but not critical for this simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers all three parameters with descriptions. Description adds value: explains that message_id comes from prior calls, topic defaults to file/env var, and clarifies when to pass topic explicitly. Exceeds baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool checks for replies to a prior ask/request_ack. Distinguishes from siblings by specifying its role as a follow-up to timeouts or cancellations, complementing send_alert, send_critical, ask, and request_ack.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly describes when to use: after a timeout from ask/request_ack or after client cancellation. Provides guidance on interpreting the response for approval gates. Lacks explicit 'when not to use' but context is sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• askA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description fully discloses blocking nature, timeout behavior, return structure, and that approval is only true on Yes. Also mentions persistence of replies for ~30 minutes and cancellation handling. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single well-structured paragraph: first sentence states purpose, then usage guidelines, return format, timeout instructions. No wasted words. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 6 parameters, no output schema, and no annotations, the description covers all necessary information: blocking, return shape, edge cases (timeout, cancellation), and follow-up via check_reply. No gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema already describes all 6 parameters (100% coverage). Description adds context beyond schema: explains default priority (4) reasoning, how topic fallback works, and that message_id is used for later checking. Adds meaningful guidance.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it asks a yes/no question and blocks for answer. Distinguishes from siblings like send_alert (one-way) and check_reply (polling) by emphasizing its role as a blocking approval gate.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly specifies 'use it before doing something consequential or irreversible' and gives concrete examples (deleting prod data, force-pushing). Provides when-not: on timeout, use check_reply instead of re-asking. Warns against treating non-approval as go-ahead.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• request_ackA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses blocking behavior, timeout behavior, return values on success and failure, and retention of replies for ~30 min via check_reply. It also notes potential early cancellation by MCP clients, providing comprehensive behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured paragraph of ~100 words. It front-loads the primary purpose, then provides use cases, return format, and timeout guidance without unnecessary repetition or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description thoroughly explains return values, required parameters, optional parameters with usage scenarios, and fallback behavior (check_reply). It covers all aspects needed for correct invocation, given the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds useful context beyond the schema for the 'topic' parameter (use explicit topic per project) and 'timeout_seconds' parameter (default 120s, cancellation behavior). This incremental value justifies a 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The title and description clearly state the tool sends a message requiring human acknowledgement and blocks until receipt. It specifies the verb (send/request), resource (human acknowledgement), and blocking behavior, differentiating from siblings by emphasizing the blocking and ack requirement.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool (when the human must see and confirm receipt) and provides concrete examples (heads-up, checkpoint, about to start long run). It also mentions the alternative check_reply on timeout, offering clear guidance on when it is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.