Server Quality Checklist

Tool Scores

• set-ntfy-topicB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description should disclose behavioral traits. It only mentions 'no restart needed', but does not explain implications of changing topic mid-session, whether the change is persistent, or what happens to ongoing notifications. Lacks detail on side effects and conditions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded with the core action. However, it could be structured to include more detail without being wasteful; currently it is slightly under-specified for a 2-parameter tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is adequate for a simple tool but fails to address the baseUrl parameter. Completeness is acceptable for a minimal tool but has a clear gap given the presence of an optional parameter.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains the 'topic' parameter implicitly but does not mention the 'baseUrl' parameter at all, leaving its purpose unclear for the agent.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Change') and resource ('ntfy topic'), specifying it is for the session and does not require restart. This distinguishes it from siblings send-ntfy and wait-and-read-inbox, which are about message sending and receiving.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context (changing topic in session) but does not explicitly state when to use compared to siblings or when not to use it. 'No restart needed' hints at immediate effect but lacks comprehensive guidance on appropriate scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• send-ntfyA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavior. It only describes the action as 'publish' without mentioning side effects, idempotency, permission requirements, or rate limits. The description lacks transparency on common behavioral aspects for a send operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of two concise sentences with essential information front-loaded. The first sentence establishes the core purpose, and the second provides a useful cross-reference. No superfluous content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists, return values need not be explained. The description covers the basic operation but omits potential constraints (e.g., message length, priority semantics) and error scenarios. It assumes the topic is configured in mcp.json without discussing what happens if it's not. Completeness is adequate but not thorough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description partially compensates by listing all optional parameters and noting priority range (1-5). However, it does not explain the purpose of tags or attachUrl, nor does it provide formats or examples, leaving some semantics unclear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Publish a message to the configured ntfy topic', specifying the verb and resource. It lists supported optional fields (title, priority, tags, attachUrl). This differentiates it from siblings set-ntfy-topic (configuration) and wait-and-read-inbox (response handling).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises to use wait-and-read-inbox after sending, providing sequential guidance. It does not discuss when not to use this tool or prerequisites, but the context is clear: it is for sending a message to a preconfigured topic.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• wait-and-read-inboxA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses behavioral traits: it waits indefinitely (subject to timeout), uses existing subscription, and returns only on new message. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences, front-loaded with the core function, each providing essential information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's waiting behavior and 3 parameters, the description covers the key aspects including timeout and return condition. Output schema exists, so return values are covered.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description does not add extra meaning beyond what the input schema already describes for each parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (wait for new messages) and resource (inbox on configured topic), and distinguishes from siblings (send-ntfy, set-ntfy-topic) which are different tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains that the tool blocks until a new message arrives and notes the client-side timeout, providing clear usage context. However, it does not explicitly state when not to use it or compare with siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.