Server Quality Checklist

Tool Scores

• dry_runB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It accurately notes that no dispatch occurs, but does not disclose any other behavioral aspects like simulation behavior or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise and front-loaded, with no wasted words. However, it borders on under-specification.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given three parameters, a missing output schema explanation, and five sibling tools, the description is insufficient to fully guide the agent without additional context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds no information about the parameters (body, title, severity) despite 0% schema description coverage, leaving the agent to guess their meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states what the tool does: it shows the output of `notify()` without actually dispatching, distinguishing it from the actual notify tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage as a preview mechanism, but it does not explicitly state when to use or when not to use it, nor does it mention alternatives beyond the sibling tool name.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• notify_oneA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavior. It states that notifications are not persisted, which is a key behavioral trait. However, it does not mention error handling, rate limits, or whether the tool is synchronous. The existence of an output schema suggests a return value, but the description omits any discussion of results.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loading the core purpose in the first sentence and usage guidance in the second. Every word adds value, with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple and the description covers purpose and usage context. However, it omits parameter semantics (despite low schema coverage) and does not explain the output format, even though an output schema exists. For a tool with 4 parameters and no schema descriptions, the description is incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, and the description does not explain individual parameters. It only loosely associates 'ad-hoc Apprise URL' with the 'url' parameter, but provides no details on formats, allowed values for 'severity', or the role of 'body' and 'title'. The description fails to compensate for the missing schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool sends a notification to a single ad-hoc Apprise URL and notes that it is not persisted. This distinguishes it from siblings like 'notify' (persisted channels) and 'dry_run' (testing), providing a specific verb-resource-scope combination.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Useful for one-off pages where the channel isn't in `APPRISE_URLS`', giving clear context for when to use this tool. It implies the alternative 'notify' for persisted channels, though it does not name the sibling directly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• health_checkA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description mentions self-check and inventory, implying read-only behavior, but does not explicitly state side effects or what exactly is checked. With no annotations, more detail would be beneficial.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise, two aspects covered in one short sentence with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While output schema exists and no input params, the description does not explain how health_check differs from list_configured_channels or what 'inventory' entails. Slightly incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, so schema coverage is 100%. The description does not need to add param info beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Server self-check + channel inventory' clearly states the tool performs a health check and lists channels. It is distinct from sibling tools that focus on listing, notifying, or dry runs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use or avoid. The purpose is implied but lacks details about prerequisites or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• notifyA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, placing the full burden on description. The description states it sends a notification, but omits potential side effects (e.g., external calls, rate limits, persistence). It is adequate but lacks depth for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence is very concise, but underspecified. Could include more context (e.g., that it dispatches to all channels or uses defaults). Still, it is well-structured and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity and presence of output schema, the description covers the essential purpose. It is complete enough for an agent to understand the action, though slightly more context about behavior would improve it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description does not add parameter information beyond the schema, which is sufficient. No additional semantic value is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states verb 'Send' and resource 'notification to all configured channels'. It distinguishes from sibling tool 'notify_one' by specifying 'all configured channels'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for broadcasting to all channels, but does not explicitly state when to use this tool versus alternatives like 'notify_one' or 'dry_run'. However, sibling names and context make the use case reasonably clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_configured_channelsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses that credentials are redacted, which is an important behavioral detail. However, it does not mention ordering, pagination, or whether the list is complete, but given the simplicity, this is acceptable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that conveys the essential information without any waste. It is appropriately sized and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters, an existing output schema, and distinct sibling tools, the description is fully sufficient. It explains the output content (channels with redacted credentials) and the tool's purpose, leaving no gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters, and schema coverage is 100%, so the description does not need to add parameter information. The baseline for 0 parameters is 4, and the description adds no extra semantics, which is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists channels and includes a key detail about credentials being redacted. The verb 'list' and resource 'channels' are precise, and it naturally distinguishes from sibling tools like notify and health_check which perform different actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by stating the action, but does not explicitly guide when to use this tool over alternatives or mention any preconditions. Since siblings have distinct purposes (e.g., notifying, health checking), the context is clear but not explicitly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.