Server Quality Checklist

Tool Scores

• get_accountA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already establish read-only, idempotent, non-destructive behavior. The description adds a useful behavioral disclosure beyond annotations: access token values are redacted and only labels/timestamps are returned. This gives the agent accurate expectations about sensitive data handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences convey scope, contents, and redaction behavior with no filler. The most important field list is front-loaded, and the security note earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a parameterless read-only account lookup with an output schema and rich annotations, the description covers what the tool returns and how sensitive tokens appear. No critical information is missing for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and schema coverage is effectively 100%, so there is no parameter ambiguity to resolve. Per the baseline for parameterless tools, no further description is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description enumerates the resource ("configured credentials") and the exact data returned (identity, role, tier, limits, usage), making the purpose clear. It lacks an explicit verb like "retrieve" and does not explicitly contrast with siblings, so it falls just short of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The intended use is implied: an agent needing account/credential details for the current configuration would call this. There is no explicit when-to-use guidance or comparison with alternatives such as get_server_info, so the guidance is only implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_messageA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds meaningful behavioral context beyond those annotations: the message is cached and returns the untruncated body, action buttons, and attachments. This gives the agent a fuller picture of what a successful call provides.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, each earning its place: the first states the core operation and return details, the second tells the agent where valid IDs come from. There is no filler, repetition, or unnecessary explanation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a low-complexity read tool with an output schema, full schema parameter coverage, and safety annotations, the description is largely complete. It could mention error behavior or the cached nature's implications, but those are minor gaps given the available structured metadata.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both id and topic including the topic default. The description adds that IDs originate from list_messages or publish_message, which is helpful for sourcing the parameter but does not change the semantic meaning of either parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Fetches a single cached message in full.' It adds what 'full' means—untruncated body, action buttons, and any attachment—so the agent knows exactly what this tool returns and can distinguish it from list_messages or get_account.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear invocation context: it is for fetching one message by ID, and it explicitly says IDs come from list_messages or the result of publish_message. It does not explicitly state when not to use it, but the sibling names and this provenance guidance make the intended usage clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_userA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare destructiveHint=true, readOnlyHint=false, and idempotentHint=true. The description adds meaningful context beyond these by disclosing the two-step confirmation flow, the admin requirement, and the cascading removal of access grants. This goes beyond what annotations alone convey.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no filler. The first sentence states the core effect, and the second explains the required prerequisite and two-call sequence. Highly efficient and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive tool, the description covers the critical operational facts: admin requirement, confirmation token workflow, and the cascading effect on access grants. An output schema exists, so return values are covered. Nothing essential is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with both username and confirm_token already documented. The description reinforces the confirm_token workflow but does not add new parameter-level details beyond what the schema and call flow already imply, matching the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Removes'), a clear resource ('an account'), and the exact scope ('every access grant attached to it'). This distinguishes it unambiguously from sibling tools like delete_messages or manage_user_access.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit usage guidance: an admin account is required, and the tool must be called twice—first without the token to receive it, then again with it. It does not explicitly name alternatives, but for a distinct delete-user operation that is less critical.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_topic_accessA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds meaningful behavioral context beyond the annotations: the endpoint performs no publishing, tests only the read side, and the read/write separation in ntfy is a common source of confusion. This is consistent with readOnlyHint, idempotentHint, and destructiveHint.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no filler: the core purpose is front-loaded, and the important caveat about read/write separation follows immediately. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers scope, behavior, and the key edge case that causes confusion. With an output schema present, return-value details are already handled externally, so nothing essential is missing for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the topics parameter is already fully documented with pattern, cardinality, and default behavior. The description adds no extra parameter-level meaning beyond restating that topics are checked, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Reports whether') and resource ('configured credentials may SUBSCRIBE to each topic'), and explicitly clarifies that it does not publish. This clearly distinguishes it from publishing-related siblings like publish_message.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly frames the tool as testing read-side/subscribe access only, and warns that write-only tokens will be denied here while still being able to publish. It gives strong contextual guidance, though it does not explicitly name an alternative tool to use for publishing or other access checks.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_usersA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnly/idempotent annotations, the description discloses meaningful behavior: admin authorization is required, NTFY_TOPICS environment restrictions affect which grants are reported, and wildcard grants are expanded or entirely omitted based on allowed topics. This is exactly the kind of non-obvious behavior an agent needs to know before invoking the tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and efficiently structured: the first sentence states the core purpose, the second gives the key prerequisite, and the final paragraph covers an important environment-specific edge case. Every sentence earns its place, with no filler or repetition of schema details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the admin requirement, server-restriction behavior, wildcard expansion semantics, and the underlying question the tool answers. Since an output schema exists, return-value details are not needed, and the description is complete enough for an agent to select and call the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already describes all three parameters fully, so the baseline is 3. The description adds useful context about how topic matching and wildcard grants behave under server restrictions, which enriches the meaning of the topic and username filters without replacing schema-level parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names the exact resource ('users') and the specific payload ('per-topic grants'), immediately answering what the tool does. It also frames the tool as 'the answer to who can read or write topic X', which makes its purpose concrete and distinct from siblings like get_account or check_topic_access.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a clear use case, the 'who can read or write topic X' scenario, and an important prerequisite: an admin account, with get_server_info mentioned as the way to check qualification. However, it does not explicitly state when not to use this tool or name alternatives such as check_topic_access for narrower queries.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• publish_messageA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses non-obvious behavior beyond annotations: one request per topic, partial failure semantics, the need to check each 'ok' field, and the returned id also being the sequence id usable with update_message. This is the kind of behavioral context that prevents an agent from assuming atomicity or misinterpreting the result.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three tight paragraphs, each earning its place: one for the purpose, one for the critical partial-failure behavior, and one for the id/update workflow. The most important caveat is front-loaded in the second paragraph before any deeper detail. No filler or redundant restatement of schema content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 14-parameter tool with no required parameters and an output schema, the description appropriately focuses on what the schema cannot convey: non-atomic multi-topic publishing, per-topic result checking, and the relationship between returned ids and update_message. The output schema covers return values, and the input schema covers parameters, so the description fills the remaining semantic gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 14 parameters, including defaults, formats, and enums. The description adds no additional parameter-specific meaning; it focuses on return behavior and multi-topic semantics. With full schema coverage, the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The first sentence states a specific verb and resource: 'Sends a notification to one or more topics.' This is immediately distinguishable from sibling tools like get_message, delete_messages, or update_message, and the rest of the description clarifies its publish-oriented role by referencing update_message as a revision step rather than a publishing alternative.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for using this tool: ntfy lacks multi-topic publish, so it sends per-topic requests and reports outcomes separately. It also explicitly routes follow-up revisions to update_message, explaining a concrete workflow. It does not list exclusions or when-not-to-use scenarios, but the purpose is clear enough and the update_message guidance provides practical selection context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_messagesA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses rich behavioral context beyond annotations: two-step confirmation, that nothing is removed from cache, that list/get still return the message, that a delete event itself appears in listings, that ntfy 2.16.0+ is required, and that older servers return ok:false in non-error results. This far exceeds the minimal destructiveHint/idempotentHint annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but every sentence carries critical caveats: cache behavior, repeated-delete warning, and version compatibility. It is front-loaded with the core action and confirmation requirement. Slightly verbose, but the density of essential operational details justifies the length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the full invocation flow, the meaning of deletion, edge-case behavior, and failure modes for older servers. An output schema exists, so return values need not be described. For a destructive tool with a confirmation token, nothing essential is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds value by explaining the confirm_token flow ('call once without it to receive the token, then again with it') and clarifying that sequence_ids may refer to scheduled, not-yet-delivered notifications. Still, most parameter meaning is already captured in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description states a specific verb and resource: 'Deletes notifications and cancels scheduled ones that have not been delivered yet.' It also distinguishes itself from siblings by explaining that list_messages and get_message still return the message afterward due to the event-based delete semantics. This clearly differentiates the tool from read-only or update siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear usage procedure: call without confirm_token to receive it, then call again with it. Warns against re-deleting and explains how to interpret per-id results on older servers. However, it does not explicitly name alternatives like mark_messages_read or update_message or state when to prefer delete over them, so it lacks full when/when-not guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_messagesA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnly/idempotent annotations, the description discloses significant behavioral traits: oldest-first ordering, the next_since pagination loop, retention-dependent empty results, shortened message bodies, and the meaning of the 'updates' field. This gives the agent a realistic expectation of results and side effects without contradicting the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description has three focused paragraphs, each carrying distinct information: core behavior and pagination, topic discovery constraints, and result interpretation caveats. No sentence is redundant or filler; it is concise for the complexity of the tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 9 parameters, full schema coverage, an output schema present, and safety annotations provided, the description still covers the essential behavioral caveats an agent would need: pagination mechanics, retention ambiguity, body truncation, and update entries. The tool is complex enough that these additions are necessary, and they are all present.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds meaningful semantic context beyond the schema, especially for the 'since' parameter by explaining the next_since round-trip pattern and for the overall ordering semantics. This is more than a restatement, so it earns a 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Polls the cached messages of one or more topics, oldest first.' It clearly distinguishes itself from get_message by stating message bodies are shortened and pointing to get_message for full content, and from list_users/get_account by explaining topic discovery. An agent can immediately tell what this tool does and how it differs from siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly routes to get_message when a full message body is needed, and mentions get_account or list_users for topic discovery. It provides clear context for when this tool is appropriate (polling cached messages, using pagination via next_since) but does not comprehensively list when-not-to-use alternatives across the full sibling set. This is strong guidance but not exhaustive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• manage_user_accessA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description goes well beyond the annotations by disclosing that removing access breaks running publishers, granting access exposes traffic, deny is the only way to carve exceptions from wildcard grants, and revoke restores broader rules. This aligns with destructiveHint=true and readOnlyHint=false, and no contradiction with annotations is present.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is organized into four short paragraphs: purpose and prerequisites, destructive rationale, pattern/action semantics, and server restrictions. It is front-loaded with the core action, and each sentence adds operational value without meaningful padding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive mutation tool with four parameters, a confirmation flow, wildcard constraints, and server-specific restrictions, the description covers all key invocation factors. The presence of an output schema means return-value documentation is already handled, so nothing essential is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3, but the description adds meaningful context beyond the schema: it explains the wildcard pattern semantics, the practical difference between deny and revoke, and the NTFY_TOPICS restriction. It does not reach 5 because the raw parameter meanings are already well documented in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Sets or removes an account's access to a topic or topic pattern,' which is a specific verb plus resource and clearly distinguishes this mutation tool from the sibling check_topic_access. It also names the required admin and confirmation gate, so there is no ambiguity about what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for when to use the tool: it requires an admin account, is destructive in both directions, and is gated by a confirmation token. It also explains wildcard behavior and NTFY_TOPICS restrictions. However, it does not explicitly say 'use check_topic_access instead for read-only checks,' so it stops short of full when-to-use versus alternative guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mark_messages_readA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations cover idempotency and non-destructiveness, but the description adds meaningful behavioral context: messages remain in server cache, are still readable via list_messages, and older servers return per-id ok:false results without an overall error. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded with the core behavior, followed by a useful sibling distinction and a version-specific warning. Every sentence contributes operational value without unnecessary padding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema and annotations, the description is complete: it covers behavior, the key alternative, version constraints, and a subtle failure mode. Nothing essential for correct invocation is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so both parameters are already documented in the schema. The description adds no significant parameter-level meaning beyond what the schema provides, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Clears notifications on subscribers' devices') and a clear resource. It also distinguishes itself from delete_messages by explaining the exact behavioral difference, so an agent can tell which tool to use.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly contrasts this tool with delete_messages, explaining when each is appropriate. It also adds a version requirement (ntfy 2.16.0 or newer) and warns about per-id result checking, giving clear operational guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_messageA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Even with annotations marking this as destructive and idempotent, the description adds substantial behavioral context: the sequence_id only exists for cached messages, the cache stores revisions with the original preserved, list_messages surfaces an "updates" field, older servers fail silently, and a person-confirmation/token flow is required. These details go far beyond what the annotations alone convey and help the agent anticipate side effects and failure modes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but every sentence earns its place: purpose, cache/revision semantics, version compatibility, and human confirmation. It is front-loaded with the core concept and then layers the necessary caveats without fluff. The length is justified by the tool's complexity and the critical failure modes it must warn about.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists and the input schema fully documents parameters, the description covers the remaining contextual needs: when the operation is available, what happens for unsupported versions, how revisions appear in list_messages, and the exact confirmation protocol. Nothing essential for correct invocation is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds extra value by explaining that only supplied fields are sent, that sequence_id is tied to cached messages from publish_message, and that confirm_token is part of a two-call confirmation flow. This cross-parameter behavior is not fully evident from the schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: "Replaces the content of a notification already published." It also distinguishes itself from related operations by saying subscribers see the change "in place instead of receiving another one," which directly contrasts with publish behavior. This is more than adequate to differentiate update_message from publish_message and the other siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly indicates when this tool is appropriate: for already-published, cached notifications that should be revised in place. It also gives important exclusions: messages published with cache=false cannot be updated, and older ntfy versions silently fall back to publishing a new notification. It does not explicitly name publish_message as the alternative for non-cached messages, but the contrast is implied strongly enough that an agent can route correctly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_userA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations supply readOnly=false, idempotent=false, and destructive=false, and the description adds substantial behavioral context beyond these: it requires admin privileges, cannot create admins via the API, triggers a human confirmation step, and warns that passwords may remain in the transcript. This is exactly the kind of hidden behavior an agent needs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core action, then covers preconditions, post-creation access, admin limitations, confirmation behavior, and a privacy warning. Every sentence contributes necessary information without padding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists and the annotations cover safety hints, the description is complete. It covers authentication requirements, what to do after creation, how to create admins, the confirmation protocol, and a privacy concern. No critical operational context is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds meaningful value for confirm_token by explaining the two-call flow, and for password by warning about transcript exposure. It does not add much for username or tier, but the schema already documents those well.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Creates a non-admin account on a self-hosted instance.' It clearly distinguishes this from sibling tools like delete_user, list_users, and manage_user_access by specifying scope (non-admin) and context (self-hosted).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit usage context: it requires an admin account, explains that new accounts need manage_user_access to reach topics, and notes that admin creation must go through the ntfy CLI instead. It also explains the confirmation flow with a token, so an agent knows when to call once vs. twice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_server_infoA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Although annotations already indicate a safe read-only operation, the description adds valuable behavioral detail: sections are 'fetched independently,' unavailable sections 'do not fail the call,' and the 'version' field may be absent without admin credentials. This goes well beyond the annotations and helps the agent interpret partial responses correctly.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact but information-dense. The opening sentence defines the tool's purpose, the second gives the most important usage context, and the third explains failure-tolerant behavior. No sentence is wasted.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no parameters, an output schema present, and annotations declaring safety, the description covers the remaining important context: public accessibility, credential independence, per-section error behavior, and the admin-only 'version' field. An agent has everything needed to invoke and interpret this tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the empty schema fully covers parameter semantics. The baseline for zero-parameter tools is 4, and the description appropriately focuses on behavior rather than inventing parameter guidance.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reports 'Health, capabilities and usage of the ntfy instance,' naming the specific resource. It also differentiates itself from sibling tools by noting it is 'the one tool that works before the credentials are right,' removing ambiguity about its scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit guidance: it is 'a good first call after setup' and works before credentials are correct. This tells the agent when to select this tool even without naming an alternative, which is sufficient given the unique role of the tool among its siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.