Server Quality Checklist

Tool Scores

• create-webhookD
  1.8/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states the action ('create') without explaining what happens after creation (e.g., whether it returns an ID, how webhooks are triggered, authentication needs, rate limits, or error handling). This leaves critical behavioral traits undocumented for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with a single sentence, 'Create a webhook', which is front-loaded and wastes no words. However, this conciseness comes at the cost of under-specification, but per the scoring rules, it earns full marks for brevity and structure.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a webhook creation tool (a mutation with no annotations, no output schema, and incomplete parameter documentation), the description is severely inadequate. It doesn't cover what the tool returns, how to handle the created webhook, or integration with sibling tools. The lack of behavioral and parameter details makes it incomplete for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 50% (only the 'event' parameter has a description listing possible values). The description adds no parameter semantics beyond what's in the schema—it doesn't explain what 'url' is for (e.g., endpoint for notifications) or provide context for the 'event' enum values. With low schema coverage, the description fails to compensate, leaving half the parameters poorly documented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Create a webhook' is a tautology that restates the tool name without adding meaningful context. It specifies the verb ('create') and resource ('webhook'), but lacks details about what a webhook is for in this system or what it does. It doesn't distinguish from siblings like 'delete-webhook' or 'list-webhooks' beyond the obvious action difference.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a valid URL or event type), when it's appropriate (e.g., for real-time notifications), or when not to use it (e.g., for one-time checks). There's no reference to sibling tools like 'list-webhooks' for viewing existing webhooks.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete-webhookC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool deletes a webhook, implying a destructive mutation, but lacks details on permissions required, whether deletion is reversible, error handling, or side effects. This is a significant gap for a mutation tool without annotation support.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste—'Delete a webhook by ID'—front-loading the core action and resource. It's appropriately sized for a simple tool, making it easy for an AI agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a destructive operation with no annotations, 0% schema coverage, and no output schema), the description is inadequate. It lacks critical context such as behavioral traits, parameter details, and usage guidelines, making it incomplete for safe and effective tool invocation by an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for undocumented parameters. It mentions 'by ID', which adds meaning to the 'webhookId' parameter, but doesn't explain what constitutes a valid ID (e.g., format, source from 'list-webhooks'), leaving the parameter semantics incomplete for the single required parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Delete') and target resource ('a webhook by ID'), making the purpose unambiguous. It distinguishes from siblings like 'create-webhook' and 'list-webhooks' by specifying deletion. However, it doesn't explicitly contrast with all siblings (e.g., 'get-events' or 'send-email'), keeping it from a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing an existing webhook ID), exclusions, or comparisons to sibling tools like 'list-webhooks' for finding IDs. Usage is implied but not explicitly stated, leaving gaps for an AI agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• send-emailC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure but offers none. It doesn't mention whether this is a read-only or destructive operation, authentication requirements, rate limits, error conditions, or what happens upon success. For a tool that sends emails (a clear side effect), this is a significant gap in behavioral transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is maximally concise at just 5 words, with no wasted language. It's front-loaded with the core function and gets straight to the point without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex email-sending tool with 11 parameters, no annotations, no output schema, and low schema coverage, the description is completely inadequate. It provides only the most basic functional statement without any of the contextual information needed to use the tool effectively or understand its behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With only 18% schema description coverage and 11 parameters, the description provides zero information about parameters. It doesn't mention any of the key parameters (from, to, subject, attachments, etc.) or their semantics. The description fails to compensate for the low schema coverage, leaving most parameters undocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Send') and resource ('email') with the specific service ('using SendLayer'), making the purpose immediately understandable. However, it doesn't distinguish this tool from potential sibling email-sending tools (though none are listed among the actual siblings).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives, prerequisites, or constraints. It's a bare statement of function without any contextual usage information that would help an agent decide when this is the appropriate tool to invoke.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-eventsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions 'optional filters' but doesn't disclose behavioral traits like pagination (implied by retrieveCount), rate limits, authentication needs, or what happens with missing parameters. This is inadequate for a tool with 5 parameters and no output schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise at 4 words, front-loaded with the core purpose. Every word earns its place, with no redundant information, making it efficient for an agent to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters, 60% schema coverage, no annotations, and no output schema, the description is incomplete. It doesn't compensate for the gaps in behavioral transparency or parameter understanding, leaving the agent with insufficient context for effective tool invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 60%, with 3 of 5 parameters having descriptions. The description adds minimal value by mentioning 'optional filters' but doesn't explain parameter interactions, defaults, or semantics beyond what the schema provides. Baseline 3 is appropriate given moderate schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Retrieve events with optional filters' clearly states the verb ('retrieve') and resource ('events'), and mentions the optional filtering capability. However, it doesn't differentiate from sibling tools (which are webhook/email related), though this may not be necessary since they serve different domains.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention any prerequisites, context for filtering, or relationships with sibling tools like list-webhooks, leaving the agent to infer usage scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list-webhooksB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. 'List registered webhooks' implies a read-only operation but doesn't specify whether it requires authentication, returns paginated results, includes error handling, or has rate limits. This leaves significant gaps in understanding the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core purpose and appropriately sized for a simple list operation with no parameters.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter list tool with no annotations and no output schema, the description is minimally adequate. It states what the tool does but lacks details on return format, error conditions, or behavioral constraints. Given the simplicity, it meets basic requirements but leaves room for improvement.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description appropriately doesn't mention parameters, which aligns with the schema. A baseline of 4 is applied since no parameters exist and the description doesn't introduce unnecessary complexity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'List registered webhooks' clearly states the verb ('List') and resource ('registered webhooks'), making the tool's purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'get-events' which might also retrieve webhook-related data, so it doesn't reach the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get-events' or 'create-webhook'. It lacks explicit instructions on context, prerequisites, or exclusions, leaving the agent to infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.