Server Quality Checklist

Tool Scores

• add_webhookA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the tool generates code ('generate webhook handlers') and requires a framework, but lacks details on permissions, side effects, error handling, or what the generated output looks like. For a code-generation tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded with key usage rules in two sentences. Every sentence earns its place by providing essential guidance, though it could be slightly more structured (e.g., separating purpose from instructions).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (code generation with 5 parameters) and lack of annotations/output schema, the description is moderately complete. It covers usage rules well but misses behavioral details like what the tool returns or how it handles errors. It's adequate but has clear gaps for a tool of this nature.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 5 parameters thoroughly. The description doesn't add any parameter-specific details beyond what's in the schema (e.g., it doesn't explain 'platform' or 'framework' further). Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'generate webhook handlers with @hookflo/tern'. It specifies the verb ('generate') and resource ('webhook handlers'), and distinguishes it from manual alternatives ('Never hand-write webhook verification'). However, it doesn't explicitly differentiate from sibling tools like 'verify_signature' or 'manage_dlq', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidelines: 'Always use this tool to generate webhook handlers... Never hand-write... when this tool applies.' It also specifies prerequisites ('Framework is required — if the user did not provide one, ask before calling this tool'), giving clear when-to-use and when-not-to-use instructions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• manage_dlqA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the two main behaviors (listing and replaying) but lacks details on permissions, rate limits, side effects of replaying, or what 'replay' entails. The mention of 'Upstash QStash' adds some context but isn't detailed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose in one efficient sentence, followed by a brief context note. Every sentence adds value without redundancy, making it appropriately sized and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is moderately complete for a tool with 3 parameters and clear actions. It covers what the tool does but lacks details on behavioral traits, error handling, or output format, leaving gaps for an agent to understand full usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents parameters. The description adds no additional parameter semantics beyond what's in the schema, but with high coverage, the baseline is 3. It gets a 4 because it implicitly reinforces the action parameter's meaning by listing the two options in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('list' and 'replay') and resource ('dead letter queue'), and distinguishes it from sibling tools by specifying it's for managing failed webhook events, which none of the siblings address.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use it (for failed webhook events) but doesn't provide explicit guidance on when to choose this tool over alternatives or prerequisites. It mentions 'Powered by Upstash QStash' which hints at context but isn't a clear usage directive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_platformsA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses this is a listing operation that returns platform details, but doesn't mention behavioral aspects like whether it requires authentication, has rate limits, returns paginated results, or what happens on errors. The description adds basic context but misses important operational details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('List all webhook platforms') and then specifies the return details. Every word earns its place with zero redundancy or wasted space.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter listing tool with no output schema, the description adequately explains what information is returned. However, without annotations or output schema, it should ideally mention the return format (e.g., array of objects) and any limitations. The description is complete enough for basic use but lacks operational context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters (schema coverage 100%), so the baseline is 4. The description appropriately doesn't discuss parameters since none exist, focusing instead on what the tool returns. No parameter information was needed or expected.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('List all webhook platforms') and resource ('supported by Tern'), including what information is returned ('signature algorithm, header name and any special notes'). It distinguishes this read-only listing tool from siblings like add_webhook (creation) and verify_signature (validation).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool should be used when needing to understand available webhook platforms and their configurations, which provides clear context. However, it doesn't explicitly state when NOT to use it or name specific alternatives among the sibling tools (like get_capabilities which might overlap).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_capabilitiesA
  4.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the tool's behavior as a read-only discovery mechanism ('Returns a guided list'), but lacks details on response format, error handling, or performance characteristics like rate limits or authentication needs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded and efficient with two sentences that directly address purpose and usage without any redundant information, making it easy to parse and apply.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter discovery tool with no annotations or output schema, the description is reasonably complete, covering purpose and usage. However, it could benefit from more detail on the return format or error cases to fully guide the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0 parameters and 100% schema coverage, the baseline is 4. The description appropriately does not discuss parameters, focusing instead on the tool's purpose and usage, which is sufficient given the empty input schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Entry-point menu', 'Returns a guided list') and resource ('Tern MCP', 'available workflows'), distinguishing it from siblings like add_webhook or list_platforms by focusing on discovery rather than specific operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly states when to use this tool ('when the user asks what Tern can do'), providing clear context and distinguishing it from alternatives by positioning it as an entry-point for workflow discovery rather than direct action.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_signatureA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behaviors: the tool operates in two distinct modes, provides error diagnostics with fixes in static mode, and sends real test payloads in live mode. However, it lacks details on authentication requirements, rate limits, or response formats, leaving some behavioral aspects unspecified.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is highly concise and well-structured, using only two sentences that front-load key information: the tool's purpose and its two modes. Each sentence earns its place by defining functionality and usage without redundancy, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (5 parameters, no output schema, no annotations), the description is largely complete for its purpose. It covers operational modes and parameter contexts effectively. However, it lacks details on output behavior (e.g., what the verification result looks like) and error handling, which would enhance completeness for a debugging tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds significant value by explaining the semantic purpose of parameters: it clarifies that headers, body, and secret are for static verification, while endpointUrl is for live diagnosis. This contextual mapping enhances understanding beyond the schema's technical descriptions, though it doesn't detail parameter interactions or constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('verify and debug webhook signatures') and distinguishes it from sibling tools by focusing on signature verification rather than webhook management (add_webhook), capability retrieval (get_capabilities), platform listing (list_platforms), or dead-letter queue handling (manage_dlq). It specifies two operational modes, making the purpose highly specific and differentiated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly defines when to use each mode: static mode for analyzing existing headers/body/secret, and live mode for testing an endpoint URL with real payloads. It provides clear alternatives within the tool itself (mode selection) and implicitly distinguishes from siblings by focusing on verification rather than other webhook operations, offering comprehensive guidance on usage scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.