Server Quality Checklist

Tool Scores

• list_requestsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral transparency. It only says 'List', failing to disclose pagination, ordering, or whether results are sorted. Schema hints at filtering but description adds no behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise at 7 words, with no wasted words. However, it could be slightly more informative without losing conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of 5 parameters, an output schema, and no annotations, the description is insufficient. It omits critical details like filtering options, pagination, and return format, leaving the agent underinformed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 40%, yet the tool description adds no parameter information beyond what the schema already provides. It does not explain the meaning of parameters like limit, method, or inbox_id.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool lists webhook requests from a HookRay inbox, with a specific verb and resource. It differentiates itself from siblings like create_webhook_inbox or inspect_request.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives. The purpose is implied by the name and description, but there is no direct comparison or exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• wait_for_requestB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses the blocking behavior and timeout result, but with no annotations, it falls short on details like side effects, idempotency, or whether it can be cancelled. The timeout parameter is mentioned implicitly but not fully explained.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise (one sentence) and front-loaded, but it is borderline under-specified. Every word earns its place, but it lacks context for effective tool invocation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and no annotations, the description is too sparse for a blocking tool with three parameters. It fails to explain what the timeout result looks like, how to interpret the response, or that the call is long-running.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description does not explain any of the three parameters (inbox_id, timeout_seconds, since). With only 33% schema description coverage, the description should compensate, but it adds no parameter-specific meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool blocks until a new webhook arrives or returns a timeout result. The verb 'wait' matches the tool name, and the resource 'request' is implied. It distinguishes from siblings like 'create_webhook_inbox' (creation) and 'inspect_request' (inspection).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage as a blocking wait for new webhooks, but it doesn't explicitly state when to use this over alternatives like 'list_requests' or 'replay_request'. No guidance on prerequisites (e.g., create inbox first) or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• inspect_requestB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It implies read-only behavior by using 'inspect', but does not disclose any further behavioral traits such as permissions, rate limits, or side effects. The description is adequate but minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence of 12 words, very concise and front-loaded with the verb 'Inspect'. However, it omits necessary parameter details, which slightly reduces effectiveness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema (not shown), the description need not explain return values. However, the tool has 0% schema parameter coverage, and sibling tools are distinct, so the description is mostly complete but lacks parameter semantics, which impacts completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, meaning the input schema provides no explanations for inbox_id and request_id. The description does not clarify these parameters, leaving the agent to infer their meaning from context. This is insufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool inspects full headers, body, query params, and metadata for a captured request. It distinguishes from sibling tools like create_webhook_inbox, list_requests, replay_request, and wait_for_request, which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives, nor any prerequisites or exclusions. The description only states what the tool does without providing context for appropriate usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• replay_requestB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits. It only says 'replay', hinting at network activity but not explaining side effects, whether it modifies original data, or any authentication requirements. Schema fields like timeout_seconds suggest execution duration but lack elaboration.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One sentence of 15 words, front-loaded with the core action. No unnecessary words. Every part is essential.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite 5 parameters and 3 required ones, the description omits context about what 'captured' means, how to obtain inbox_id/request_id, and what the output schema returns. The tool is moderately complex but the description is too sparse.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 40% (only destination_url and override_method have descriptions). The tool description adds no parameter meaning beyond what's in the schema. For inbox_id and request_id, no guidance is provided on where to obtain them.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (replay), the resource (captured HookRay request), and the destination (any reachable from this machine). It distinguishes from sibling tools like create_webhook_inbox, inspect_request, etc., which do not perform replay.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for replaying requests but does not explicitly say when to use this tool versus alternatives like inspect_request or wait_for_request. There is no when-not guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_webhook_inboxA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only states creation and return URL. It does not disclose side effects, authentication requirements (persistent requires Pro plan/API key), or any other behavioral traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that is front-loaded with the action and result, containing no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one optional parameter and an output schema, the description is adequate. However, it could be strengthened by mentioning its role in a workflow alongside sibling tools like wait_for_request.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the description does not add additional meaning beyond the schema's built-in description of the 'persistent' parameter. The output URL mention is about return value, not parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (Create), the resource (HookRay webhook inbox), and the output (public capture URL). It is specific and distinguishes from sibling tools which handle existing requests rather than creating inboxes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage (before capturing webhooks) but lacks explicit guidance on when to use this tool versus alternatives, or any prerequisites like needing an API key for persistent inboxes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.