Server Quality Checklist

Tool Scores

• macaroons_searchA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It does disclose that results are ranked and include price, predicate hash, and freshness, which is helpful. However, it does not describe sorting behavior, pagination semantics beyond the schema's limit parameter, error conditions, or the exact structure of the returned listings.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences with no filler. It front-loads the core purpose and then efficiently adds return-value highlights and the key cost-related behavior.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple search tool with a small parameter set and full schema coverage, the description is mostly complete. It tells the agent what the tool does, what results contain, and that no payment is needed; minor gaps such as exact response formatting and sort criteria are not critical for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents both parameters. The description adds no additional meaning beyond repeating the concept of 'matching an intent' and does not elaborate on how the 'limit' affects ranking or result completeness.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Search') and resource ('Macaroon Network') and clearly states the purpose: finding capabilities matching an intent. It also distinguishes itself from sibling tools by explicitly describing a read-only discovery operation with ranked results, rather than metadata access, purchase, or execution.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this is the discovery tool to use before purchasing or executing, but it does not explicitly name alternatives or say when not to use it. The statement 'Free, no payment required' provides useful context that this is a non-purchase operation, but it stops short of routing the agent away from siblings like macaroons_metadata.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• macaroons_purchaseA
  4.2/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden and does an excellent job: it discloses hold-invoice settlement, refund-on-failed-predicate, no-wallet default behavior, and the need to pay a returned invoice manually. Nothing important is hidden.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three tight sentences, with the core purpose first, the payment settlement mechanism second, and the no-wallet fallback last. Every clause earns its place and the unusual L402 behavior is explained compactly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the complex payment semantics, refund path, and fallback despite no output schema or annotations. It is slightly incomplete about what a successful paid response contains, but the phrase 'change-events' and rich input schema make the remaining gap minor.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The tool-level description mostly restates what the predicate and resume_macaroon schemas already say, adding no substantive parameter-level insight beyond the structured descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and object ('Purchase change-events for a feed target') and names the resource being acted on. The paid/retrieval nature and the feed-target scope distinguish it from search, metadata, and execute siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explains the payment flow and fallback behavior, and references feed target id from macaroons_metadata, giving some contextual guidance. However, it never explicitly states when to prefer this tool over macaroons_search or macaroons_execute, nor does it give when-not-to-use conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• macaroons_metadataA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It does well by stating the operation is free, requires no payment, and returns specific fields. It could have been more explicit that it performs no purchase or execution, but the 'Get' and 'before purchasing' framing strongly implies a read-only, non-destructive action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tight sentences, front-loaded with the main action and outcome. Every sentence adds value: what it does, what it returns, and that no payment is required. No filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter metadata lookup with no output schema, the description provides enough context: when to call it, what it returns, and its cost/behavior. It is sufficient for an agent to select and invoke the tool correctly in the purchase workflow implied by the siblings.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%: the schema already documents target_id clearly. The description adds only the phrase 'feed target' which aligns with the schema, but does not enrich parameter semantics meaningfully beyond the schema. Baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Get'), a clear resource ('metadata for a feed target'), and concrete outputs ('freshness and content_hash'). It also distinguishes itself from purchase-related siblings by explicitly saying it is for 'before purchasing' and requires no payment.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear usage timing: use this when you want free metadata 'before purchasing'. It does not explicitly name alternatives or exclusion conditions, but the 'before purchasing' signal is enough to orient the agent relative to siblings like macaroons_purchase and macaroons_execute.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• macaroons_executeA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden, and it delivers: it explains the hold-invoice settlement condition, refund on bridge errors or predicate failures, and the no-wallet default that returns payment_required with a real invoice. This is unusually concrete about payment behavior and error paths.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three dense sentences with the main action first, followed by payment mechanics and the default no-wallet behavior. Every sentence adds information and none repeats schema content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The complex payment flow, refund behavior, and no-wallet fallback are well covered. However, since there is no output schema, the description could be more explicit about the success return shape and how refund/error responses are surfaced; it only details the payment_required return path.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds workflow-level meaning around predicate (settlement condition) and resume_macaroon (usable only after a prior payment_required) beyond the parameter descriptions. It still leaves max_spend_sats to the schema, which is acceptable because the schema already documents it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb ('Execute') and resource ('paid marketplace capability by capability_id'), and the payment mechanism makes it distinct from sibling search/metadata/purchase tools. It is immediately clear what operation the tool performs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description makes the tool's role clear—executing a paid capability and paying via L402—but does not explicitly state when to prefer it over macaroons_purchase or search. Context implies it, but no direct exclusions or alternatives are named.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.