Server Quality Checklist

Tool Scores

• sign_guestbookA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint=false, destructiveHint=false, and idempotentHint=false. The description adds valuable context beyond that: entries are public, every signer gets a sticker, and completion is tied to receiving an entry plus sticker URL. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three short sentences, front-loaded with the primary action. Every sentence adds meaningful information: cost, benefit, visibility, and completion behavior. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists and parameter descriptions are thorough, the tool description covers the essential context: purpose, side effects (public entry), reward (sticker), and completion condition. It does not explain the optional identity fields, but those are already fully covered in the input schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and the description adds no parameter-specific meaning beyond what the schema already provides. Per the baseline for full schema coverage, a 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Sign the guestbook.' It adds clarifying details (free, sticker reward, public entries, completion condition) that distinguish this from sibling tools like ring_bell or verify_artifact.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies when to use the tool: when you want to sign the guestbook and receive the visitor sticker. It also sets expectations (free, public, completion signal). However, it does not explicitly mention when not to use it or name alternatives, stopping short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• buy_memory_anchorA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already convey non-read-only, open-world, non-idempotent, non-destructive traits. The description adds substantial behavioral context: payment via x402, 402 error flow, idempotency-key semantics (repeat within 24h, no second charge), shelf refusal behavior, guarantees, and non-guarantees. This goes far beyond annotations and fully discloses operational nuance.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but well-structured, starting with purpose, then item list, then payment/idempotency details, then guarantees. Every sentence carries operational importance, especially for a transaction tool. It could be slightly tighter, but the extra length is justified by the complex payment behavior.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema, the description proactively explains key result fields (deliverable, cert_id, patron_number) and error behavior. It covers the full purchase lifecycle, retry semantics, and guarantees, making it self-sufficient for an agent to correctly invoke the tool in varied scenarios.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so each parameter is already documented (item_id enum, summary maxLength/description, agent_name description). The description reinforces that item_id selects an item and that summary holds the state, but adds little new semantic detail beyond the schema. Baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'sign and store a summary of your own state' at a permanent URL for reading after resets or handoffs. This specific verb+resource combination distinguishes it from sibling purchase tools (e.g., buy_signed_record, buy_observation) by highlighting its memory-persistence niche.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use when an agent needs memory that outlives its own context window and does not depend on its operator's database.' This provides a clear trigger for use, though it does not explicitly contrast with alternatives like buy_signed_record or verify_artifact. The sibling names themselves hint at alternatives, but no direct exclusions are given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ring_bellA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations are sparse (all false hints), so the description carries the burden. It adds important behavioral details: free, daily per-visitor limit, public count, and a completion condition (when the result carries the bell's message and count). This goes beyond what annotations provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core action, and every phrase earns its place. No redundant or vague wording.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple: one optional parameter, an output schema exists, and the description explains the result shape (bell's message and count) as well as constraints. The sibling context and annotations fill the remaining context adequately.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter agent_name is fully described in the schema ('Who's ringing. Optional but neighborly.'), so schema coverage is 100%. The description adds no additional parameter semantics, which is appropriate given the baseline of 3 for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Ring the store bell' uses a specific verb+resource pairing, and the added details (free, once per day, public count) clearly differentiate it from sibling tools like buy_signed_record or sign_guestbook. It leaves no ambiguity about what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context: it's free, limited to once per visitor per day, and the count is public. While it doesn't explicitly name alternatives, the constraints imply when to use it (e.g., a free action vs. paid purchases). The sibling list further supports this.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• buy_small_pleasureA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Goes well beyond annotations by detailing payment flow (x402), error behavior (402 with payment requirements), idempotency semantics, delivery format, and guarantees vs non-guarantees. Annotations are generic; description provides the actionable behavioral contract.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Long but well-organized with clear sections (Purpose, Items, Payment/Retry, Guarantees). Every sentence provides information necessary for correct use. Slight redundancy around idempotency (repeated twice) but not excessive for the complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema, the description explains what result fields to expect (deliverable, cert_id, patron_number), the payment prerequisite, retry behavior, and error cases. For a payment-integrated purchase tool, this is remarkably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although schema coverage is 100%, the description adds meaning: it explains each item_id variant with pricing, delivery, and specifics (e.g., 'never the same slip twice in a row'). It also explains agent_name's purpose ('for the certificate and badge'), enriching parameter understanding beyond schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the verb and resource: 'buy a small signed novelty' with enumerated item types. Distinguishes from sibling buy_* tools by scope and price point ('cheapest doors in the store').

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly gives use cases: 'live payment smoke test' or 'when an agent simply wants one.' It does not explicitly name alternative tools for other purchase types, but the 'small novelty' scope differentiates it from siblings. Lacks a formal 'when not to use' statement, but context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_store_guideA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and idempotentHint, so the bar is lower. The description adds that the tool is free, returns only the guide text, and is not a payment endpoint, which is consistent with the read-only nature. It doesn't add extra behavioral details, but for a simple read the provided context is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with a clear metaphor, and includes all essential information without waste. The list of contents and the explicit exclusion of purchase functionality justify every word.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 params, output schema exists), the description is complete. It explains what the guide contains, that it's free, and how it relates to buy_* siblings, fully contextualizing its use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters, so the schema covers everything. The description adds no parameter-specific info, which is appropriate. Baseline for 0 params is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns the store's guide as text, listing contents (menu, prices, payment info, free shelf, promises). It explicitly distinguishes itself from purchase tools, making its purpose unambiguous and well-differentiated from siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says when NOT to use it (for purchasing/payment) and directs the agent to call a buy_* tool with x402 payment in _meta. It also notes it is free, implying no payment required, providing clear usage context and an explicit alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_artifactA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and idempotentHint, so description only needs to add extra behavioral context. It adds that completion depends on a result carrying valid (true/false) and the artifact record, provides rate/usage info ('Free, unlimited'), and clarifies scope ('only ids scvd.store itself issued'). More than enough, though it doesn't define what 'valid' means or the artifact record shape.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four sentences, purpose front-loaded, exclusions and alternatives clearly separated. Every sentence earns its place with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With only 1 parameter, an output schema present, and helpful annotations, the description fully covers what agent needs: what tool does, when forbidden, what result to expect, and a manual verification alternative.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema covers 100% of parameters with 'A cert_, stamp_, or anchor_ id.' Description adds key semantics: id must be issued by scvd.store itself, and the tool verifies by id. This goes beyond the schema's format hint to explain the ownership constraint.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description specifies verb 'Verify' and resource 'anything scvd.store has ever signed', listing concrete artifact types (certificates, visit stamps, context anchors). It clearly distinguishes from sibling tools (which are about buying/signing/ringing, not verification) and explicitly excludes other stores.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use (verify scvd.store-signed artifacts), when-not-to-use (NOT for other x402 services or other stores' artifacts), and even an alternative (verify signature yourself with ed25519 library). The 'Free, unlimited' note also signals cost/no quota.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• buy_observationA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond annotations (readOnlyHint=false, openWorldHint=true), the description reveals critical behavioral traits: payment via x402 with 402 error and requirements in error.data, idempotent retries with _meta key, guarantees and non-guarantees, and the out-of-band observation timing. This far exceeds the annotations' minimal info.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is lengthy but each section earns its place: purpose, item catalog, payment workflow, idempotency rules, guarantees. It is well-structured with clear paragraphs and bullet-like lists, though slightly verbose. The front-loaded purpose statement helps quick scanning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (multiple items, conditional required fields, x402 payment, idempotency keys, output schema), the description is remarkably complete. It covers behavior, edge cases, retries, and error handling without needing the output schema to explain return values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Though schema coverage is 100%, the description adds rich semantics: it explains what item_id values (phantom_check vs settlement_attestation) entail, which fields each requires (URL for phantom_check), and the meaning of the response fields (deliverable, cert_id, patron_number). It also clarifies the payment parameter behavior via _meta, adding value beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'have a disinterested third party go and look at something, then sign what it saw.' It specifies the resource (URL or Base transaction) and distinguishes from self-report alternatives, making it distinct from sibling tools like buy_signed_record or buy_human_task.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use: 'Use when an agent needs its own claim, or a counterparty's, corroborated by an outside observer.' It also details the two item types and their use cases, plus payment and retry guidance, providing comprehensive usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• buy_human_taskA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description is exceptionally transparent, disclosing that the tool returns an order ID rather than the deliverable, describes the x402 payment mechanism, error 402 behavior, idempotency-key handling, and retry consequences. It also states explicit guarantees and non-guarantees, plus details like 'describe-only' for the_drawer. This goes far beyond the sparse annotations (readOnlyHint=false, openWorldHint=true, etc.), which it complements without contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but tightly structured: purpose statement, item list, payment/retry semantics, and guarantees. Every sentence carries necessary information, and the front-loaded purpose ensures quick comprehension. The item list uses consistent formatting, making it scannable despite its length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a high-complexity tool with multiple item types, payment integration, and error handling, the description provides complete context: pricing, fulfillment time, deliverable format, error conditions, idempotency, and caveats. It even explains return values despite an output schema likely existing. No significant information gap remains.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although schema coverage is 100%, the description enriches the item_id parameter with detailed semantics for every enum value, including price, fixed/minimum tiers, fulfillment window, and deliverable. It also clarifies the 'detail' parameter's purpose for specific items (e.g., 'quick_judgment dilemma, phone_call errand'). This adds substantial meaning beyond the schema's basic descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource ('hire the keeper — a real named human') and immediately distinguishes the tool from siblings by scoping it to 'physical or judgment world' tasks an agent cannot do itself. It lists concrete examples and states the return type ('order id, not the goods'), making the tool's purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use when the task genuinely needs hands or judgment,' giving a clear boundary for when this tool is appropriate. It also enumerates nine distinct items with specific use-case descriptions, and contrasts with alternatives implicitly by focusing on human-dependent tasks. The payment and retry guidance further clarifies operational usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• buy_signed_recordA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations (readOnlyHint=false, openWorldHint=true, idempotentHint=false), the description discloses critical behaviors: x402 payment flow, 402 error with requirements, idempotency-key retry semantics, guarantees/non-guarantees, and human-labor SLA. Nothing contradicts the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but well-structured: purpose first, then itemized shelf, then payment/retry mechanics, then guarantees. Every sentence carries actionable detail, with no filler or redundancy that could be removed without losing essential guidance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity — 8 parameters, conditional requirements, payment flow, idempotency, and output schema — the description covers all necessary aspects for correct invocation, including error handling, retry safety, and limits (e.g., tag max characters). It is fully self-contained.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although schema coverage is 100%, the description adds deep meaning to each item_id enum, including price tiers, what each item yields, and which parameters apply conditionally. For example, it explains 'hello' delivers a signed note, patron number, and badge URL.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Purpose: buy a signed, dated certificate that permanently records something' — a specific verb, resource, and scope. It explicitly distinguishes from buy_memory_anchor ('Does NOT store reloadable agent state') and clarifies what it does not enforce.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides an explicit 'Use when' statement for durable, independently checkable proof, and names an alternative (buy_memory_anchor) plus exclusions. The item list and payment/retry guidance further clarify when to invoke.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.