Server Quality Checklist

Tool Scores

• parse_invoiceC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure but offers minimal information. It states the tool parses an invoice but doesn't describe what parsing entails (e.g., extracting amount, destination, expiry), whether it validates the invoice, what happens with invalid input, or the format of results. For a tool with zero annotation coverage, this leaves critical behavioral traits unspecified.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without any fluff or redundancy. It is appropriately sized for a simple tool and front-loaded with the core action, making it easy to scan and understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (parsing a structured data format), lack of annotations, and absence of an output schema, the description is insufficiently complete. It doesn't explain what parsing yields (e.g., decoded fields like amount or payee), handle edge cases (e.g., invalid invoices), or provide context for use with siblings. For a tool with no structured output documentation, the description should compensate more.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the single parameter 'invoice' documented as 'the bolt11 invoice'. The description adds no additional semantic context beyond this (e.g., explaining BOLT-11 format requirements or examples). According to scoring rules, when schema coverage is high (>80%), the baseline is 3 even without param details in the description, which applies here.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Parse') and the target resource ('a BOLT-11 lightning invoice'), making the purpose immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'request_invoice' (which likely creates invoices) or 'fiat_to_sats' (which converts currency), leaving room for potential confusion about when to choose this specific parsing tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a valid invoice string), contrast with siblings (e.g., use 'parse_invoice' to decode an existing invoice vs. 'request_invoice' to generate one), or specify typical contexts (e.g., after receiving a payment request). This lack of usage context could lead to incorrect tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• request_invoiceC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action ('Request an invoice') but doesn't describe what happens after the request (e.g., whether it generates a payment request, returns an invoice string, or triggers a network operation). It also lacks information about permissions, rate limits, or error conditions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's front-loaded with the core action and resource, making it easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 4 parameters, no annotations, and no output schema, the description is insufficient. It doesn't explain what the tool returns (e.g., an invoice string, payment request object, or confirmation), behavioral traits like network effects or error handling, or how it relates to sibling tools. The lack of output schema increases the need for return value explanation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 50% (2 of 4 parameters have descriptions). The description doesn't add any parameter-specific information beyond what's in the schema. However, it implies the purpose of the parameters (lightning_address for recipient, amount for invoice value), which aligns with the schema. Baseline 3 is appropriate given the partial schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Request an invoice') and the target ('from a lightning address'), providing a specific verb+resource combination. However, it doesn't explicitly differentiate from sibling tools like 'parse_invoice' (which likely analyzes existing invoices) or 'fiat_to_sats' (which converts currency).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There's no mention of prerequisites, when-not-to-use scenarios, or how this differs from sibling tools like 'parse_invoice' (which might handle existing invoices) or 'fiat_to_sats' (which might be a prerequisite for determining the amount).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fiat_to_satsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the conversion action but lacks details on how the conversion works (e.g., source of exchange rates, accuracy, latency), whether it's read-only or has side effects, or any error handling. This leaves significant gaps for a tool that performs calculations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's front-loaded and directly communicates the core function without unnecessary elaboration, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 parameters, no output schema, no annotations), the description is minimally adequate. However, it lacks context about the conversion process (e.g., rate source, timestamp), which could be important for accurate usage. Without annotations or output schema, more detail would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with clear documentation for 'currency' and 'amount'. The description adds no additional meaning beyond the schema, such as examples or constraints (e.g., valid currency codes, amount ranges). Since the schema does the heavy lifting, a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Convert fiat amounts to sats' clearly states the tool's function with a specific verb ('Convert') and resources ('fiat amounts' to 'sats'), making the purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'parse_invoice' or 'request_invoice', which appear to be related to Bitcoin transactions but serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention any prerequisites, context for conversion (e.g., real-time rates, historical data), or exclusions, leaving the agent to infer usage based on the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.