Server Quality Checklist

Tool Scores

• lightning_create_invoiceC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states the tool creates an invoice but provides no information about what happens after creation (e.g., where the invoice is stored, how it's delivered, whether it's immediately payable), authentication requirements, rate limits, or error conditions. The 'Price: 5 sat' statement adds confusion rather than clarity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief (two short phrases) but not optimally structured. The 'Price: 5 sat' statement adds confusion rather than value. While concise, it's not effectively front-loaded with the most critical information, and the second phrase creates ambiguity rather than clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool that creates financial transactions with 3 parameters and no annotations or output schema, the description is inadequate. It doesn't explain what a successful invocation returns, error conditions, or practical usage context. The 'Price: 5 sat' statement creates more questions than answers about the tool's actual behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the schema already documents all three parameters thoroughly. The description adds no meaningful parameter information beyond what's in the schema. In fact, the 'Price: 5 sat' statement potentially contradicts the 'amount_sats' parameter which allows variable amounts. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Create') and resource ('Lightning invoice'), making the purpose immediately understandable. However, it doesn't distinguish this tool from potential sibling tools (like 'nostr_zap' which might also involve Lightning payments), and the inclusion of 'Price: 5 sat' is confusing and potentially misleading since the input schema shows the amount is configurable via 'amount_sats' parameter.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. While 'Price: 5 sat' might imply a fixed price scenario, this contradicts the parameter schema which allows variable amounts. There's no mention of prerequisites, context, or comparison with sibling tools like 'nostr_zap' or 'verify_receipt'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• nostr_publish_eventC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions a price ('3 sats'), which implies a cost, but doesn't clarify if this is per event, how payment works (e.g., via 'payment_preimage'), or other behaviors like rate limits, error handling, or what 'publish' entails (e.g., to default relays). The description is too sparse for a mutation tool with payment aspects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with two short sentences that are front-loaded with the core action. There is no wasted verbiage, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with payment requirements and no annotations or output schema, the description is incomplete. It doesn't explain the payment flow (e.g., that 'payment_preimage' is required after paying an invoice from another tool), what happens on success/failure, or how it differs from siblings like 'nostr_sign_event'. The cost mention is vague without context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds no additional meaning about parameters beyond implying a cost context with 'Price: 3 sats', which loosely relates to 'payment_preimage'. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Sign and publish') and resource ('a Nostr event to relays'), making the purpose specific and understandable. However, it doesn't distinguish this tool from its sibling 'nostr_sign_event' (which likely only signs without publishing) or 'nostr_post_note' (which might be a specialized version for kind 1 notes).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'nostr_sign_event' or 'nostr_post_note'. It mentions a price ('3 sats'), which hints at a cost context, but doesn't explain prerequisites (e.g., needing to pay first) or when to choose this over other publishing methods.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• nostr_sign_eventC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions a price ('2 sats') and implies a payment process via the 'payment_preimage' parameter, but doesn't explain key behaviors like authentication needs, rate limits, error handling, or what the signing process entails (e.g., is it atomic, reversible?). This is inadequate for a mutation tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two sentences, front-loading the core purpose. The second sentence about price adds context but could be integrated more smoothly. Overall, it's efficient with minimal waste, though not perfectly structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (a signing tool with payment integration), no annotations, and no output schema, the description is incomplete. It lacks details on return values, error cases, and behavioral traits, making it insufficient for safe and effective use by an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters well. The description adds no additional meaning beyond what's in the schema (e.g., it doesn't clarify 'kind' values or 'tags' structure). Baseline 3 is appropriate as the schema does the heavy lifting, but the description doesn't compensate or enhance parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Sign a Nostr event') and the mechanism ('using a sovereign signing daemon'), which is specific and actionable. However, it doesn't explicitly differentiate from sibling tools like 'nostr_post_note' or 'nostr_publish_event', which might involve signing, so it lacks sibling differentiation for a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It mentions a price of '2 sats', which hints at a cost context, but doesn't explain prerequisites (e.g., needing a payment first) or compare to other Nostr tools in the sibling list, leaving usage unclear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_receiptC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions a price ('Price: 5 sat'), which hints at cost/rate-limiting, but lacks details on permissions, response format, error conditions, or what 'verified' entails. For a tool with no annotations, this is insufficient behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise and front-loaded: two sentences with zero waste. The first sentence states the core purpose, and the second adds cost information efficiently, making it easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and 2 parameters, the description is incomplete. It lacks details on what verification returns, error handling, or how 'payment_preimage' relates to the price mentioned. For a verification tool with cost implications, more context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters well. The description adds no additional parameter semantics beyond implying 'event_id' is for verification and mentioning payment, but doesn't clarify 'payment_preimage' usage. Baseline 3 is appropriate as the schema handles most documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Verify an action receipt by event ID. Confirms it was signed by Jorgenclaw's key.' It specifies the verb ('verify'), resource ('action receipt'), and key verification detail. However, it doesn't explicitly distinguish this verification tool from sibling tools like 'create_action_receipt' or 'nostr_sign_event', which prevents a score of 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides minimal guidance: it implies usage for verifying receipts and mentions a price, but offers no explicit when-to-use rules, alternatives (e.g., vs. 'nostr_sign_event'), or prerequisites. Without clear context or exclusions, this falls short of higher scores.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_action_receiptB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It mentions pricing (21 sats) and third-party signing, but lacks critical behavioral details: whether this is a write operation, if it requires authentication, rate limits, what happens on failure, or the structure of the returned receipt. The description is insufficient for a mutation tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose. However, it could be more structured by separating the pricing information or adding a brief note on use cases.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with 5 parameters, no annotations, and no output schema, the description is incomplete. It lacks details on behavioral traits, error handling, return values, and how it differs from siblings like 'verify_receipt'. The pricing note is helpful but insufficient for full contextual understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all parameters. The description adds no parameter-specific information beyond what's in the schema, maintaining the baseline score of 3 for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Create a signed action receipt'), identifies the resource (receipt with kind 1111), and distinguishes it from siblings by mentioning third-party attestation via Jorgenclaw's signing key. It goes beyond a tautology by explaining the attestation mechanism and pricing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when attesting an agent's action with third-party signing, but provides no explicit guidance on when to use this versus alternatives like 'verify_receipt' or other sibling tools. No exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• nostr_get_notesB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses a cost ('Price: 5 sats'), which is a behavioral trait, but lacks other critical details such as rate limits, authentication needs, error handling, or what 'recent' means temporally. The description does not contradict annotations (none exist).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise and front-loaded with essential information (purpose and cost) in just two sentences. Every word earns its place, with no wasted text or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (4 parameters, no annotations, no output schema), the description is incomplete. It lacks details on return values, error conditions, or how the payment preimage relates to the cost mentioned. The cost disclosure is helpful, but more behavioral and output context is needed for adequate completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds no additional meaning beyond what the schema provides (e.g., it doesn't explain interactions between parameters like author vs hashtag). Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('fetch') and resources ('recent kind 1 notes'), and distinguishes it from siblings by specifying the type of content (notes vs profiles, events, etc.). It also mentions the cost ('5 sats'), which is unique among sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for fetching notes by author or hashtag, but does not explicitly state when to use this tool versus alternatives like 'nostr_fetch_profile' or 'nostr_post_note'. It provides some context (e.g., 'recent'), but lacks explicit exclusions or comparisons to other tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• nostr_zapA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses key behavioral traits: it's a payment tool with a fixed tool fee (50 sats) and a maximum zap amount (5000 sats). However, it doesn't mention authentication requirements, rate limits, error conditions, or what happens on success/failure, leaving significant gaps for a financial transaction tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise with just two sentences that are front-loaded with essential information. Every word earns its place: first sentence states the core action, second sentence provides critical constraints (price and limits). No wasted words or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a payment tool with 4 parameters, no annotations, and no output schema, the description is incomplete. While it covers purpose and basic constraints well, it lacks crucial context about authentication, error handling, return values, and how the payment preimage parameter relates to the transaction flow. The description doesn't fully compensate for the missing structured data.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds marginal value by reinforcing the max amount constraint for 'amount_sats' and implying the tool fee context, but doesn't provide additional semantic meaning beyond what's in the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Zap a Nostr user via Lightning') with the resource ('Nostr user'), and distinguishes it from siblings by specifying it's a payment action (vs. profile fetching, note posting, etc.). It provides concrete details about pricing and limits that further clarify its purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for sending Lightning payments to Nostr users, but doesn't explicitly state when to use this vs. alternatives like 'lightning_create_invoice' or 'create_action_receipt'. It provides some context with price and max amount, but lacks explicit guidance on prerequisites or comparison with sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• nostr_fetch_profileA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes the tool's purpose and adds critical context: it requires payment ('Price: 5 sat'), which implies a transactional behavior not obvious from the schema. However, it doesn't detail error handling, rate limits, or authentication needs beyond payment.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise and front-loaded, with every sentence earning its place: the first sentence defines the tool's purpose and inputs, and the second sentence adds crucial payment information. There is zero wasted text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (fetch operation with payment), no annotations, and no output schema, the description is somewhat complete but has gaps. It covers the purpose and payment requirement but lacks details on return values, error cases, or behavioral nuances like what happens if payment fails. It's adequate but not fully comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters thoroughly. The description adds minimal value beyond the schema by implying the pubkey is required and mentioning payment context, but it doesn't provide additional syntax or format details. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Fetch a Nostr profile'), identifies the resource type ('kind 0'), and specifies the input methods ('by npub or hex pubkey'). It distinguishes itself from siblings like nostr_get_notes (which fetches notes) and nostr_post_note (which creates content).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by mentioning the required parameter ('pubkey') and the payment requirement ('Price: 5 sat'), but it doesn't explicitly state when to use this tool versus alternatives like nostr_get_notes or provide clear exclusions. The context is somewhat clear but lacks explicit guidance on tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• nostr_post_noteA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds valuable context about the combined sign+publish operation and the cost ('Price: 21 sats'), which aren't obvious from the schema. However, it doesn't cover other important behaviors like error handling, response format, or network effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise (two brief sentences) and front-loaded with the core purpose. Every word earns its place: the first sentence defines the action and method, while the second provides critical cost information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations and no output schema, the description does the minimum viable job. It explains what the tool does and mentions the cost, but doesn't describe what happens after posting (success/failure responses, event ID return, etc.). Given the complexity of a paid Nostr operation, more behavioral context would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters thoroughly. The description doesn't add any parameter-specific information beyond what's in the schema, maintaining the baseline score of 3 for adequate coverage through structured data alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Post a kind 1 text note'), the resource ('to Nostr'), and the method ('sign + publish in one call'). It distinguishes this from sibling tools like 'nostr_sign_event' (sign only) and 'nostr_publish_event' (publish only) by emphasizing the combined operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('Post a kind 1 text note') and implies alternatives by mentioning the combined 'sign + publish' operation, suggesting separate tools exist for these functions. However, it doesn't explicitly state when NOT to use it or name specific alternatives beyond what's inferred from siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.