Server Quality Checklist

Tool Scores

• commit_intentA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explains the tool's purpose (submitting commitments with privacy controls) and use cases, but lacks details on behavioral traits like error handling, rate limits, authentication requirements, or what happens after commitment (e.g., is it reversible?). It does not contradict annotations (none exist).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded with the core purpose in the first sentence, followed by usage contexts. It avoids redundancy, but the second sentence could be slightly more concise (e.g., merging privacy aspects). Overall, it earns its place with minimal waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (6 parameters, no output schema, no annotations), the description is adequate but has gaps. It explains the tool's purpose and use cases well, but lacks details on behavioral aspects (e.g., side effects, error responses) and does not address output or post-commitment behavior, which is important for a tool with privacy-sensitive operations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds no specific parameter semantics beyond the high-level mention of privacy controls ('Control what is revealed: hide amounts, identity...'), which aligns with but does not expand upon the schema. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Submit a sealed-bid commitment for a trading intent') and resource ('trading intent'), distinguishing it from siblings like create_intent (which creates intents) or explain_intent (which explains them). It also specifies the protocol context (Hashlock protocol — hashlock.markets).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('peer-to-peer trading, private negotiations, agent-to-agent settlement, dark pool orders, or any crypto exchange where privacy and zero slippage matter'), but it does not explicitly state when NOT to use it or name alternatives among the sibling tools (e.g., when to use validate_intent instead).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_intentA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions that the tool works for 'human traders, autonomous AI agents, and institutional counterparties' and involves 'verified counterparty' trading, which adds useful context about the user types and verification requirements. However, it doesn't disclose critical behavioral traits like whether this creates a pending transaction, requires authentication, has rate limits, or what happens upon execution failure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded with the core purpose in the first sentence. However, the second sentence becomes somewhat redundant with the first, and the final usage guideline sentence could be more tightly integrated. Overall efficient but with minor structural improvements possible.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex trading tool with 34 parameters and no output schema, the description provides adequate context about what the tool does and when to use it. However, it lacks information about what happens after creation (e.g., does it return an intent ID? where is the intent stored? how is it executed?), which would be important for a creation tool with no output schema documentation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description mentions key parameters like 'what you give, what you want, privacy level, KYC tier, and settlement terms,' which adds semantic meaning beyond the 100% schema coverage. However, with complete schema documentation already present, the description doesn't significantly enhance parameter understanding beyond what's already in the structured fields.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Create a trading intent to swap, buy, sell, exchange, or convert any asset' with specific examples (crypto tokens, real-world assets, stablecoins) across multiple blockchains. It distinguishes from siblings by focusing on creation rather than committing, explaining, parsing, or validating intents.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: 'Use this whenever a user wants to trade, swap, buy, sell, convert, or exchange any digital asset with a verified counterparty.' This clearly defines when to use this tool versus alternatives, though it doesn't explicitly mention when not to use it or name specific sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• parse_natural_languageA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool 'understands requests' and 'supports English and Turkish', which adds context about language capabilities. However, it lacks details on error handling, rate limits, or authentication needs, leaving behavioral gaps for a parsing tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, followed by examples and usage guidelines in two efficient sentences. Every sentence adds value—no redundancy or wasted words—making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (parsing natural language) and no annotations or output schema, the description is mostly complete. It covers purpose, usage, and examples, but lacks details on output format or error cases, which could be helpful for an AI agent invoking the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters fully. The description implies the 'text' parameter usage through examples but adds no additional syntax or format details beyond what the schema provides. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Convert everyday language into a structured trading intent' with specific examples like 'sell 10 ETH for USDC above 4000'. It distinguishes from siblings by focusing on natural language parsing rather than intent creation, validation, explanation, or commitment.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance is provided: 'Use this whenever a user describes a crypto trade, swap, exchange, or asset conversion in natural language.' It specifies supported languages (English and Turkish) and gives clear examples of when to apply the tool, with no contradictory or missing exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_intentA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool 'catches missing fields, invalid token amounts, chain mismatches, and business rule violations,' which gives some behavioral context (e.g., it performs validation checks). However, it doesn't describe the response format, error handling, or what happens on success/failure, leaving gaps in behavioral understanding for a validation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is highly concise and well-structured: two sentences that front-load the core purpose and follow with critical usage guidance. Every sentence earns its place by providing essential information without waste, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (validation with business rules), no annotations, and no output schema, the description is mostly complete. It covers the purpose, usage context, and validation scope. However, it lacks details on the return format or what constitutes a valid vs. invalid result, which would be helpful for an agent to interpret outcomes. Still, it provides sufficient context for basic use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the single parameter ('intent' as a JSON string). The description adds no additional parameter semantics beyond implying the JSON should represent a crypto trading intent. Since the schema does the heavy lifting, the baseline score of 3 is appropriate, with no extra value from the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('validate a crypto trading intent') and resource ('intent'), distinguishing it from siblings like commit_intent (which submits) and create_intent (which creates). It explicitly mentions what it catches (missing fields, invalid amounts, etc.), making the purpose highly specific and differentiated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: 'Always validate before committing a swap, trade, or exchange.' This directly tells the agent when to use this tool (before committing) and implies alternatives (like commit_intent for submission). It clearly establishes the tool's role in the workflow relative to siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• explain_intentA
  4.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It describes the tool's function (explaining intents) and context (pre-commitment verification), but lacks details on behavioral traits like error handling, response format, or performance characteristics. It doesn't contradict annotations, but doesn't fully compensate for their absence.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: the first sentence states the core purpose, and the second sentence provides usage guidelines. Every sentence earns its place with no redundant or vague language, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (explaining trading intents), no annotations, and no output schema, the description is reasonably complete. It covers purpose, usage, and parameter context, but lacks details on output format or error handling, which would be helpful for an agent to use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the single parameter 'intent' as 'The intent JSON to explain.' The description adds value by clarifying what the intent contains (trading details like crypto, amounts, blockchain) and the output's nature ('plain-language explanation'), but doesn't provide additional syntax or format details beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Get a plain-language explanation') and resources ('trading intent'), specifying what aspects are explained (crypto/tokens/assets, amounts, blockchain, privacy/KYC settings). It distinguishes from siblings like 'create_intent' or 'validate_intent' by focusing on explanation rather than creation or validation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool: 'to confirm swap/trade/exchange terms with your user before they commit.' This provides clear context for usage (pre-commitment verification) and distinguishes it from alternatives like 'commit_intent' (for finalizing) or 'parse_natural_language' (for initial parsing).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.