Server Quality Checklist

Tool Scores

• get_conversation_messagesC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden for behavioral disclosure. It only states the action ('Get messages') without detailing traits like read-only nature (implied but not explicit), pagination behavior (hinted by 'before' parameter but not explained), rate limits, authentication needs, or error handling. For a tool with no annotations, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no wasted words. It's front-loaded with the core action and resource, making it easy to parse. Every word earns its place, achieving optimal conciseness for such a basic statement.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and a tool with 3 parameters (including pagination via 'before'), the description is incomplete. It doesn't explain return values (e.g., message format, ordering), behavioral aspects like pagination, or how it fits with siblings (e.g., 'list_conversations' to get IDs). For a read operation in a conversational context, more guidance is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents parameters (conversationId, limit, before). The description adds no additional meaning beyond what's in the schema, such as explaining how 'before' enables pagination or typical use cases for 'limit'. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the basic purpose ('Get messages in a specific conversation') which is clear but vague. It specifies the verb 'Get' and resource 'messages' with scope 'in a specific conversation', but doesn't differentiate from siblings like 'list_conversations' or provide details about what 'messages' entails (e.g., text, metadata, ordering). This is adequate but lacks specificity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description doesn't mention prerequisites (e.g., needing a conversation ID), exclusions, or comparisons to siblings like 'list_conversations' for broader context or 'send_message' for related actions. This leaves the agent without context for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_messageC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only states the action without behavioral details. It doesn't mention permissions required, whether deletion is permanent or reversible, rate limits, or error conditions. For a destructive operation, this leaves significant gaps in understanding the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with zero wasted words. It's front-loaded with the core action and resource, making it immediately scannable and efficient for an agent to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive tool with no annotations and no output schema, the description is insufficient. It lacks critical context like success/error responses, side effects, or safety warnings. Given the complexity of deletion operations and the absence of structured behavioral hints, more completeness is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the 'messageId' parameter as a UUID. The description adds no additional semantic context beyond what's in the schema, such as where to obtain the ID or format nuances. Baseline 3 is appropriate when the schema handles parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Delete') and target resource ('a bot message by ID'), making the purpose immediately understandable. It doesn't distinguish from siblings like 'edit_message' or 'send_message' beyond the verb, but the specificity is adequate for basic understanding.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'edit_message' or 'send_message', nor any prerequisites or constraints. The description only states what it does, leaving the agent to infer usage context from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• edit_messageC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. While 'Edit' implies a mutation operation, it doesn't specify permissions required, whether edits are reversible, rate limits, or what happens to the original message. The description is too minimal for a mutation tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core action and resource, making it immediately clear despite its brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations, 50% schema coverage, and no output schema, the description is inadequate. It doesn't explain what 'edit' entails (e.g., partial vs. full updates), error conditions, or return values. More context is needed given the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 50% (only 'messageId' and 'text' have descriptions). The description mentions 'previously sent bot message' which hints at the 'messageId' parameter, but adds no details about 'type', 'buttons', or their relationships. It doesn't compensate for the coverage gap, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Edit') and resource ('a previously sent bot message'), making the purpose immediately understandable. It doesn't differentiate from sibling tools like 'delete_message' or 'send_message', but it's specific enough to understand what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'send_message' for new messages or 'delete_message' for removal. There's no mention of prerequisites (e.g., needing an existing bot message) or constraints (e.g., time limits for editing).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_token_pricesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states what the tool does but doesn't describe important behavioral aspects like whether it's read-only (implied by 'Get'), potential rate limits, data freshness, error conditions, or response format. This leaves significant gaps for an agent to understand how to use it effectively.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that efficiently communicates the core functionality without any unnecessary words. It's front-loaded with the essential information and wastes no space on redundant or verbose explanations.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations and no output schema, the description is insufficiently complete. It doesn't explain what the return values look like (e.g., price format, timestamp, error handling), nor does it address behavioral aspects like rate limits or data sources. Given the context of financial data retrieval, more completeness would be expected.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the parameter 'mints' is fully documented in the schema. The description doesn't add any additional parameter information beyond what's already in the schema, which is acceptable given the high schema coverage. The baseline score of 3 reflects adequate but not exceptional parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get') and resource ('current USD prices for Solana tokens'), making the tool's purpose immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'get_token_list' or 'get_wallet_balance', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_token_list' (which might list tokens without prices) or 'get_wallet_balance' (which might include token values). There's no mention of prerequisites, limitations, or appropriate contexts for usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_transaction_historyC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure but offers minimal information. It doesn't specify if this is a read-only operation, what permissions are needed, rate limits, or the format of returned data. The description is too basic for a tool that likely interacts with blockchain data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's appropriately sized and front-loaded, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of blockchain transactions and no output schema, the description is incomplete. It doesn't explain what data is returned (e.g., transaction types, timestamps, amounts) or behavioral aspects like error handling. This leaves significant gaps for an agent to use the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all parameters (address, limit, before). The description adds no additional meaning beyond what's in the schema, such as explaining the relationship between parameters or typical usage patterns, meeting the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the resource 'transaction history for a Solana wallet address', making the purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'get_transaction_status' or 'get_wallet_balance', which also retrieve transaction-related data, so it doesn't reach the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'get_transaction_status' for specific transaction details or 'get_wallet_balance' for balance checks, leaving the agent to guess based on context alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_userC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only states the action without disclosing behavioral traits like permissions needed, rate limits, or what information is returned. It's a basic read operation, but lacks details on safety, response format, or constraints beyond the conversation scope.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that efficiently states the purpose without waste. It's front-loaded and appropriately sized for a simple tool, though it could be slightly more structured with added context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and a simple input schema, the description is incomplete. It doesn't explain what user information is retrieved, response format, or any behavioral aspects, leaving gaps for the agent to infer usage in a conversational context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents the single 'userId' parameter. The description adds no additional meaning beyond implying the user must be in a conversation, which doesn't enhance parameter understanding. Baseline 3 is appropriate as the schema handles the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('information about a user'), specifying the user must have a conversation with the bot. However, it doesn't distinguish this from potential sibling tools like 'get_bot_info' or 'list_conversations' in terms of scope or relationship, keeping it from a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides minimal guidance by implying usage for users in conversation with the bot, but it lacks explicit when-to-use rules, prerequisites, or alternatives. No comparison to siblings like 'list_conversations' or 'get_bot_info' is made, leaving the agent with vague context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• send_messageC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool 'sends a message', implying a write/mutation operation, but doesn't cover critical aspects like authentication needs, rate limits, error handling, or what happens on success (e.g., message ID returned). This is inadequate for a tool that modifies state without structured safety hints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose and lists supported features. It avoids redundancy and wastes no words, though it could be slightly more structured (e.g., separating purpose from features) for optimal clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a state-modifying tool with 6 parameters, no annotations, and no output schema, the description is insufficient. It lacks details on behavioral traits (e.g., side effects, response format), usage constraints, and error scenarios, leaving the AI agent with significant gaps despite the comprehensive input schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all 6 parameters. The description adds minimal value by listing supported content types ('text, buttons, images, and card carousels'), which loosely maps to the 'type' parameter and hints at 'buttons' and 'cards', but doesn't provide additional syntax or usage details beyond what's in the schema. This meets the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('send a message') and specifies the source ('from the bot') and target ('to a user in a specific conversation'), which distinguishes it from sibling tools like 'edit_message' or 'delete_message'. However, it doesn't explicitly differentiate from other messaging tools (none present in siblings), so it's not a perfect 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions 'Supports text, buttons, images, and card carousels', which implies usage for different message types, but provides no explicit guidance on when to use this tool versus alternatives (e.g., 'edit_message' for updates, 'delete_message' for removal) or any prerequisites like required permissions. This leaves gaps for an AI agent to infer correct usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• send_transactionC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states the action ('Send') which implies a write operation with network effects, but doesn't disclose critical traits: whether this is irreversible, requires specific permissions, has rate limits, returns a transaction ID, or handles errors. For a network submission tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that states the core purpose without unnecessary words. It's appropriately sized for a single-parameter tool and front-loads the essential action. Every word earns its place with no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of sending blockchain transactions (a write operation with financial implications), no annotations, and no output schema, the description is insufficiently complete. It doesn't explain what happens after sending (e.g., returns transaction hash, confirmation status), error conditions, or safety considerations. For this context, more behavioral and output information is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the single parameter 'signedTransaction' fully documented in the schema as 'Base64-encoded signed transaction'. The description adds no additional parameter semantics beyond what the schema provides, such as format details, validation requirements, or example values. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Send') and resource ('pre-signed Solana transaction to the network'), making the purpose immediately understandable. It distinguishes from siblings like 'simulate_transaction' by specifying actual network submission rather than simulation. However, it doesn't explicitly contrast with other transaction-related tools beyond the name difference.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (like needing a signed transaction first), when not to use it (e.g., for testing vs. production), or direct comparisons to siblings like 'simulate_transaction' for testing before sending. The agent must infer usage context from tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_welcome_messageC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states this is a 'Set' operation (implying mutation/write), but doesn't describe what happens to existing welcome messages, whether this requires specific permissions, what the response looks like, or any side effects. This is inadequate for a mutation tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without any wasted words. It's appropriately sized and front-loaded with the essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations and no output schema, the description is insufficient. It doesn't explain what happens when the tool is invoked (e.g., does it replace existing welcome messages?), what the return value is, or any error conditions. The description should provide more behavioral context given the lack of structured metadata.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters thoroughly. The description adds no additional parameter information beyond what's in the schema. This meets the baseline expectation when schema coverage is complete.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Set') and resource ('welcome message shown when a user starts a conversation'). It doesn't explicitly distinguish from siblings like 'edit_message' or 'send_message', but the context of 'welcome message' provides some implicit differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, timing considerations, or compare with sibling tools like 'edit_message' for modifying existing messages or 'send_message' for general messaging.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_bot_infoB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states this is a read operation ('Get information'), implying it's non-destructive, but doesn't cover aspects like authentication requirements, rate limits, error conditions, or the format of returned data. For a tool with zero annotation coverage, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action ('Get information about the current bot app') and lists specific fields without unnecessary details. Every word earns its place, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It doesn't explain the return format (e.g., JSON structure), potential errors, or dependencies. For a tool that retrieves bot metadata, more context is needed to ensure the agent can use it effectively without structured output guidance.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description appropriately doesn't discuss parameters, focusing instead on the tool's purpose. This meets the baseline for a parameterless tool, but doesn't add extra value beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('information about the current bot app'), specifying what fields are retrieved (name, description, status, webhook URL). It distinguishes from siblings like 'get_user' or 'get_wallet_balance' by focusing on bot metadata rather than user or transaction data. However, it doesn't explicitly contrast with all siblings, so it's not a perfect 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., bot authentication), exclusions, or suggest other tools for related tasks (e.g., 'set_webhook' for updating the webhook). This leaves the agent without context for optimal tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_wallet_balanceB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states what the tool does but lacks details on traits like rate limits, error handling, data freshness, or authentication requirements. This is a significant gap for a tool that likely queries external data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core functionality without any wasted words. It directly communicates the tool's purpose in a compact form, making it easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (querying balances with USD conversion), lack of annotations, and no output schema, the description is minimally adequate. It covers the basic purpose but misses behavioral details and output expectations, leaving gaps in context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with the single parameter 'address' fully documented in the schema. The description adds no additional parameter information beyond implying it's for a Solana wallet, which the schema already covers. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get') and resources ('SOL and SPL token balances for a Solana wallet address, including USD values'), making the purpose immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'get_token_prices' or 'get_transaction_history', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a valid wallet address), exclusions, or comparisons to sibling tools like 'get_token_list' or 'get_transaction_history', leaving usage context unclear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_conversationsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions 'with pagination', which adds some behavioral context, but fails to disclose other traits such as whether this is a read-only operation, any rate limits, authentication needs, or what the return format looks like. For a list tool with no annotations, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('List all conversations for the bot app') and adds a key behavioral trait ('with pagination') without unnecessary words. Every part of the sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity is low (a simple list operation with 2 parameters) and schema coverage is high, the description is minimally adequate. However, with no output schema and no annotations, it lacks details on return values or behavioral constraints, leaving gaps in completeness for effective agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents the 'limit' and 'offset' parameters. The description adds no additional meaning beyond what the schema provides, such as explaining pagination mechanics or default behaviors, resulting in a baseline score of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('conversations for the bot app'), making the purpose understandable. However, it does not explicitly differentiate from sibling tools like 'get_conversation_messages', which might retrieve messages within a specific conversation, leaving some ambiguity in sibling distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions 'with pagination', which implies usage for handling large datasets, but provides no explicit guidance on when to use this tool versus alternatives like 'get_conversation_messages' or other list-related tools. There are no when-not or alternative usage instructions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_webhookB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden but lacks critical behavioral details. It mentions configuring a webhook but doesn't disclose whether this overwrites existing settings, requires authentication, has rate limits, or what happens on success/failure. The description is insufficient for a mutation tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose without unnecessary words. Every part earns its place by specifying the action, resource, and event types concisely.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given this is a mutation tool with no annotations and no output schema, the description is incomplete. It doesn't explain the impact (e.g., replaces existing webhook), return values, error conditions, or security requirements, leaving significant gaps for agent understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters ('url' and 'events') thoroughly. The description adds no additional parameter semantics beyond what the schema provides, such as format examples or event type specifics, meeting the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Configure') and resource ('webhook URL where the bot receives events'), with explicit mention of event types ('messages, callbacks'). It distinguishes from siblings like 'set_welcome_message' by focusing on webhook configuration rather than message content.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. While it implies configuration for event reception, it doesn't specify prerequisites (e.g., bot setup), exclusions, or compare with other tools like 'send_message' for event handling.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_transaction_statusB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'confirmation status' but does not detail what that entails (e.g., success/failure states, network latency, error handling, or rate limits). This leaves significant gaps in understanding the tool's operational behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence with zero wasted words. It front-loads the core purpose efficiently, making it easy to parse and understand immediately without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It does not explain what the return value includes (e.g., confirmation details, error responses) or address potential behavioral aspects like network dependencies or failure modes, leaving the agent with insufficient context for robust use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the 'signature' parameter fully documented. The description does not add any additional meaning beyond what the schema provides, such as format examples or validation rules. Baseline 3 is appropriate given the schema's comprehensive coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Check') and resource ('confirmation status of a Solana transaction'), distinguishing it from siblings like get_transaction_history (historical data) and simulate_transaction (pre-execution analysis). It precisely communicates the tool's function without redundancy.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for verifying transaction confirmation, but does not explicitly state when to use this tool versus alternatives like get_transaction_history or simulate_transaction. No guidance is provided on prerequisites or exclusions, leaving usage context inferred rather than defined.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_latest_blockhashA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states what the tool does but doesn't describe important behavioral aspects like whether this is a read-only operation, if it has rate limits, what authentication might be required, or what format the blockhash is returned in. The description is functional but lacks operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that communicates the essential information without any wasted words. It's front-loaded with the core purpose and includes necessary context about Solana transactions. Every word earns its place in this compact description.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's zero parameters and no output schema, the description provides adequate basic information about what the tool does and why it's used. However, for a blockchain tool that likely returns time-sensitive data (blockhashes expire), more context about the return format, freshness, or typical usage patterns would be helpful, especially with no annotations to provide behavioral context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters (schema coverage 100%), so the baseline is 4. The description appropriately doesn't discuss parameters since none exist, and instead focuses on the tool's purpose and context. No parameter information is needed or missing.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Get') and resource ('latest blockhash'), and explicitly explains its purpose ('needed for building Solana transactions'). This distinguishes it from sibling tools like get_transaction_history or get_transaction_status by focusing on a specific blockchain infrastructure element rather than transaction data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context ('needed for building Solana transactions'), suggesting this tool should be used when preparing transactions. However, it doesn't provide explicit guidance on when NOT to use it or name specific alternatives among the sibling tools (like which tools might require a blockhash vs. which don't).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_token_listA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It describes the tool as a read operation ('Get'), which implies it's non-destructive and likely read-only, but does not disclose behavioral traits like rate limits, authentication needs, or response format. The description adds basic context but lacks details on how the list is structured or any limitations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the key action ('Get') and resource ('list of supported SPL tokens'), with no wasted words. Every part of the sentence earns its place by specifying what data is returned.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (0 parameters, no annotations, no output schema), the description is complete enough for basic understanding. However, it lacks details on behavioral aspects like response format or limitations, which could be helpful for an AI agent. It meets minimum viability but has gaps in transparency.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description does not add parameter details beyond the schema, but this is appropriate given the lack of parameters. Baseline is 4 for 0 parameters, as the description adequately covers the tool's purpose without redundant information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the resource 'list of supported SPL tokens', specifying the data returned includes 'mint addresses, symbols, and metadata'. It distinguishes from siblings like get_token_prices (which returns prices, not token lists) and get_transaction_history (which returns transaction data, not token metadata).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when token metadata is needed, but does not explicitly state when to use this tool versus alternatives like get_token_prices or other data-fetching tools. It provides clear context for retrieving token information but lacks explicit exclusions or named alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• simulate_transactionA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes the tool's function (simulation, error checking, fee estimation) but lacks details on permissions needed, rate limits, whether it consumes resources, or what the output looks like (e.g., success/failure indicators). For a tool with no annotations, this is adequate but leaves gaps in operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the key action ('simulate a Solana transaction') and purpose ('check for errors and estimate fees before sending'). Every word earns its place with no redundancy or waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (simulation with potential side effects), no annotations, and no output schema, the description is minimally complete. It covers the core purpose and usage but lacks details on behavioral traits (e.g., whether it's read-only, error formats, or fee estimation accuracy), which are important for an agent to use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the single parameter ('transaction' as a Base64-encoded string). The description adds no additional parameter semantics beyond what the schema provides, such as format examples or validation rules. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('simulate', 'check for errors', 'estimate fees') and resource ('Solana transaction'), distinguishing it from siblings like 'send_transaction' (which actually sends) and 'get_transaction_status' (which retrieves status). It explicitly indicates this is for pre-send validation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool: 'before sending' a transaction, providing clear context for its use. It distinguishes from alternatives by implying this is for simulation only, not actual sending (contrasted with 'send_transaction'), and for error/fee estimation rather than status checking (contrasted with 'get_transaction_status').

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.