Server Quality Checklist

Tool Scores

• get_delivery_statisticsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool retrieves statistics, implying a read-only operation, but doesn't cover aspects like authentication requirements, rate limits, data freshness, or what 'recent messages' entails. This leaves significant gaps in understanding the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It is front-loaded and appropriately sized for its function, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete. It doesn't explain what statistics are returned (e.g., success rates, delivery times), how data is formatted, or any error conditions. For a tool with one parameter but missing structured context, more detail is needed to fully guide the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the 'limit' parameter well-documented. The description adds no additional parameter details beyond what the schema provides, such as clarifying 'recent messages' or statistical metrics. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get') and resource ('SMS delivery statistics and success rates from recent messages'), making the tool's purpose understandable. However, it doesn't explicitly differentiate from sibling tools like 'check_sms_status' or 'get_sms_messages', which might also involve SMS message data, so it doesn't reach the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'check_sms_status' or 'get_sms_messages'. It lacks context on prerequisites, such as needing recent messages or specific account permissions, leaving the agent with minimal usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_sms_messagesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure but offers minimal information. It doesn't mention whether this is a read-only operation, what permissions might be required, potential rate limits, or how results are returned (e.g., pagination, format). While 'Retrieve' implies reading, the lack of detail about behavioral traits is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise at just one sentence with no wasted words. It's front-loaded with the core purpose and efficiently communicates the essential function without unnecessary elaboration, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete for a retrieval tool with two parameters. It doesn't explain what the returned data looks like, how historical messages are organized, or any behavioral constraints. While the schema covers parameters well, the overall context for proper tool invocation remains inadequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with both parameters ('direction' and 'limit') fully documented in the schema itself. The description adds no additional parameter information beyond what's already in the structured schema, so it meets the baseline expectation but provides no extra value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Retrieve') and resource ('SMS message history from 46elks'), making the tool's purpose immediately understandable. However, it doesn't explicitly differentiate this retrieval tool from sibling tools like 'check_sms_status' or 'get_delivery_statistics', which might also involve SMS message data access.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'check_sms_status' or 'get_delivery_statistics'. It doesn't mention prerequisites, appropriate contexts, or exclusions, leaving the agent with insufficient information to make informed usage decisions among related SMS tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• send_smsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure but offers minimal information. It mentions the service provider ('46elks') but doesn't cover critical aspects like authentication requirements, rate limits, error handling, or what happens upon sending (e.g., confirmation, costs, delivery status). This is inadequate for a mutation tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with just four words ('Send SMS message via 46elks'), making it front-loaded and efficient. Every word contributes essential information (action, resource, provider), with no wasted sentences or redundant phrasing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a mutation tool (sending SMS) with no annotations and no output schema, the description is incomplete. It lacks details on behavioral traits (e.g., costs, delivery guarantees), error scenarios, or what the tool returns, leaving significant gaps for an AI agent to understand the tool's full context and implications.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, meaning all parameters are documented in the input schema itself. The description adds no additional parameter semantics beyond what's already in the schema (e.g., it doesn't explain parameter interactions or provide examples), so it meets the baseline for high schema coverage without compensating with extra value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Send SMS message') and specifies the service provider ('via 46elks'), which provides a specific verb+resource combination. However, it doesn't differentiate this tool from its siblings like 'check_sms_status' or 'get_sms_messages', which would require explicit comparison to achieve a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There are no mentions of prerequisites, when-not-to-use scenarios, or comparisons with sibling tools like 'estimate_sms_cost' for planning or 'check_sms_status' for verification, leaving the agent without contextual usage instructions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_account_balanceB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It mentions the tool's purpose but doesn't disclose behavioral traits like authentication requirements, rate limits, whether it's read-only or has side effects, or what specific information is returned. The description is functional but lacks operational transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that states the tool's purpose clearly. It's appropriately sized for a simple tool and front-loads the essential information without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a financial checking tool with no annotations and no output schema, the description is insufficient. It doesn't explain what specific account information is returned, the format of the balance, whether this requires special permissions, or how the agent should interpret the results. The context demands more completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters with 100% schema coverage, so the schema already fully documents the parameter situation. The description appropriately doesn't discuss parameters, which is correct for a parameterless tool. Baseline is 4 for zero parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('check') and resource ('46elks account balance and account information') with a specific purpose ('to verify funds availability for SMS sending'). It distinguishes from siblings by focusing on account-level information rather than SMS operations, though it doesn't explicitly name alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context ('to verify funds availability for SMS sending'), suggesting this should be used before sending SMS messages. However, it doesn't provide explicit guidance on when to use this versus other tools or mention any prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_sms_statusA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states the tool checks status and details, but does not disclose behavioral traits such as required permissions, rate limits, error handling, or what specific details are returned (e.g., delivery timestamps, error codes). This leaves gaps for a read operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('Check delivery status and details') without any wasted words. It is appropriately sized for a simple lookup tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (one parameter, read-only operation) and lack of annotations or output schema, the description is minimally adequate. However, it does not fully compensate for missing behavioral context (e.g., what details are returned, error cases), leaving some gaps for an agent to understand the tool's complete behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, clearly documenting the single required 'message_id' parameter. The description adds no parameter-specific information beyond what the schema provides, but with only one parameter and high schema coverage, the baseline is appropriately met without needing extra details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Check delivery status and details') and the resource ('a sent SMS'), distinguishing it from siblings like 'send_sms' (creation) and 'get_delivery_statistics' (aggregate data). It precisely communicates the tool's function without redundancy.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for checking status of sent SMS messages, but does not explicitly state when to use this tool versus alternatives like 'get_sms_messages' (which might retrieve multiple messages) or 'get_delivery_statistics' (which could provide broader analytics). No exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• estimate_sms_costA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the tool's non-destructive nature ('without sending it'), which is useful behavioral context. However, it lacks details on rate limits, authentication needs, or error handling, leaving gaps for a tool that interacts with billing/systems.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that efficiently conveys the tool's purpose and key constraint ('without sending it'). There is no wasted verbiage, and every word earns its place in clarifying the tool's function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (cost estimation with 3 parameters) and no annotations or output schema, the description is adequate but incomplete. It covers the core purpose but lacks details on output format (e.g., cost breakdown, segment count) or prerequisites, which would help an agent use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, so parameters are well-documented there. The description adds no additional parameter semantics beyond implying the tool uses 'message' and 'to' for estimation, which is already clear from the schema. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Estimate cost and message segments') and the resource ('SMS'), distinguishing it from sibling tools like 'send_sms' (which sends) and 'check_sms_status' (which checks status). It explicitly mentions 'without sending it' to differentiate from the sending functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: to estimate costs before sending an SMS. It implies an alternative ('send_sms' for actual sending) but does not explicitly name it or specify when not to use this tool (e.g., for real-time cost checks vs. historical data).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.