Server Quality Checklist

Tool Scores

• wait_for_smsA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the blocking nature and timeout behavior, which is critical for an agent to know. However, it does not mention what happens on timeout (returns null, throws), whether the returned code is consumed, or any side effects such as the rental being invalidated. The description adds some behavioral context but leaves key details undisclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short and front-loaded: the first sentence states exactly what the tool does. The second sentence provides context about its role in an automated flow, though it slightly overstates by implying it rents the number ('rent a number and get the code') when it only waits. Despite this minor inaccuracy, it's concise and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With three parameters, no output schema, and no annotations, the description should clarify return behavior and edge cases. It does not explain what happens on timeout, the structure of the returned code, or whether the code is consumed. Given the tool's blocking nature and its placement among siblings (after rent_number), the description is adequate but leaves important gaps for an agent to operate fully.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so each parameter (rentalId, timeoutMs, intervalMs) already has a clear description. The tool's description adds context about the purpose (waiting for SMS/OTP) but does not significantly clarify the parameters beyond what the schema provides. Thus, baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Block until the first SMS/OTP code arrives on a rental, or time out.' It uses a specific verb ('Block') and names the resource ('a rental'), with the purpose of retrieving an SMS/OTP code. This distinguishes it from siblings like list_services (listing) and rent_number (renting), making its role unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context: 'The one-step way to rent a number and get the code in an automated flow.' This suggests using it after rent_number to wait for the code. However, it does not explicitly state when to use it versus alternatives (e.g., polling get_rental) or mention any exclusions, so guidance is only implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_rentalA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of describing behavior. It clearly states what information is returned (status, phone number, and SMS messages received so far), and the phrase 'so far' implies a point-in-time snapshot. While it could mention error handling or side effects, this is a read-only operation and the description is sufficiently transparent for its simple nature.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that directly states what the tool does. Every word adds value, with no fluff or repetition. It is appropriately sized for the tool's simplicity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple get operation with one parameter and no output schema, the description gives a solid overview of the return contents (status, phone number, SMS messages). It does not detail the status values or message structure, but given the tool's limited scope and the presence of sibling tools, it provides enough context for an agent to use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema's description for rentalId is clear and meaningful ('The rentalId returned by rent_number'), so schema coverage is 100%. The tool description itself does not add further parameter detail, but the schema already provides the necessary context, meeting the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Check' and clearly identifies the resource: a rental's status, phone number, and SMS messages. This distinguishes it from sibling tools like list_services (which lists services) and rent_number (which rents a number), so the agent knows exactly what this tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used to retrieve current information about a rental, and the parameter description (in the schema) notes the ID comes from rent_number. This gives clear context for when to call it, though it does not explicitly compare it to alternatives like wait_for_sms. Still, the purpose is clear enough to guide correct selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• rent_numberA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral transparency burden. It discloses the monetary side effect ('Debits the account balance'), the return value ('rentalId'), and the nature of the number ('real UK phone number'). It does not cover failure modes or prerequisites, but the disclosed side effect is crucial for a rental tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences: the first states the purpose and capability, the second states the cost side effect and how to proceed with the result. Every word earns its place, with no fluff or unnecessary repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the essential aspects for a mutating tool with no output schema: what it does, what it costs, what it returns, and how to use the result. Minor omissions include error handling and explicit requirement to call list_services first, but the schema covers the serviceSlug source, so the description remains largely complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters with descriptions and enum values. The tool description adds minimal parameter context beyond restating 'given service' and mentioning the return value, which does not exceed the baseline 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Rent') and a clear resource (a real UK phone number for a service) with the intended purpose (receiving an SMS/OTP verification code). It clearly differentiates from sibling tools: list_services lists services, get_rental/wait_for_sms check rental status or wait for SMS.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you need a UK number for OTP verification and explicitly mentions the follow-up tools get_rental or wait_for_sms, creating a clear workflow. It does not explicitly state exclusions or contrast with alternatives, but the context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_servicesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of disclosing behavioral traits. The verb 'List' implies a read-only operation with no side effects, and the mention of 'prices' indicates the return value includes pricing. For a simple listing tool, this is sufficient transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one sentence, front-loaded with the verb and object, and includes helpful examples ('WhatsApp, Telegram, Google, Discord, etc.'). No unnecessary words, perfectly concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple: no parameters, no output schema, and a clear listing purpose. The description fully covers what the tool does and what it returns (services and prices), and the sibling tools provide surrounding context. Nothing essential is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool accepts zero parameters, and the input schema is empty, so schema coverage is 100%. The description correctly omits parameter details, as there are none to explain. A baseline of 4 applies for tools with no parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and a clear resource ('services sms-florin can rent a UK phone number for'), with examples and pricing. It clearly distinguishes itself from sibling tools like rent_number and get_rental, which perform different actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for browsing available services and their prices before renting. It doesn't explicitly state when not to use it or name alternatives, but the purpose is self-evident given the sibling tool names and context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.