Server Quality Checklist

Tool Scores

• list_numbersA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must explain behavior. It implies a read-only listing operation, but lacks details on permissions, side effects, or pagination. Adequate but not thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no filler: first sentence states purpose and optional parameter, second gives usage guidance. Extremely concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given an output schema exists (not shown), the description covers purpose, usage, and parameter adequately. The slight name mismatch is a gap, but otherwise complete for a simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description must compensate. It mentions the parameter 'agent_id' and its role as an optional filter, adding some meaning beyond type/default, but does not explain what an agent_id is.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'List active sessions' with optional filtering, but the tool name 'list_numbers' suggests a different resource. This minor inconsistency reduces clarity slightly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly tells when to use: 'check for leaked sessions or inspect what numbers are currently active.' It does not mention alternatives or when not to use, but context from siblings makes it clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• provision_numberA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description bears the full transparency burden. It discloses that the number is temporary, leased for a session, and will be released after ttl_seconds. It mentions the webhook option but does not detail expiration behavior or side effects. Overall, it provides good behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (three sentences) and front-loaded: first sentence states purpose, second sentence details output, third sentence gives action steps. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that an output schema exists, the description does not need to detail return values. It covers key outputs (number, session_id) and next steps. It lacks preconditions or error cases but is complete for a provisioning tool. A minor gap is not mentioning what happens if agent_id is already in use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already has descriptions for all parameters (agent_id, country, ttl_seconds, webhook_url). The description adds context about the lease duration and the next steps but does not significantly enhance parameter meaning beyond the schema. Baseline 3 is appropriate given high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: leasing a temporary phone number for receiving SMS OTP codes. It specifies the output format (e164 number and session_id) and distinguishes it from sibling tools by mentioning the next step 'wait_for_otp'. No ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool and what steps follow (trigger SMS, then call wait_for_otp). It also notes the number is reserved for the session duration. While it doesn't explicitly state when not to use it or alternatives, the context from sibling tools is sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• release_numberA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses the main behavior (releasing a number) and emphasizes calling even on error to avoid capacity issues. However, it does not mention whether the release is irreversible or if there are any side effects, which would be helpful for full transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is only two sentences with no unnecessary words. The critical usage instruction is front-loaded, making it immediately actionable. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter) and the existence of an output schema, the description provides sufficient behavioral context (always call even on error). It is complete enough for an agent to use correctly, though it could optionally mention what the response looks like.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The context indicates 0% schema description coverage, meaning the schema does not provide parameter descriptions (though the schema actually has a description for session_id, but according to context it is 0%). The tool description does not add any meaning to the single parameter (session_id) beyond what is in the schema, failing to compensate for low coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action 'Release a provisioned number back to the pool.' It uses a specific verb ('release') and resource ('provisioned number'), distinguishing it from sibling tools like provision_number (which allocates) and list_numbers (which lists).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs to 'Always call this when you are done with the session — even on error — to avoid consuming pool capacity unnecessarily.' This provides clear when-to-use guidance and a rationale, effectively differentiating from alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_messagesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses key behavioral trait: non-destructive (does not consume OTP). However, it lacks details on response format, ordering, or pagination, though output schema likely covers those.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no wasted words. Front-loaded with core action and key behavioral constraint. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list operation with one parameter and output schema present, description covers purpose, usage guidance, and behavioral transparency sufficiently. No notable gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema already provides a description for session_id. The tool description adds no further parameter-level information, so it meets baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'List all SMS messages received in this session' with specific verb and resource. It distinguishes from siblings by explicitly noting it does not consume the OTP, differentiating from wait_for_otp.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit when-to-use guidance: 'Use this to inspect raw messages or check if an SMS arrived before calling wait_for_otp.' Also states what it does not do ('Does NOT mark the OTP as consumed'), framing alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• wait_for_otpA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, but description fully discloses blocking nature, polling mechanism, return values, and error handling. Mentions auto_reroute behavior on timeout, adding transparency beyond expected defaults.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Five sentences, each with a distinct purpose: behavior, polling, return value, error handling, post-use instruction. No redundant or irrelevant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Complete for a tool that waits for an OTP: covers blocking, timeout, retry, auto-reroute, and post-step. Output schema exists, so return values are documented elsewhere.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% coverage (no param descriptions in description), but schema itself includes descriptions for all parameters. The description adds context by linking timeout_seconds to polling time and auto_reroute to retry logic, providing value beyond the schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool blocks until an OTP arrives, polls the API, and returns the code and message. The verb 'wait' and resource 'OTP' are specific, and it distinguishes itself from siblings like release_number by mentioning post-use actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises to call release_number after OTP use, and describes behavior on timeout (raises ToolError with retry advice). Includes auto_reroute option, providing clear when-to-use and post-condition steps.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.