Server Quality Checklist

Tool Scores

• get_messagesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already include readOnlyHint=true (the tool is read-only) and openWorldHint=true (results may change between calls). The description goes beyond this by disclosing that reading doesn't consume send quota, that messages have a few seconds of delay, and that 'from' is inclusive while 'to' is exclusive to avoid double-counting. It also mentions that encrypted messages cannot be searched. These behavioral traits are not implied by the annotations and add significant value.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is detailed but well-organized, opening with the core purpose and then layering usage patterns (defaults, polling, batch checks). It is front-loaded with essential info (no device id needed, direction default) and later details are relevant to specific use cases. It is longer than ideal but every sentence adds value, and the structure helps the agent parse the key information without unnecessary fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (9 parameters, no output schema, no required params), the description is comprehensive: it covers direction semantics, polling mechanics, batch handling, device filtering, and search limitations. The absence of an output schema is partly mitigated by describing what results include (delivery status per row, next_cursor). Minor gaps include not specifying pagination details for direction='all' or explicitly saying that results are ordered, but the overall completeness is strong.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all nine parameters. The description adds crucial semantics beyond the schema: it clarifies the default direction and its rationale, explains that 'from' is inclusive and 'to' is exclusive for boundary handling, and describes the cursor's role in polling. It also integrates parameters like sms_batch_id and device_ids into workflows. The description substantially enhances the agent's understanding, justifying a score above the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads SMS messages from the user's textbee account, covering all devices without needing device IDs. It explicitly contrasts with siblings: it is for reading, not sending (send_sms) or listing devices (list_devices). The scope, direction defaults, and batch filtering are all specified, making its purpose unambiguous and distinct.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides rich usage guidance: it says when to use the default direction 'received' (for replies or one-time codes), when to use 'sent' (reviewing outgoing), and when to use 'all' or 'sms_batch_id' (checking delivery). It describes a complete polling pattern with order, from, and cursor, and explicitly advises waiting and retrying instead of tight-polling. No siblings are alternatives for reading, but the description covers all relevant scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_devicesA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, so the description's statement 'sends no SMS' adds behavioral context beyond annotations, reinforcing the read-only nature. It also mentions the details of what is returned (last checked-in time, message counts), which is transparent. No contradiction; the description complements the annotations well. The 4 reflects solid but not exhaustive behavioral detail (e.g., no mention of data freshness).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no fluff, with the purpose and key content in the first sentence and usage triggers in the second. The description is efficiently structured and front-loaded, every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list operation with no parameters and no output schema, the description fully covers what it does, what it returns (list of fields), when to use it, and what it doesn't do (no SMS). An agent has all necessary information to call it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. The description explicitly states 'Takes no arguments,' aligning with the empty schema and leaving no ambiguity about parameter expectations. Nothing more is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists Android phones registered to the account, enumerating the specific fields returned (id, name, enabled state, default sender, last check-in, message counts). It uses a specific verb 'list' and a clear resource, distinguishing it from send_sms by explicitly noting it sends no SMS.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use triggers: 'when a send fails with a device error, when the user asks which phone will be used, or when you need a device_id.' It also implies exclusion from sending by stating 'sends no SMS,' giving clear guidance on when not to use it (for sending). This is complete and actionable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• send_smsA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare openWorld=true and idempotentHint=false, and the description adds real-account consequences (message counts against plan quota), automatic device/account selection, silent SIM id failure, and non-retry advice. It falls slightly short of explaining the strict return/error shape on failure, but on the whole the description adds meaningful context beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three dense but well-structured paragraphs grouped by concern — send/cost, device selection rule, recipients, scheduled send, SIM fallback. Every sentence carries operational value; no filler or repetition despite the length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers a surprising amount of edge cases: plan limits, max recipients, quota-capped segments, queue-enabled scheduling, a specific source device, account defaulting, a boolean for queue failure in get_messages, and a full sibling handoff. The only missing piece is what the success response looks like — but the omission is defensible because get_messages is assigned for delivery status.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers 100% of parameters, and the prose substantially raises the bar: E.164 validation, recipients counted as one each, long messages over 1600 chars segmented and each segment counted, omit device_id by default and source it from list_devices, sched_at requires future+queuing, sim_subscription_id silently ignored. This is much more than a restated schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb–object pairing: “Send an SMS” through the user's own textbee account/Android phone. Scope is sharply delimited from siblings: unlike list_devices/get_messages, this sends. Distinctive constraints are stated (E.164 recipients, auto-selected phone, explicit device_id only when the user names a device).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit operational rules: omit device_id in almost every case; pass it only when the user asks for a specific phone; do not speculative-send, do not retry a send that may have succeeded; use sched_at only with queuing, and route batch status to get_messages. No ambiguity about when to use the tool or its siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.