Server Quality Checklist

Tool Scores

• check_segment_sizeA
  3.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full responsibility. It states a readonly query behavior but does not disclose any potential side effects, permissions, or limitations (e.g., rate limits, data freshness). The description is minimal and lacks depth for behavioral understanding.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, with the first sentence stating the function and the second providing usage context. It is front-loaded and contains no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple and the description covers the basic function and usage context. However, without an output schema or annotations, the description does not specify the return format or details like whether it returns all segments or only accessible ones. This leaves some ambiguity for the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so schema coverage is 100%. Since there are no parameters, the description naturally adds no parameter-specific info. Baseline for 0 parameters is 4, and the description meets that.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action: 'Look up audience segments to see how many subscribers each contains.' This is a specific verb-resource combination. The sibling tools (add_subscriber, get_keyword_id, list_campaigns, list_subscribers) are distinct, so there is no ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: 'Use BEFORE drafting an SMS blast so the agent can scope the campaign correctly.' This tells the agent when to invoke the tool, though it does not explicitly mention when not to use it or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_subscriberA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided; description adds that tool returns created subscriber and mentions compliance requirement, but lacks details on errors, rate limits, or side effects beyond schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no redundant information. First sentence states purpose, second adds return value and compliance context. Efficient and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 3-parameter creation tool with no output schema, the description covers essential aspects: action, compliance, return value. Lacks detail on nested 'properties' object but sufficient for basic use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, and description repeats schema descriptions without adding new insight beyond rephrasing. Baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states action (add), resource (phone number to SMS list), and specific mechanism (using keyword ID). Emphasizes TCPA/CTIA compliance, distinguishing it from sibling tools like list_subscribers or get_keyword_id.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides context that keyword_id ties to consent flow, implying need for prior existence, but does not explicitly state when to use or alternatives. Lacks explicit when-not guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_campaignsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without any annotations, the description implies a read-only operation but does not explicitly state behavioral traits like whether it is safe or if there are rate limits. The term 'retrieve' suggests no side effects, but more detail would improve transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is only two sentences (22 words), front-loaded with the action, and every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with one optional parameter and no output schema, the description adequately conveys what the tool does and when to use it, though it does not detail the return format.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers the only parameter (limit) with 100% description, making the baseline 3. The tool description adds no additional meaning beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Retrieve' and the resource 'recent SMS marketing campaigns and their deployment status', which distinguishes it from sibling tools like list_subscribers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says it is 'useful for an AI agent to check what has already been sent before drafting a new one', providing clear context for when to use the tool, though it does not mention when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_subscribersA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided. The description implies a read operation but does not disclose limitations, pagination, or authentication needs. It is adequate for a simple list tool but could be more explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no wasted words. It front-loads the purpose and provides a usage hint efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity (1 param, no output schema), the description is complete: it explains the output (IDs and phone numbers), specifies 'active' subscribers, and gives a usage context. No gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers the single parameter 'limit' with a description. The tool description does not add extra meaning to the parameter beyond the schema, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Query', the resource 'active SMS subscriber list', and what it retrieves ('subscriber IDs and phone numbers'). It distinguishes from sibling tools like add_subscriber (add) and check_segment_size.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions it is 'useful for downstream targeting decisions', implying a use case. However, it does not explicitly state when not to use or compare to alternatives like check_segment_size.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_keyword_idA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses the tool's behavior: it performs a lookup and returns keyword ID and reply message. No hidden side effects or constraints are omitted.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, each essential: the first states what the tool does, the second explains why it matters. No extraneous content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple lookup tool with no parameters and no nested objects, the description provides sufficient context: it lists returned fields (ID and reply message) and ties to a sibling tool. No output schema exists, but the description covers the output.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the description adds no parameter info beyond what the schema provides (none needed). Schema coverage is 100%, meeting the baseline for 0 parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool 'looks up all SMS opt-in keywords' and returns 'each keyword's ID and reply message,' clearly distinguishing it from sibling tools like add_subscriber.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description directly guides usage by stating 'Use the returned ID when calling add_subscriber,' linking the tool to a sibling and explaining the purpose, though it doesn't explicitly state when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.