Server Quality Checklist

Tool Scores

• get_usage_detailsA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the need for login credentials, but does not explicitly state that this is a read-only operation, what happens with invalid credentials, or any side effects. For a get tool, more transparency about its non-destructive nature would be expected.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, directly states the purpose, and includes the key prerequisite. It is front-loaded with the action and scope, with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity and lack of output schema, the description adequately covers the returned content (data, call, text usage) and the time scope (current billing period). It also mentions required credentials. However, it could be more complete by explicitly noting the read-only nature or possible absence of output for empty usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with both username and password described. The description does not add additional meaning beyond the schema, so a baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool gets a detailed breakdown of data, call, and text usage for the current billing period. This specific verb+resource+scope distinguishes it from sibling tools like get_account_overview or get_bill_history.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you need current usage details, but it does not explicitly say when to use this tool versus alternatives. The mention of AT&T login credentials is a prerequisite, not usage guidance against other tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_account_overviewA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses a key behavioral trait: AT&T login credentials are required. The action 'get' implies a read-only operation, but it doesn't detail error handling, rate limits, or what happens with invalid credentials.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the primary purpose, no filler. The second sentence adds the prerequisite efficiently, making the description concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with two parameters and no output schema, the description adequately covers the purpose, included summary contents, and the auth prerequisite. It lacks usage guidance and return format details, but this is acceptable given the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with username and password already described. The description's mention of 'AT&T login credentials' simply reinforces the schema but adds no new parameter meaning beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it retrieves a summary of the AT&T account, listing specific contents (current plan, balance due, usage summary). This specific verb+resource distinguishes it from siblings like get_usage_details and get_bill_history, which target narrower aspects.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for obtaining an overall account summary but provides no explicit guidance on when to choose this over sibling tools like get_usage_details or get_bill_history. The prerequisite (requires AT&T login credentials) is mentioned but not as a usage guideline.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_bill_historyA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It does disclose the authentication requirement, which is a meaningful behavioral trait, but it does not confirm read-only operation or explain any side effects. The word 'View' implies read-only but is not explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences, front-loaded with the core purpose, and contains zero extraneous information. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple view tool with only two parameters and no output schema, the description adequately covers purpose and prerequisites. It mentions that bills include dates and amounts, giving the agent a reasonable expectation of output. It could be enhanced by noting any limitations (e.g., date range or bill status), but it is fundamentally complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, and the parameter descriptions already explain username and password. The description adds the context that these are AT&T login credentials, but that's redundant with the schema. Since schema fully documents parameters, the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'View past AT&T bills including dates and amounts.' This uses a specific verb ('View') and resource ('past AT&T bills'), and it distinguishes from sibling tools like get_usage_details (usage data) and pay_bill (payments).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions a key prerequisite ('Requires AT&T login credentials') but does not explicitly compare with alternatives or state when to use this tool over siblings. Usage context is implied but not directly communicated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_upgrade_eligibilityA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden. It discloses authentication requirements, which is useful, and hints at the output (eligible date and fees). However, it does not explicitly state whether this is a read-only operation or if there are any side effects, relying on the verb 'check' to imply safety.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence that front-loads the action and purpose, then adds relevant details (eligible date, fees, credentials). There is no fluff or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with three parameters and no output schema, the description is mostly complete: it explains the purpose, output highlights, and authentication. However, it does not describe the behavior when the optional line_number is omitted, nor does it fully detail the return structure beyond eligible date and fees, leaving some room for ambiguity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description covers 100% of parameters, so the baseline is 3. The description adds little beyond the schema, only implicitly connecting 'line' to the optional line_number parameter. It does not provide additional semantic detail about the parameters, so it remains at the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly specifies the tool's function: checking upgrade eligibility for a line on an AT&T account, including eligible date and upgrade fees. This is a specific verb+resource+scope that distinguishes it from sibling tools like get_account_overview or pay_bill.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool (to check upgrade eligibility) and notes a prerequisite (AT&T login credentials). However, it does not explicitly mention alternatives or when not to use it, which would have made it a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pay_billA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It clearly discloses that the final payment confirmation must be completed manually (indicating the tool only initiates, not completes), and that login credentials are required. This adds meaningful security and process context beyond a simple action statement.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the main purpose, and includes only essential additional information (security requirement, auth requirement). Every sentence contributes meaning, with no repetition or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema and no annotations, the description provides adequate context: it explains the action, the key manual confirmation step, and credential requirements. It could provide more detail about post-call expectations (e.g., response format), but the manual confirmation hint gives the agent a clear workflow signal.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters (username, password, amount, payment_method). The description does not add additional parameter-level semantics beyond what is in the schema, such as constraints on amount or how payment_method is used. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Initiate a one-time bill payment'), the resource ('AT&T account'), and differentiates this as the only payment action among siblings, which are all read-only (get_account_overview, get_usage_details, get_bill_history, check_upgrade_eligibility). This makes the purpose unambiguous and distinguishable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool (to pay a bill) and provides clear context about the payment process. However, it does not explicitly name alternatives or state when not to use it. The manual confirmation note serves as a usage warning, but no explicit when-not or alternative tool comparison is present.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.