Server Quality Checklist

Tool Scores

• get_bill_historyA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. 'Retrieve' and 'past billing history' communicate that this is a read-only historical lookup, which is useful, but the description does not discuss authentication expectations, response shape, or any limitations. It adds some behavioral context but not comprehensive coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that front-loads the main action and resource, then adds the most relevant output details. There is no wasted wording or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema and no annotations, so the description must cover behavior and result content. It names the key returned data fields and the read-only nature, but it does not mention usage context, alternative tools, or limitations such as how far back history goes beyond the schema's months parameter. It is adequate but has clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the parameters are already fully documented in the schema. The description does not add parameter-specific meaning, but it does help by indicating the returned data fields, which indirectly clarifies the value of the history request. This meets the baseline for full schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Retrieve' and a clear resource, 'past billing history for a Spectrum account', and adds concrete content details: bill dates, amounts, and payment status. This distinguishes it from siblings like get_account_overview and pay_bill without needing further explanation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving billing history, which is distinct from paying a bill or checking outages, but it does not explicitly state when to use this tool versus alternatives or mention any exclusions. No direct routing guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_account_overviewA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the behavioral burden. 'Get' signals a read-only operation and the listed outputs clarify what is returned, but it does not mention side effects, credential sensitivity, or that no account changes are made. This is adequate but lacks deeper behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that is front-loaded with the action and resource, followed by a compact list of returned items. There is no redundant wording or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter read operation, the description covers the main return values and the schema covers inputs. It lacks explicit sibling differentiation and output formatting details, but those are less critical given the tool's low complexity and clear scope.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and both username and password already have clear descriptions. The tool description adds no parameter-level meaning beyond what the schema provides, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Get' with a clear resource, 'Spectrum account overview,' and explicitly lists the returned information: current plan, balance due, payment due date, and data usage summary. This distinguishes it from siblings like pay_bill, check_outages, and schedule_technician.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The contents imply use when an agent needs a summary of account status, plan, balance, due date, or data usage. However, it does not explicitly say when to use this tool over get_bill_history or get_service_details, nor does it mention alternatives or exclusion cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_service_detailsA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the disclosure burden; it reveals that only active services are included and that filtering is optional, but it does not describe the shape of the returned details, auth behavior, or potential side effects. The verb 'Get' implies read-only, which mitigates the gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no filler; the core scope is front-loaded and the optional filter is a natural second sentence. Every part earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read tool with a fully described schema, the description is mostly sufficient, but with no output schema it does not clarify what 'details' will be returned (e.g., plan, status, pricing). It also does not distinguish itself from get_account_overview, leaving some selection ambiguity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description only restates the optionality of service_type and adds no new semantics beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific action ('Get details') and resource ('active Spectrum services'), and enumerates service types to clarify scope. It is immediately distinguishable from sibling tools like pay_bill and check_outages.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for when to use it: when details about active Spectrum services are needed, with optional service_type filtering. It does not explicitly exclude alternatives or route to get_account_overview, so it misses the top tier.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pay_billA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries a substantial burden and does well by disclosing that it only loads the payment form and that final submission requires user confirmation. This prevents an agent from assuming the payment is fully executed, which is critical for a financial action. It falls short of a 5 because it does not describe any response, confirmation token, or what happens after the form is loaded.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is just two sentences and is front-loaded with the core purpose. The note about user confirmation is placed at the end and earns its place by conveying the most important behavioral caveat.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The schema covers inputs well and the description conveys the primary behavioral caveat, but with no output schema the description does not explain what the tool returns or what the next step is after user confirmation. For a financial action this is a meaningful gap, though not a fatal one.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already documents all 4 parameters with 100% coverage, so the description is not required to explain them. It only adds 'the specified amount' regarding the amount parameter, which maps to the schema but provides no new semantic detail beyond it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action ('Initiate a one-time bill payment') and the resource ('a Spectrum account'), making the tool's purpose unambiguous. It also distinguishes itself from the sibling read-only and service-management tools by focusing on payment initiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context: use this tool to start a one-time payment with a specified amount. It does not explicitly name alternatives or exclusions, but among the listed siblings this is the only payment-related tool, so selection is straightforward.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• schedule_technicianA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry behavioral disclosure. It explicitly states that the tool populates the scheduling form and that final submission requires user confirmation, a meaningful workflow trait: the call does not fully complete the appointment and a user step remains. It does not disclose broader side effects, but the key nuance is present.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core purpose and then the critical confirmation caveat. No redundant phrasing or filler; every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 6-parameter action with no output schema and no annotations, the description gives the core purpose and the confirmation requirement but omits what the caller should expect in the result — a form preview, a confirmation screen, or a success message. It also does not fully explain the multi-step nature beyond the confirmation requirement, leaving the agent to infer how to proceed after the call.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3, and the description adds no parameter-level detail beyond the schema. The schema itself documents username/password, contact_phone, preferred_date, and issue_description, though preferred_time's schema text is mislabeled as a date rather than a time; the tool description does not correct this.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource — 'Schedule a Spectrum technician visit' — and scopes it to 'service issues, installation, or equipment problems,' which clearly distinguishes it from the read-only/billing sibling tools like get_account_overview and pay_bill. No ambiguity about what operation is being invoked.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear context for when to invoke: any service issue, installation, or equipment problem requiring a technician. However, it does not explicitly name alternative tools or state when not to use the tool, such as using check_outages for outages instead. Thus it gives clear context without explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_outagesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral disclosure burden. It clearly indicates this is a read-only check operation and discloses the optional authentication path and its effect on results. It doesn't detail return format or side effects, but 'Check' strongly implies a non-mutating query.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence conveys the purpose, scoping method, and optional authentication behavior without any filler. Every phrase adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description doesn't clarify whether at least one of address or ZIP is required, even though the schema marks all parameters optional. With no output schema, the description also fails to describe what the tool returns, leaving the agent uncertain about response shape and how login affects results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already documents each parameter (100% coverage), but the description adds relational meaning: ZIP code or address are alternative location inputs, and username/password are optional login credentials. This helps an agent understand how to combine parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description uses a specific verb ('Check') with a clear resource ('active Spectrum service outages') and scoping method ('by ZIP code or address'). This clearly distinguishes it from sibling tools like pay_bill or schedule_technician.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for when to use the tool: checking outages by location, with optional login for account-specific info. It doesn't explicitly name alternatives or exclusions, but the sibling tools cover obviously different domains, so the intended usage is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.