Server Quality Checklist

Tool Scores

• compare_carriersA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds behavioral context beyond the input schema: it states the tool is unaunthenticated and carrier-neutral (does not declare a winner). However, with no annotations provided, the description carries full burden. It does not disclose pagination, error handling, or what happens when no carriers match, leaving some behavioral ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is four sentences, each serving a distinct purpose: stating the action, listing carriers, noting optional narrowing, explaining neutrality, and indicating auth status. No extraneous words; it is front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has two optional parameters and no output schema, the description covers the key aspects: what it returns, the carriers involved, optional filtering, neutrality, and authentication. It is complete enough for a simple comparison tool, though additional details about the structure of the comparison or when to use it vs. siblings would elevate it further.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already has 100% coverage with descriptions for both parameters. The description adds 'narrow the result' which clarifies the role of the parameters but does not provide new semantic details beyond the schema. Baseline for high coverage is 3, and the added value is marginal.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns a structured comparison of the five major individual disability carriers, listing them by name. It also mentions optional narrowing by profession and priority, and contrasts with sibling tools by specifying carrier-neutral framing. This effectively differentiates it from siblings like estimate_benefit_cap_gap or quote_request.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no explicit guidance on when to use this tool versus alternatives. While it mentions carrier-neutral framing and unaunthenticated access, it does not indicate scenarios where siblings like quote_request or estimate_benefit_cap_gap would be more appropriate. A user is left to infer usage from the tool's purpose alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_education_articleA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses return type (metadata + link) and auth status ('Unauthenticated'), but does not cover potential side effects, rate limits, or data freshness.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with purpose, no redundant information. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers purpose, return type, and auth for a simple tool. Could mention matching behavior (e.g., exact or substring) for completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema already describes the only parameter as 'Topic slug or article title substring.' Description adds example topics but no additional semantic value beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the verb 'Retrieve' and the resource 'named education article'. Examples of topics differentiate it from sibling tools like compare_carriers or quote_request.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies usage for retrieving education articles by topic, but does not explicitly specify when to use this tool versus alternatives or provide any exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_specialty_guideA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Since no annotations are provided, the description carries the full burden of behavioral disclosure. It reveals that the tool is 'Unauthenticated' (implying a read-only, public operation) and describes the return format. While it could mention potential error cases (e.g., invalid profession), it adequately discloses key behavioral traits beyond what the schema provides.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: two short sentences that are front-loaded with the key action and resource. Every word adds value, with no repetition or unnecessary details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple lookup tool with one parameter and no output schema, the description is fairly complete. It states the input purpose, the output type (structured metadata plus link), and authentication status. However, it omits details about error handling (e.g., what happens if profession is not found) and does not specify whether the guide is versioned or time-sensitive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage for the single parameter is 100%, including a helpful description with examples. The tool's description does not add additional meaning beyond what is already in the schema. Therefore, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Retrieve', the resource 'Seaworthy Insurance Agency coverage guide', and specifies it's for a specific profession or medical specialty. It also indicates what is returned (structured metadata plus link). This distinguishes it from sibling tools like compare_carriers or estimate_benefit_cap_gap.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide explicit guidance on when to use this tool versus alternatives. It does not mention when not to use it or suggest other tools for related tasks. Usage is only implied by the function's name and description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• estimate_benefit_cap_gapA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses that the tool is 'Pure math, no external calls. Unauthenticated.', which tells the agent it's a safe, read-only operation with no side effects. This is sufficient for a simple computation, though it could mention error handling or output format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: two sentences. The first sentence immediately states the purpose (verb+resource), and the second adds behavioral context. No unnecessary words; every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is clear for a simple math tool, but it lacks details about the return value or output format (no output schema exists). It also doesn't explain the calculation logic beyond 'gap between cap and target replacement'. For a tool with multiple parameters affecting the result, this leaves some ambiguity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides 100% coverage with descriptions for each parameter. The tool description does not add additional meaning beyond the schema; it only references the overall purpose. Baseline is 3 when schema coverage is high and description adds no extra parameter context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Compute' and clearly identifies the resource: the income replacement gap between a group LTD benefit cap and a target replacement. It distinguishes itself from sibling tools (e.g., compare_carriers, quote_request) which serve different purposes, and adds clarifying context like 'Pure math, no external calls.'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implicitly indicates when to use: for calculating the benefit cap gap. It mentions 'Pure math, no external calls. Unauthenticated.' which assures the agent it's a safe computation. However, it does not explicitly state when not to use or provide alternatives like sibling tools, so it's clear but lacks explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• quote_requestA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses the write operation (creates a Lead), the follow-up window (1 business day, 8am-5pm Pacific), and restrictions on data collection. It does not explicitly state idempotency, but this is acceptable for a lead submission tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three concise sentences, each adding value: main purpose, consent requirement, follow-up timing, and data restrictions. It is front-loaded and efficient with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (12 parameters, 9 required, no output schema), the description covers key aspects: purpose, consent, forbidden data, and follow-up. It omits return value details but that is acceptable without an output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds overall context like consent and data restrictions but does not provide new meaning for specific parameters beyond the schema's own descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb (submit) and resource (disability insurance quote-comparison request to Seaworthy Insurance Agency sales pipeline), and it specifies that it writes a Lead to Salesforce. This distinguishes it from sibling tools like compare_carriers or get_education_article.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on required user consent before submission and prohibited data (SSN, medical history, banking details). It also mentions follow-up timing. However, it does not directly compare to sibling tools or state when to use this versus compare_carriers, though the context implies its unique purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_ridersA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description notes the tool is 'Unauthenticated,' which is a behavioral trait. It doesn't disclose other traits like rate limits or side effects, but for a read-only list operation, the provided information is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that front-loads the purpose and lists all riders. Every word contributes meaning, with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters, no output schema, and is a simple listing, the description fully specifies the output content by naming all riders and stating it provides definitions and trade-offs. No gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters, so schema coverage is 100% trivially. Following the guideline, baseline is 4, and the description adds no parameter details as none are needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it returns structured definitions and trade-offs for specific riders, listing them explicitly. It distinguishes itself from sibling tools like compare_carriers or quote_request, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implicitly indicates when to use: when needing definitions of the listed riders. It doesn't provide explicit when-not or alternatives, but the context is clear given the tool's simplicity and listing of riders.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.