Server Quality Checklist

Tool Scores

• get_user_ordersC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, yet the description only states it fetches orders. It omits behavioral traits such as pagination, ordering, read-only nature, or authentication requirements. This leaves significant gaps for an agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no wasted words. It is efficiently front-loaded, but could include more context without becoming verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with one parameter and no output schema, the description is minimally complete. However, missing details like response format or pagination limit its utility. Could be improved with brief behavioral info.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (parameter 'userId' described as 'User ID, such as usr_001'). The description adds no extra semantic meaning beyond the schema example. Baseline 3 applies as schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Fetch all orders') and the specific resource ('for a single user'). It distinguishes from sibling tools like 'get_orders' by limiting scope to one user. However, it does not explicitly frame this in contrast to alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when or when not to use this tool. It does not mention alternatives or exclude cases like needing orders for multiple users. The implication is weak and relies on sibling names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_user_business_profileA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden. It uses 'Fetch' suggesting a read-only operation but does not confirm non-destructiveness, error behavior (e.g., if user not found), authentication requirements, or performance implications.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that includes the verb, resource, and a list of included data. Every word earns its place, with no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description fairly comprehensively lists the data returned (subscription, orders, invoices, etc.). However, it lacks details on pagination, nesting, or whether all data is always present. Still, for a single-user fetch, it is mostly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter 'userId' has a description in the schema ('User ID, such as usr_001.'), giving 100% schema coverage. The tool description does not add additional meaning beyond what the schema provides, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Fetch' and the resource 'one user', listing the included data: subscription, orders, invoices, payments, refunds, and business metrics. It distinguishes from sibling tools like 'get_indian_users' (geographically filtered) and 'get_orders' (orders only).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for fetching a full business profile but does not explicitly state when to use it versus siblings (e.g., 'get_business_summary' for a summary). No when-not-to-use or alternative guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_business_summaryA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description should disclose behavioral traits such as read-only nature or potential performance impact. It only states what data is aggregated, without detailing side effects or access requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence front-loads the verb and resource, with no extra words. Every part earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and no parameters, the description should elaborate on the shape of the return value (e.g., counts, totals) or pagination. The current description is adequate but lacks sufficient detail for a complete understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and schema coverage is 100% trivial. The description adds value by listing the included entities (users, orders, etc.), which helps the agent understand the scope of the aggregate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the verb 'Fetch' and a specific resource 'aggregate business metrics' across multiple entities, clearly distinguishing it from sibling tools that target individual entities like users or orders.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for high-level overviews but does not specify when to choose this over sibling tools like get_orders or get_user_business_profile, nor does it mention any prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_ordersA
  3.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided; description only states 'Fetch all business orders' without disclosing potential performance impacts, pagination, or data freshness. Lacks behavioral details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence with no redundant words. Front-loaded with action and resource, followed by usage guidance. Efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Adequate for a simple fetch-all tool with no parameters or output schema, but lacks behavioral details (e.g., ordering, limits) that would improve completeness given no annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Zero parameters, so schema coverage is 100% trivially. Description adds no parameter-specific info but highlights purpose, meeting baseline for no-param tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states verb 'Fetch' and resource 'all business orders', distinguishing from sibling tools like get_user_orders by scope and use cases.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Lists specific analyses (revenue, product category, etc.) for which this tool is suitable, but does not explicitly state when not to use it or mention alternatives like get_user_orders.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_indian_usersA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It only says 'Fetch' but does not disclose any behavioral aspects like authentication needs, rate limits, or whether it returns a complete list or paginated results. For a simple read-only tool, this is minimally adequate but lacks depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one sentence, front-loading the core action and resource, then adding use-case examples. Every word is meaningful with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters and no output schema, the description sufficiently covers its purpose and usage context. It could be improved by noting the output format or any limitations, but it is complete enough for a simple list-fetch operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. The description adds value by specifying 'all Indian test users', confirming the lack of filtering. This aligns with the empty schema and provides clarity beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Fetch') and the specific resource ('all Indian test users'), which immediately distinguishes it from sibling tools like get_orders (orders) or get_user_business_profile (single user profile).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description lists use cases (customer segmentation, location analysis, plan analysis, profile lookup), providing clear context for when to use this tool. However, it does not explicitly state when not to use it or contrast with alternatives like get_user_orders.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.