Server Quality Checklist

Tool Scores

• productA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations provide readOnlyHint and openWorldHint; the description adds extensive behavioral detail: data_source field, special case for BSR on ~17% of catalog, cost structure, miss handling, marketplace specifics (Walmart returns only intelligence blocks), and free lane limitations. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is lengthy but well-structured: front-loaded with purpose, then usage, return details, special cases, cost. Every sentence adds value, though some cost details could be condensed. It earns a 4 for being comprehensive without being rambling.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description thoroughly explains return data (current price, BSR, rating, etc., plus history fields), special cases, supported marketplaces, cost model, and error handling. It covers all likely agent needs for invoking the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% (all parameters have descriptions in schema), baseline 3. The description adds extra context: asin can be Walmart item ID, mode=live requires key, country mappings to marketplaces. This adds value beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Full dossier for ONE known product: its current snapshot plus its observed history,' which is a specific verb and resource. It explicitly distinguishes from sibling tools 'shopping' and 'search' by noting they return current values while this returns history.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use scenarios (e.g., 'user has a specific ASIN... and asks about price history') and lists what it is NOT for ('DON'T USE to discover products from a keyword (use shopping) or to pull a filtered list (use search)'). It names alternatives clearly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• searchA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, but the description adds substantial context: cost limits (1 credit per 25 rows, 30 daily queries), anchor requirement (q or brand), coverage (BSR universe), and that empty results are not billed. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with sections (USE WHEN, DON'T USE, RETURNS, COVERAGE, COST) and is front-loaded with the main purpose. Every sentence adds essential information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (15 parameters, no output schema), the description covers the return fields, cost details, anchor requirement, and scope. It could mention pagination more explicitly, but overall it is quite complete for an agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds value beyond schema by explaining anchor requirement, sort direction logic (bsr ascending, others descending), and literal keyword behavior for q. This enriches parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it is a 'Filtered query over the tracked-product warehouse' and distinguishes from sibling tools (shopping, product) by specifying when not to use it. The verb 'query' and resource are explicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage scenarios: 'USE WHEN the user wants a structured list matching explicit criteria' and contrasts with 'DON'T USE for best X buying advice (use shopping) or for a single known product (use product).' This gives clear when-to-use and alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• shoppingA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations set `readOnlyHint: true`, but the description adds substantial behavioral context: ranking uses observed marketplace signals with no affiliate or sponsored bias; zero results means genuine no match, never a best-effort guess; data is revealed-preference; and details about result grouping, freshness, and citability. This goes well beyond what annotations provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is lengthy but well-structured with clear sections (USE WHEN, DON'T USE, RETURNS, HANDOFF, EXAMPLE, etc.). Every sentence serves a purpose, but some parts (e.g., detailed market-specific language rules) could be condensed. Overall, it balances completeness with organization.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (4 parameters, no output schema), the description is exceptionally complete. It covers behavior, error handling (zero results), handoff patterns, cost details, and language-specific guidance. It leaves no major gaps for an agent to infer.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds extensive parameter guidance: query style (literal keywords, not semantic), language rules (e.g., umlauts, brand+noun pairing), marketplace selection, and handling of zero results. It explains the `group` parameter's modes and the `detail` parameter's cost implications, significantly enriching the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Curated product discovery: a shopping keyword in, a ranked and grouped shortlist out'. It specifies the verb (search/discover), resource (products), and distinguishes from siblings by explicitly stating when to use `product` (when ASIN is known) and `search` (when filtered dataset is wanted).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidelines: 'USE WHEN the user asks 'best X', 'find me a Y under $Z', 'what should I buy', or wants a shortlist' and 'DON'T USE when the product is already identified by ASIN (use product), or when the user wants a filtered dataset rather than a recommendation (use search)'. It also includes handoff instructions to the `product` tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.