Server Quality Checklist

Tool Scores

• compare_partsC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description must carry the full burden. It states 'compare' which suggests read-only behavior, but does not disclose side effects, permissions, or return value structure. Minimal transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely short (one sentence) but lacks necessary details. Concise but under-specified. Not enough structure for effective use.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given two parameters, no output schema, and no annotations, the description is severely incomplete. It omits what 'compare' returns, how 'criteria' is used, and any usage context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% and the description does not explain any parameters. 'parts' and 'criteria' are undefined. The description adds no meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The verb 'compare' and resource 'part numbers or candidates' are clear. It specifies a numeric range (2-10). However, it does not explain what 'compare' entails (e.g., output similarities, differences), so it's slightly vague.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus siblings like 'search_parts' or 'suggest_alternates'. The description implies comparing multiple parts but does not specify context or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lookup_partC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description bears full responsibility for behavioral disclosure. It does not address read-only nature, authentication requirements, rate limits, or error handling (e.g., part not found).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, which is concise but lacking essential detail. It is front-loaded but fails to earn its place by omitting crucial information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema is provided, and the description does not specify what information is returned (e.g., specs, pricing, availability). For a lookup tool, this is a significant gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain any of the three parameters (partNumber, kind, supplier). The mention of 'manufacturer or supplier' is ambiguous and does not clarify the 'kind' enum or 'supplier' field.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action (look up) and resource (manufacturer or supplier part number). It is specific but does not differentiate from sibling tools like search_parts, which may have overlapping functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives such as search_parts or suggest_alternates. The description lacks any context for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_partsC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description must disclose behavioral traits. It does not mention result format, pagination, error handling, or side effects. The description is too minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very short (one sentence) but lacks necessary detail. While concise, it sacrifices informativeness. It is not front-loaded with critical guidance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (6 parameters, nested objects, no output schema), the description is severely incomplete. It fails to explain return values, parameter semantics, or behavioral context. Essential information is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, yet the description provides zero explanation of any of the 6 parameters, including nested objects like constraints and visualHints. The agent has no guidance on parameter usage beyond schema names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it searches for engineering parts across configured distributor and marketplace suppliers, which is a specific verb+resource. It differentiates from sibling tools like lookup_part or suggest_alternates.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives. Sibling tools exist but are not mentioned. The description lacks context on prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• extract_visual_part_hintsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided; the description only mentions that raw images are not processed, but fails to disclose behavioral traits such as side effects, rate limits, or output format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two front-loaded sentences, each earning its place by clarifying purpose and constraints, though it sacrifices parameter details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 15 parameters, nested objects, no output schema, and no annotations, the description is severely incomplete, lacking return value info and parameter guidance.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage and no parameter explanations in the description, the AI agent receives no additional meaning beyond parameter names, which is insufficient for a tool with 15 complex parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool normalizes image-recognition observations into searchable part hints, distinguishing it from siblings like search_parts (text-based) and compare_parts.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by requiring observations from Codex/Claude vision, but does not explicitly specify when to use this tool versus alternatives, nor does it list exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• enrich_bomB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It states it enriches with data but does not disclose whether it modifies the input, returns enriched data, or any side effects. No mention of authentication, rate limits, or what happens with missing parts.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence of 11 words, very concise. It avoids redundancy. However, it is so brief that critical details are omitted, balancing conciseness against completeness. For a 3-param tool, it could be slightly expanded without losing conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (nested array input, optional suppliers list, no output schema), the description is severely incomplete. It provides no information about return values, error handling, or parameter constraints. An AI agent would lack guidance on how to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, yet the description adds no meaning to the parameters. It does not explain 'items', 'suppliers', or 'pricingQuantity'. The schema provides basic structure, but the description fails to clarify expected input format, defaults, or constraints beyond what schema implies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: enrich BOM-like rows with supplier availability, pricing, and sourcing caveats. The verb 'enrich' and resource 'BOM-like rows' are specific, and the enrichments are listed. It distinguishes from siblings like 'lookup_part' (single part lookup) and 'compare_parts' (comparison) by focusing on batch enrichment.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you have a list of parts needing supplier data, but no explicit guidance on when to use this over alternatives like 'lookup_part' or 'suggest_alternates'. No when-not-to-use or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• suggest_alternatesB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits. However, it only says 'suggest', which implies a read-only operation, but does not confirm lack of side effects, rate limits, or other critical behaviors for a mutation-free tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is front-loaded with the action and efficiently conveys core purpose without unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 3 parameters, no output schema, and no annotations, the description is too brief. It omits return value expectations, parameter details, and behavioral context, leaving the agent with significant gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, meaning the description adds no explanation for the 3 parameters. Although 'reason' enum overlaps with listed contexts, the description does not explicitly map to the parameter, and 'mustMatch' is entirely unmentioned.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('suggest alternate parts') and the specific contexts (unavailable, costly, obsolete, second-source), making the purpose distinct from sibling tools like compare_parts or search_parts.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool by listing triggering conditions (unavailable, costly, etc.), but does not explicitly state when not to use it or compare it to alternative tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.