Server Quality Checklist

Tool Scores

• get_familyA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses read-only behavior, the unknown-slug response (found:false, never an error), and details what data is returned. It does not cover auth requirements or rate limits, but for a one-parameter read-only lookup, the transparency is strong.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loading the main purpose and then providing behavioral details. Every phrase contributes value, with no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool with no output schema, the description fully covers what the tool returns, how to obtain the slug, and error behavior. It leaves no significant gaps for an agent to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already describes the slug parameter with an example, but the description adds crucial context by specifying the slug comes from search_models or lookup_part pageUrl. This gives the agent a clear source for valid slugs, going beyond the schema's basic label.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Fetch') and clearly defines the resource ('full source-backed record for one product family by slug') and its contents. It distinguishes itself from siblings by focusing on a single family's comprehensive record, unlike search or part-related tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explains when to use this tool—when you have a slug from search_models or lookup_part pageUrl—and clarifies that an unknown slug returns found:false with a hint. However, it does not explicitly state when to prefer alternatives like bulk_check or list_vendors, so it lacks explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• bulk_checkA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully carries the behavioral burden. It discloses read-only, no auth, never partial/destructive behavior, per-row results, summary counts, and the found:false behavior for unknown or malformed entries. This exceeds typical descriptions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two dense sentences deliver all essential information: scope, alternatives, return shape, safety, and edge-case behavior. No word is wasted, and key details are front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema exists, but the description spells out the per-part row fields (status, EOSL date, source URL, page URL) and summary counts. Combined with full parameter documentation and explicit edge-case handling, the tool is fully understood without additional context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already covers the parameter thoroughly with min/max items, deduplication, and punctuation tolerance. The description adds only the 'up to 200' and 'in one call' context, which is marginal on top of the schema. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool checks up to 200 part numbers in one call, names the specific resource (part numbers), and immediately distinguishes it from lookup_part for single-part use. The verb 'check' plus the listed return fields leaves no ambiguity about purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says to use this for multiple part numbers and names lookup_part as the alternative for a single part. The 200-part limit and read-only/no-auth note also give concrete context for safe use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_vendorsA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description takes full responsibility for behavioral disclosure. It explicitly states 'Read-only, no parameters' and describes the output (family counts and vendor page URLs), fully conveying the tool's behavior for a simple list operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, with the purpose front-loaded and usage guidance following. Every sentence earns its place; no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no params, no output schema), the description covers all relevant aspects: purpose, usage, safety (read-only), and expected output. It is complete for an AI agent to select and invoke this tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. The description confirms 'no parameters' and adds no unnecessary detail, which is appropriate for a parameterless tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'List all vendors tracked by EOSL.ai with family counts and vendor page URLs.' It uses a specific verb ('list') and resource ('vendors'), and distinguishes itself from sibling tools by noting it should be used first to check vendor coverage.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance is provided: 'Use this first to check whether a vendor is covered at all; for specific models use search_models, for a part number use lookup_part.' This clearly tells when to use this tool versus alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_modelsA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It discloses that the tool is 'Read-only,' returns 'up to 10 families,' and includes specific returned fields ('status, EOSL window, and page URL'). This gives the agent a solid understanding of side effects and output.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is tight and front-loaded, using two sentences to convey purpose, matching behavior, read-only nature, output limit, return fields, and alternatives. Every sentence adds value; no fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with no output schema, this description is complete: it explains what the tool does, how to use it, what it returns, and when to use other tools. It covers discovery, matching, result size, and output fields, so an agent has enough context to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already fully documents the single 'query' parameter (coverage 100%), so the baseline is 3. The description adds meaningful semantics beyond the schema by specifying 'case-insensitive substring' and providing an example ('nexus 9300'). This enhances parameter understanding without being necessary for basic use.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Search tracked product families by vendor, product line, or series name.' It uses a specific verb ('Search') and resource ('product families'), and differentiates from siblings by mentioning lookup_part and get_family for other use cases.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says when to use this tool: 'Use this for discovery when you have a name.' It also names alternatives with their appropriate contexts: 'for an exact part number use lookup_part, and use get_family with a returned slug for the full record.' This is clear, actionable guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lookup_partA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses behavior: read-only, no auth, matching algorithm (exact, punctuation-insensitive, Fortinet aliases), family-name fallback with matchedVia flag, and found:false behavior. No contradictions with annotations since none exist.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is information-dense but well-organized, front-loading purpose and scope, then return fields, then matching details. Each sentence contributes unique value with no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity, the description is remarkably complete: input, output fields, matching behavior, fallback, no-auth requirement, and no-guarantee case. Without an output schema, it sufficiently prepares an agent for invocation and interpretation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although schema covers 100% of parameters, the description adds meaning beyond it: vendor restricts match, part_number can be a model name, and matching is exact then fallback. It gives concrete examples (FG-60E, WS-C3850) and explains the vendor hint's role, greatly enhancing schema info.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool looks up one hardware part number or model name in the EOSL.ai database, using a specific verb and resource. It clearly distinguishes from siblings by mentioning bulk_check for multiple parts and specifying the matching fallback behavior.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'for many parts use bulk_check', an alternative tool. It also describes when a family-name match occurs and that no match returns found:false, giving agents clear conditions for use versus not using the tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.