Server Quality Checklist

Tool Scores

• get_site_infoB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, and the description only states 'Get metadata' without disclosing behavioral traits such as read-only nature, destructive potential, authentication requirements, or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that efficiently conveys the tool's purpose. However, it could be slightly expanded to include the company_id parameter explanation without losing conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema, no annotations), the description provides a reasonable overview. However, it lacks details about the company_id parameter and potential return structure, making it only minimally adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema describes 'company_id' as a required string with no description (0% coverage). The tool description does not explain what the company_id represents or how to obtain it, leaving the parameter's semantics unclear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Get metadata about a company' and lists specific metadata types (platform, agent instructions, available tools), distinguishing it from siblings like discover_companies or search_products.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives like discover_companies, or when not to use it. The description does not mention preconditions or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_categoriesA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions listing all categories but doesn't address pagination, result format, or potential side effects. 'List' implies read-only, but more detail on limits or ordering would improve transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences efficiently state purpose and usage context. Every word serves a purpose with no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with one parameter and no output schema, the description covers the essential: what it returns, the required parameter, and a use case. Minor omissions like whether results are ordered or limited do not significantly hinder completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It only says 'for a specific company' about company_id, adding minimal meaning beyond the schema's type 'string' and requirement. No format, validation, or example is provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List' and the resource 'product categories / collections' with scope 'for a specific company'. It effectively distinguishes from sibling tools like search_products and get_product which focus on individual items or different operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes 'Useful for browsing' which implies suitable for exploratory contexts. However, it lacks explicit when-not-to-use guidance or alternatives, though the sibling tools provide natural differentiators.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• discover_companiesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the core behavior (search companies, return ranked IDs) and the network scope. However, it does not mention rate limits, authentication needs, or other side effects. The description is adequate but not rich in behavioral detail.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: two sentences with no wasted words. The first sentence covers purpose and output, the second gives usage guidance. It is well-structured and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and no annotations, the description adequately explains the tool's function, input, and output (company IDs). It also relates to sibling tools via the usage hint. Minor absence: not detailing the format of the ranked matches beyond IDs, but still sufficient for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds context for the query parameter ('natural-language query') and the overall goal, but it does not provide additional meaning beyond the schema's existing descriptions for limit and category.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool finds e-commerce companies matching a natural language query and returns ranked matches with company IDs. It distinguishes from siblings by explicitly advising use when the user asks about a product type or brand, setting it apart from product search tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use guidance ('Use this first when the user asks about a product type or brand') and explains the output's role in subsequent product searches. However, it lacks explicit statements about when not to use it or direct alternatives beyond the implied sequence.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_productA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, description carries full burden. It discloses it returns variants, images, descriptions, stock, pricing. This is transparent for a read operation, though no mention of side effects (likely none). Could be clearer about read-only nature, but still strong.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single, front-loaded sentence with no filler. Every word adds meaning (Get, full details, specific product, enumerated fields). Ideal conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Tool has 2 simple required params, no output schema, no nested objects. Description covers main return fields. For a straightforward retrieval operation, it is complete enough. Could mention if product_id supports other formats, but not a major gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 50% (company_id no description, product_id described). Description doesn't elaborate on company_id or how to obtain it. Product_id description matches schema ('ID or slug from search results'). Baseline is 3 given moderate coverage; description adds minimal extra value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Get full details for a specific product' with specific verb (get) and resource (product), and enumerates attributes (variants, images, descriptions, stock, pricing). This distinguishes it from sibling tools like search_products (searching) and list_categories (categories).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when needing full product details, and sibling tool names provide context for alternatives. No explicit when-not-to-use guidance, but for a simple retrieval tool, context is sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_productsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Despite no annotations, the description discloses key behavioral traits: returns real-time data and lists fields (name, price, stock status, variants, URL). Could mention more (e.g., rate limits, pagination), but current coverage is solid.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core purpose. Every word earns its place; no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Adequately describes returns and provides usage context. With no output schema, missing details on pagination/sorting. Sibling context is leveraged. Overall near-complete for a search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and parameter descriptions are already clear in the schema. The tool description adds no additional semantic detail beyond the schema, so baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action ('Search for products'), resource ('within a specific company'), and provides differentiation from siblings by mentioning real-time data and specifying it is for finding specific products at the matched brand.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises to call after discover_companies, providing clear sequence context. However, it does not explicitly state when not to use or mention alternatives like list_categories for browsing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.