Server Quality Checklist

Tool Scores

• get_prestashop_statsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool returns statistics about documents, types, categories, and specialized data, but lacks details on permissions, rate limits, or whether it's read-only. For a tool with zero annotation coverage, this leaves significant gaps in understanding its operational traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded, with the main purpose stated clearly in the first sentence. The second sentence elaborates on return values, which is useful given the output schema. There's no wasted text, making it efficient, though it could be slightly more structured for clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, output schema provided), the description is somewhat complete but lacks depth. It explains what statistics are returned, but without annotations, it misses behavioral context like safety or performance. The output schema helps, but the description should ideally cover more operational aspects to be fully adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description doesn't add parameter details, which is appropriate here. It receives a baseline score of 4 because it doesn't need to compensate for any schema gaps, and the absence of parameters is handled correctly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get statistics about indexed PrestaShop documentation.' It specifies the verb ('Get statistics') and resource ('indexed PrestaShop documentation'), making the function unambiguous. However, it doesn't explicitly differentiate from sibling tools like 'get_prestashop_doc' or 'list_prestashop_docs', which focus on retrieving documents rather than statistics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools or contexts where statistics are needed over raw data, such as for monitoring or summary purposes. Without this, users must infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_prestashop_docsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It only states what the tool does (list files) and mentions parameters/returns, but doesn't describe important behavioral aspects like whether this is a read-only operation, if it requires authentication, rate limits, pagination behavior, or what happens when filters return no results.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (Args, Returns) and uses minimal words to convey the core functionality. The main sentence is front-loaded, though the formatting with separate sections could be slightly more integrated for optimal flow.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no annotations), the description covers basic purpose and parameters but lacks behavioral context. The existence of an output schema means it doesn't need to explain return values in detail, but it should provide more guidance on usage versus siblings and behavioral traits.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds significant value beyond the input schema, which has 0% description coverage. It explains that 'doc_type' filters by specific documentation types (hook, guide, tutorial, etc.) and 'category' filters by categories (basics, development, modules, etc.), providing essential semantic context that the bare schema lacks.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List' and resource 'PrestaShop documentation files', providing a specific purpose. However, it doesn't distinguish this tool from its sibling 'search_prestashop_docs' or 'list_prestashop_hooks', which would require explicit differentiation to earn a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'search_prestashop_docs' or 'list_prestashop_hooks'. It mentions filtering parameters but doesn't explain when filtering is appropriate or when other tools might be better suited for specific use cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_prestashop_hooksB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions 'full-text search' and 'formatted search results,' which gives some context about the search method and output format. However, it lacks critical details: whether this is a read-only operation, any rate limits, authentication requirements, pagination behavior, or what happens if no results are found. For a search tool with zero annotation coverage, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded: the first sentence states the core purpose, followed by clear sections for Args and Returns. Every sentence earns its place—no redundant information. It's appropriately sized for a search tool with 3 parameters, making it easy to scan and understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (search with filters), no annotations, and an output schema exists (so return values are documented elsewhere), the description is partially complete. It covers the purpose and parameters adequately but lacks behavioral context (e.g., safety, limits) and usage guidelines relative to siblings. The output schema handles return details, but the description should still address when to use this tool and any operational constraints.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the schema provides no parameter descriptions. The description compensates by explaining each parameter: 'queries: List of search terms (maximum 3)', 'hook_type: Filter by hook type (display, action)', and 'origin: Filter by origin (core, module, theme)'. This adds meaningful semantics beyond the bare schema, covering all 3 parameters. However, it doesn't provide examples, format details (e.g., case sensitivity for hook_type), or explain the 'null' default values.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Search PrestaShop hooks using full-text search.' This specifies the verb (search), resource (PrestaShop hooks), and method (full-text search). It distinguishes from siblings like 'list_prestashop_hooks' (which likely lists all hooks without search) and 'get_prestashop_hook' (which likely retrieves a specific hook). However, it doesn't explicitly contrast with 'search_prestashop_docs', which searches documentation rather than hooks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when to choose this over 'list_prestashop_hooks' (e.g., for filtered vs. complete lists) or 'search_prestashop_docs' (e.g., for hooks vs. documentation). The only implied usage is for searching hooks with specific filters, but no explicit when/when-not rules or sibling comparisons are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_prestashop_hooksB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool lists hooks and returns them organized by type and origin, but it doesn't cover critical aspects like whether it's read-only, potential rate limits, authentication needs, or error handling. For a tool with no annotations, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded, starting with the core purpose, followed by clear sections for arguments and returns. Every sentence adds value without redundancy, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity, two parameters with 0% schema coverage, no annotations, but an output schema exists, the description is reasonably complete. It covers the purpose, parameters, and return structure, though it lacks behavioral details like safety or performance traits, which slightly reduces completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains the parameters 'hook_type' and 'origin' with their filtering purposes and possible values (e.g., 'display, action' for hook_type, 'core, module, theme' for origin), adding meaningful context beyond the bare schema. However, it doesn't detail default behaviors or constraints, slightly limiting the score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'List all available PrestaShop hooks.' It specifies the verb ('List') and resource ('PrestaShop hooks'), making the function unambiguous. However, it doesn't explicitly differentiate from sibling tools like 'get_prestashop_hook' or 'search_prestashop_hooks,' which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It mentions filtering options but doesn't specify scenarios where this tool is preferred over siblings like 'search_prestashop_hooks' or 'get_prestashop_hook.' This lack of comparative context leaves usage unclear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_prestashop_docA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states the tool retrieves 'full content with metadata,' which adds some behavioral context beyond basic retrieval. However, it doesn't disclose critical traits like whether it's read-only (implied but not stated), error handling (e.g., for invalid paths), authentication needs, rate limits, or performance characteristics. For a tool with no annotations, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: the first sentence clearly states the purpose, followed by structured 'Args' and 'Returns' sections. Every sentence earns its place by adding value—no redundant or vague language. It's efficient and well-organized for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no nested objects) and the presence of an output schema (which handles return values), the description is mostly complete. It covers purpose, parameter semantics, and return context adequately. However, with no annotations, it could benefit from more behavioral details (e.g., safety, errors) to be fully comprehensive, but the output schema reduces the need for extensive return explanations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful semantics beyond the input schema. The schema has 1 parameter with 0% description coverage (just 'path' as a string). The description explains that 'path' is a 'Document path' and provides an example ('basics/installation/environments/macos-specific.md'), clarifying the expected format and usage. This compensates well for the low schema coverage, though it could specify path constraints or structure more explicitly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get the full content of a specific PrestaShop documentation file.' It specifies the verb ('Get') and resource ('PrestaShop documentation file'), distinguishing it from siblings like list_prestashop_docs (which lists files) and search_prestashop_docs (which searches content). However, it doesn't explicitly differentiate from get_prestashop_hook, which might retrieve hook documentation, making it slightly less specific than a perfect 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by mentioning 'specific PrestaShop documentation file,' suggesting it's for retrieving a known file rather than listing or searching. It doesn't provide explicit when-to-use vs. alternatives (e.g., use list_prestashop_docs to find paths first, or search_prestashop_docs for unknown content), nor does it mention prerequisites or exclusions. The context is clear but lacks detailed guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_prestashop_docsA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions the search scope and returns 'results with snippets and metadata,' but fails to disclose critical behavioral traits like pagination, rate limits, authentication needs, error handling, or whether it's read-only/destructive. For a search tool with zero annotation coverage, this leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded with the core purpose in the first sentence. The Args and Returns sections are structured for clarity, though the 'Returns' line could be more concise (e.g., merging with the first sentence). Every sentence adds value, with minimal redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 3 parameters with 0% schema coverage and an output schema present, the description does well on parameters but lacks behavioral context (no annotations). The output schema means return values needn't be explained, but for a search tool, details like result format limits or error cases would enhance completeness. It's adequate but has clear gaps in transparency.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds substantial meaning beyond the schema by explaining each parameter's purpose (e.g., 'query' for search, 'doc_type' and 'category' as filters with examples), including enum-like values for doc_type and category. This effectively documents all 3 parameters, though it could specify if filters are optional (implied by null defaults).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Search ALL PrestaShop documentation') and resource ('guides, tutorials, API docs, hooks, etc.'), distinguishing it from siblings like get_prestashop_doc (retrieves single doc) or search_prestashop_hooks (specific to hooks). The verb 'search' is precise and the scope 'ALL' is explicitly defined.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context by specifying the scope ('ALL PrestaShop documentation') and listing document types, which helps differentiate it from siblings like list_prestashop_docs (likely lists without search) or search_prestashop_hooks (hooks-specific). However, it lacks explicit when-not-to-use guidance or direct alternative naming.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_prestashop_hookA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the tool's behavior as a read-only documentation retrieval operation ('Get complete documentation'), which is clear and non-destructive. However, it lacks details on potential errors (e.g., if the hook doesn't exist), rate limits, or authentication needs, leaving some behavioral aspects unspecified.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose in the first sentence, followed by structured 'Args' and 'Returns' sections that efficiently document inputs and outputs without unnecessary verbiage. Every sentence earns its place by adding specific value, making it highly concise and well-organized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (single parameter, no nested objects), the presence of an output schema (which handles return value documentation), and the description's clear coverage of purpose, parameters, and returns, it is complete enough for effective use. The description provides all necessary context without redundancy, aligning well with the structured data available.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must compensate. It adds meaningful semantics by explaining that 'hook_name' is the 'Name of the hook' and provides concrete examples ('displayHeader', 'actionProductAdd'), which clarifies the parameter's purpose and expected format beyond the bare schema. This is valuable, though it doesn't cover all possible edge cases like invalid names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Get complete documentation') and resource ('for a specific PrestaShop hook'), distinguishing it from sibling tools like 'list_prestashop_hooks' (which lists hooks) and 'get_prestashop_doc' (which presumably gets general documentation). The verb 'Get' combined with the specific resource type makes the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by specifying it's for 'a specific PrestaShop hook,' suggesting this tool should be used when you already know the hook name. However, it doesn't explicitly state when NOT to use it or name alternatives like 'list_prestashop_hooks' for discovering hook names first, which would elevate it to a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.