Server Quality Checklist

Tool Scores

• simpleshop_download_documentsC
  2.3/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided and the description contains no behavioral details (e.g., failure modes, rate limits, authentication needs, or output format). The description fails to disclose any traits beyond the basic action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise (one sentence) but at the expense of essential detail. Every sentence should earn its place; this one is too sparse to be adequately informative.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of batch-downloading documents (multiple parameters, output schema exists but unmentioned, constraints like max 100 items), the description is grossly incomplete. No information on result format, error handling, or limits.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%. The description does not explain what the 'documents' array items represent, the meaning of 'variant' (with/without stamp), or the purpose of 'max_bytes'. With only 2 parameters, this omission is critical.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (batch-download) and resource (PDF rendering of SimpleShop documents), distinguishing it from sibling tools like 'simpleshop_find_documents'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool vs. siblings such as 'simpleshop_find_documents' or 'simpleshop_find_products'. No context on prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• simpleshop_get_metadataC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full responsibility. It only states 'Return... metadata', implying a read operation, but gives no details on side effects, authentication needs, rate limits, or output behavior. This is insufficient for a tool with 6 parameters.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one short sentence, which is concise but lacks structure. It is not front-loaded with the most critical information (e.g., parameters). Every sentence earns its place, but more content is needed to be helpful.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, 6 parameters without descriptions, and no output schema details in the description, the tool is severely under-documented. The description does not explain what metadata categories are, how to use the boolean flags, or what the return value contains.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% and the description does not mention any parameters. Each of the 6 boolean parameters controls inclusion of metadata categories, but the description adds no meaning beyond the schema defaults.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Return') and resource ('SimpleShop metadata'), and hints at its purpose ('useful for filtering and accounting classification'). It is distinct from sibling tools which focus on documents and products, but could be more explicit about what metadata covers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description offers no guidance on when to use this tool versus alternatives, no when-not-to-use conditions, and no mention of prerequisites or context. The single sentence only provides a general purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• simpleshop_get_product_salesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden for behavioral disclosure. It only says 'using the who bought export', which hints at a format but does not disclose side effects, permissions, rate limits, or whether it is read-only. Additional details like return structure or pagination are missing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single short sentence, which is concise and front-loaded. However, it omits important details that could be added without significant length, making it somewhat under-specified.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 5 parameters and an output schema, the description should explain the overall behavior, scope options, and what the 'who bought' export entails. It does not, leaving the agent with insufficient context to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 40%, with two parameters having descriptions. The tool description adds no additional meaning to any parameters, failing to compensate for the undocumented ones such as product_ids, scope, and include_raw_csv.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action 'Return' and the resource 'SimpleShop product sales', and references the specific 'who bought export' mechanism. This distinguishes it from sibling tools which deal with documents or products.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives or when not to use it. The description only states what it does without any contextual recommendations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• simpleshop_find_documentsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits. It only states the basic operation, omitting details like pagination, sorting, filtering semantics, or whether the operation is read-only. The description adds minimal behavioral context beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with no waste. It is front-loaded with the essential purpose, though the structure could include key parameter modes for clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (many filters, pagination, and an output schema), the description is adequate but not complete. It explains the two modes but omits details like sorting, pagination, and flag filtering, which are documented in the schema. The high schema coverage mitigates the gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% (the complex input schema includes descriptions for most parameters). The tool description does not add anything beyond what the schema already provides, meeting the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Find SimpleShop documents by search filters or explicit document IDs.' It uses a specific verb ('Find') and resource ('SimpleShop documents'), and distinguishes from sibling tools like `simpleshop_download_documents` (which downloads) and `simpleshop_find_products` (which finds products).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives, nor any conditions or exclusions. It lacks context for selecting between 'search' and 'by_ids' modes beyond what the schema example shows.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• simpleshop_find_productsB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, description should disclose behaviors like pagination, rate limits, or authentication, but only states basic operation. No behavioral details beyond what the name implies.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, no wasted words, purpose immediately clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Output schema exists, so return values need not be explained. However, the dual-mode operation is only hinted at; schema fills in details, but description could summarize mode behavior more clearly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the description adds no extra parameter meaning beyond high-level mention of filters and IDs. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool finds SimpleShop products using search filters or explicit IDs. It distinguishes from sibling tools like find_documents and get_metadata.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives, no when-not-to-use or prerequisites provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.