Server Quality Checklist

Tool Scores

• answer_questionA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden for behavioral disclosure. It states the refusal condition (TRENDYOL_ALLOW_WRITES=true), the public visibility, the character length requirement, and the once-only constraint. It lacks specifics on error handling or reversal, but these are important behavioral traits that are clearly covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: a clear purpose line, then constraints, then argument definitions. It is not overly long and each sentence contributes value. The 'WRITE:' prefix is a helpful signal. Minor inefficiency: the args section could be more concise, but overall it's effective.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the essential context: how to obtain question_id, why the answer is sensitive (public display), and the constraints (length, once-only). An output schema exists, which may cover return details, so its absence in the description is acceptable. For a simple write tool with two parameters, this is fairly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must explain the parameters. It does: question_id is sourced from get_customer_questions, and text must be 10-2000 characters. This adds meaning beyond the schema's bare type declarations, though it doesn't provide full validation rules or examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action: 'publish an answer to a customer question on Trendyol'. It uses a specific verb and resource, and the 'WRITE:' prefix distinguishes it from read-only siblings. The tool is clearly distinct from get_products, get_orders, get_customer_questions (which reads), update_price_and_stock (updates product), and get_claims.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides context (answer is public, 10-2000 chars, answered only once) and mentions the question_id comes from get_customer_questions, but it does not explicitly state when to use this tool versus alternatives or when not to use it. The guidance is implied rather than explicit, so an agent must infer the selection criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_claimsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full responsibility for behavioral disclosure. It explicitly states the operation is read-only and discloses that customer names are not included in the response, which adds valuable context. It also explains the returned summary fields. While it doesn't discuss error handling or pagination behavior, the key non-obvious facts (read-only, omitted field) are covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, using only a few sentences. It front-loads the purpose and read-only flag, then lists the arguments and return fields. Every sentence adds value, with no filler or repetition. The structure is clean, separating arguments from return information clearly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that an output schema exists (per context signals), the description doesn't need to enumerate every return field. It however outlines the key fields (claim id, orderNumber, claimDate, statuses) and explicitly notes the omission of customer names, which is a notable completeness element. It does not mention pagination behavior like total counts or link headers, but for a simple list tool with two params this is acceptable. Overall, it provides enough for an agent to call correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, so the description must compensate. It clearly explains both parameters: page is zero-based and size is items per page. This adds semantic meaning beyond the bare type and default in the schema, providing clarity that prevents off-by-one errors. The description fully compensates for the missing schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb ('List') and resource ('returns/claims'), and explicitly notes it is for the store and read-only. It is distinguishable from siblings like get_products and get_orders because it focuses on claims, and the first line gives a precise scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies its use case by mentioning 'returns/claims' and the data it returns, but it does not explicitly state when to use this tool versus alternatives (e.g., 'use this for claim statuses rather than orders') or provide exclusion conditions. The context makes it obvious for a human, but an AI agent would benefit from explicit routing guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_productsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It clearly discloses read-only behavior, explains pagination semantics (zero-based page, size with a cap note), and lists the exact fields returned. It does not discuss rate limits or authentication, but these are typically environment-level and not required for a read operation. The description goes beyond a bare statement of purpose.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose and read-only note, then uses a compact bullet-like format for parameters and return fields. Every sentence provides necessary operational detail with no padding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a list endpoint with four optional parameters and no annotations, the description covers all runtime considerations: parameter behavior, pagination, filtering, and return shape. It even flags a known sibling cap difference. Nothing essential is missing for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must fully compensate. Each parameter is explicitly explained: page (zero-based), size (with sibling cap callout), approved (true/false/omit semantics), and barcode (exact match). This adds far more meaning than the schema's type/default definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool lists products in the seller's Trendyol store and marks it as read-only. This is a specific verb-object-resource pairing that clearly differentiates from siblings like get_orders or get_claims, which operate on different data types.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context (listing products, read-only) but does not explicitly mention when to prefer this over alternatives or when not to use it. There is no mention of exclusions or alternative tool names, so an agent has to infer usage from the tool name and sibling context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_customer_questionsA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing behavior. It states the operation is read-only, explains the default status behavior, enumerates allowed status values, describes pagination semantics, and explicitly notes that customer user names are not included. This is strong behavioral disclosure for a read-only list endpoint.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with an opening purpose sentence, a compact Args section, and a Returns section. Every sentence carries useful information, and the most important scope information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 3-parameter read-only list tool, this description covers defaults, allowed values, pagination, return fields, and an explicit exception (user names excluded). An agent has everything needed to select the tool and invoke it correctly without consulting additional documentation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description fully compensates: status is expanded with all five allowed values and the meaning of the default, page is defined as zero-based, and size is given a maximum of 50. This adds meaningful semantics that the bare schema properties lack.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'List customer questions about the store's products.' It immediately identifies the operation as read-only, which clearly distinguishes it from mutation siblings like answer_question and update_price_and_stock.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description makes clear this is a listing tool for customer questions and is read-only, providing clear context for when an agent would choose it. However, it does not explicitly name alternatives or state when not to use it, so it stops short of full routing guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_ordersA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses read-only status, PII redaction with opt-in, API limits (page 0-49, size max 200), and the return structure (paged summary with specific fields). This is comprehensive and includes behavioral nuances like date format options and redaction defaults.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with an 'Args' block and a 'Returns' line. Each sentence and line provides necessary information without fluff. The status list is long but essential for valid usage. Front-loaded with the read-only note and the return summary, making it easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given six parameters, zero schema descriptions, and no annotations, the description covers everything an agent needs to call correctly: parameter formats, constraints, PII handling, and return fields. The output schema exists, so extra return detail is not required, but the description still provides a useful summary.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so description must compensate. It explains every parameter: status with documented enum values, start/end date formats (ISO or epoch), page zero-based with max, size max, and include_pii with redaction context. This adds meaning far beyond the bare schema, fully covering parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists order packages (shipment packages) for the store, with a read-only note. The verb 'List' and resource 'order packages' are specific, and the read-only qualifier helps distinguish it from mutation tools. The sibling tools (get_products, get_claims, etc.) are clearly about other domains, so the purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not explicitly name when to use this tool versus alternatives, nor does it mention when not to use it. However, the purpose is clear enough that an agent would infer its applicability for order queries, but it lacks explicit routing to alternatives like 'for product info use get_products'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_price_and_stockA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden, and it does so thoroughly. It discloses that this is a write operation, requires an environment variable to be set (refusal otherwise), processes asynchronously, and returns a batchRequestId. It also clarifies that only provided fields are updated, preventing assumptions that omitted fields are cleared. This is exemplary transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured, starting with a WRITE: label that immediately sets expectations, followed by the requirement, then the args block with an example and edge cases, and ending with the return format. Every sentence adds value; the example is illustrative without being redundant. It is slightly longer than strictly necessary but remains tight.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (single parameter with rich JSON semantics), the description covers all that an agent needs to call it correctly: purpose, prerequisite, item fields, optionality, batch size, async behavior, and the returned batchRequestId. The existing output schema likely documents the return shape, so not repeating it is fine. No critical gap exists.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema defines only 'items_json' (string) with no description (0% coverage). The description fully compensates by specifying it's a JSON array of item objects, giving a concrete example, listing required and optional fields, and enforcing the max 1000-item limit. An agent can construct valid input without opening any other documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'WRITE: batch-update stock quantity and/or prices for existing products.' This names a specific verb (batch-update), a concrete resource (stock quantity and/or prices), and scopes it to existing products. It clearly distinguishes itself from the sibling read tools (get_products, get_orders, etc.) which are all queries or single-item operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It states the prerequisite environment variable (TRENDYOL_ALLOW_WRITES=true) that must be set for the tool to proceed, which is a direct usage condition. It also outlines the batch nature and the asynchronous result. It does not explicitly name alternatives, but the purpose is unambiguous enough that no exclusion is needed; the context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.