Server Quality Checklist

Tool Scores

• calculateC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only states the calculation function, but does not mention whether this is a read-only operation, requires authentication, or what the output format will be. The lack of side-effect information is a significant gap for a tool with no structured metadata.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads the purpose. It is efficient and free of verbosity, though it could be slightly more informative without sacrificing brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 7 parameters and no output schema, this description is insufficient. It does not preview the response structure, clarify the role of required vs optional parameters, or hint at constraints like weight limits (which are in the schema but not summarized). The agent has to rely entirely on the schema for operational details, making the description contextually weak.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already documents all 7 parameters with descriptions (100% coverage). The description adds no additional semantic meaning beyond the schema, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states that the tool 'calculates cost and delivery times' for Russian Post, which is a specific verb and resource. It is clear enough to understand the tool's function, though it does not explicitly differentiate from the sibling 'delivery_time' which may also return delivery times. However, the emphasis on cost makes the distinction implicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus the siblings like 'delivery_time' or 'track'. No conditions, alternatives, or exclusions are mentioned, leaving the agent to infer the appropriate context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_officesB
  3/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does not mention whether the operation is read-only, what the output format looks like, any filtering logic, or potential side effects. For a search tool, this is a significant gap; the agent cannot anticipate return behavior or data scope.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence with no filler or redundant information. The key search criteria are front-loaded, making it immediately scannable. It is appropriately concise for a straightforward lookup tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having 4 optional parameters and no output schema, the description offers no usage context. It does not explain how the parameters interact (e.g., whether they are mutually exclusive filters), what the result will be, or the significance of the 'top' parameter. An agent would need to inspect the schema to understand even basic invocation, making this incomplete for a tool with this complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the parameter semantics are documented in the schema. The description adds only a high-level hint about searching by index or settlement, which aligns with existing parameter descriptions but does not introduce new meaning. The baseline of 3 is appropriate since the schema already provides sufficient parameter details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: searching post offices by postal code or settlement. It uses a specific verb (Поиск) and resource (почтовых отделений), and the mention of two distinct search criteria distinguishes it from siblings like zip_lookup and track, which serve different functions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus the siblings. There are no explicit exclusions, prerequisites, or alternative routing. The description only states what it does, leaving the agent to infer usage context, which is insufficient given the presence of similar lookup tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delivery_timeB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the full burden of behavioral disclosure. It only states the calculation of delivery times without mentioning whether it is an estimate, what data it relies on, whether it is read-only, or any side effects. This is minimal and leaves significant unknown behavior for a tool with no annotation support.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence that gets straight to the point. It is concise and front-loads the core function, with no redundant text. It could be slightly more detailed without losing conciseness, but it is appropriately sized for a simple tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has four parameters including enums, no output schema, and no annotations, the description is insufficiently complete. It does not clarify what the tool returns (e.g., estimated days, date range), how mail_type and mail_category affect the calculation, or any limitations. An agent would be left uncertain about the tool's expected output and applicability.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters including formats and defaults. The description adds no extra meaning about parameters beyond implying the two indices from the phrase 'between two indices,' which the schema already covers. Thus it meets the baseline without enhancing understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (расчёт) and the resource (сроки доставки) with a specific scope (between two indices). It differentiates from siblings like track (tracking parcels) and zip_lookup (postal code info) by focusing on delivery time calculation, so an agent can identify its purpose without ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention scenarios, exclusions, or refer to sibling tools. An agent receives no contextual hints about when delivery_time is the appropriate choice, leaving usage entirely implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• normalize_addressB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions 'through the Russian Post API', which hints at network dependency and potential latency/failures, but it does not disclose whether the operation is read-only, may modify any external state, requires authentication, or has rate limits. The tool's behavior beyond the basic operation remains opaque.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of a single, direct sentence with zero filler. It front-loads the purpose and the method, making it immediately understandable. There is no unnecessary detail or repetition, which makes it an exemplary model of conciseness for a simple tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool without an output schema, the description is minimally adequate: it states what the tool does and via what service. However, it does not describe the expected return value or any error behavior, which an agent might need for correct invocation. Given the tool's simplicity, these gaps are not critical, but the description could be more complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers 100% of the parameter descriptions, giving a clear meaning for 'raw_address' ('Address in free form for normalization'). The tool description adds no additional semantics about the parameter itself. Since schema coverage is high, the baseline of 3 is appropriate; the description does not compensate for any gaps but also does not need to.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('normalize') and a resource ('address') and specifies the method ('via Russian Post API'). This clearly distinguishes it from sibling tools like 'track' or 'zip_lookup' because the action and resource are unambiguous. It does not explicitly name alternatives, but the distinct purpose is evident from the wording.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage: if an agent needs to normalize a postal address, this tool is appropriate. However, there is no explicit guidance on when NOT to use it or what alternatives might be better for different scenarios. The context is clear from the name and description, but the guidance is implicit rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• trackA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears the full burden of behavioral disclosure. It merely states the tracking function, implying a read-only operation, but does not disclose what the response contains, potential errors, rate limits, or any side effects. This is insufficient for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that directly conveys the purpose. It contains no extraneous information and avoids repetition, making it concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one documented parameter, the description covers the core purpose and parameter semantics. However, without annotations or an output schema, it does not explain the expected return value (e.g., delivery status events) or any usage constraints, leaving a noticeable gap for a tracking tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%: the 'barcode' parameter is described with an example ('RA123456789RU'). The tool description adds no additional parameter information, so the baseline of 3 is appropriate since the schema already documents the parameter adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description is specific: 'Отслеживание почтового отправления Почты России по трек-номеру' (tracking a Russian Post postal item by track number). It clearly names the verb (tracking), the resource (Russian Post postal item), and the method (by track number), distinguishing it from sibling tools like zip_lookup or delivery_time.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a track number is available, but provides no explicit guidance on when to prefer this tool over alternatives or any exclusions. It relies on the obvious context of having a barcode, but does not mention prerequisites or exceptions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• zip_lookupA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool returns region, city, and work schedule, which is useful. However, it does not mention error behavior when a postal code is invalid or not found, nor any side effects, rate limits, or authentication requirements. For a simple lookup this is acceptable but still leaves gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that front-loads the key information (what the tool does and what it returns). There is no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one simple parameter and no output schema, the description sufficiently informs an agent of the expected output fields. It lacks explicit handling of edge cases like invalid codes, but for a lookup of this nature, it is largely complete. The presence of sibling tools with distinct purposes reduces the need for elaborate guidance.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the parameter is fully described in the schema (postal code, 6 digits, pattern). The description adds no additional meaning about the parameter itself; it only describes the output. Per the rubric, a baseline of 3 is appropriate when schema already covers parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action (lookup) and resource (postal code), listing the returned data fields (region, city, work schedule). It is easily distinguishable from sibling tools like track, calculate, and get_offices, which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool should be used when a postal code is given and region/city/schedule info is needed, but it does not explicitly state when to use it over alternatives or mention any exclusions. The purpose is clear enough, but there is no direct guidance on selection criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.