Server Quality Checklist

Tool Scores

• list_policy_deadlinesA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description bears full burden. It states the tool 'shows' data, implying read-only, but does not disclose other behavioral traits like auth needs, rate limits, or side effects. The lack of behavioral depth limits transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is concise, front-loaded with the tool's purpose, and each sentence adds value. The rationale about deadlines vs. releases is included efficiently, and parameter descriptions are compact.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (3 parameters, no required, output schema exists), the description covers usage context and parameter semantics adequately. It lacks return value explanation but the output schema handles that. Minor gaps exist (e.g., no pagination info) but overall complete for a list tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% but description provides clear semantics for all three parameters: store ('apple'/'play'/'') , scan_date (ISO date), upcoming_only (boolean). This adds significant meaning beyond the schema's type/default information, helping agents understand valid inputs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool shows a dated policy calendar, with context about how store policies are schedule-based rather than release-based. This differentiates from sibling tools like check_api_declaration or explain_rejection, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description implies usage for checking policy requirements relative to dates, but does not explicitly state when to use this tool over alternatives or provide exclusions. The context about deadlines vs. releases gives implicit guidance but lacks direct when/when-not instructions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_api_declarationA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description bears full burden. It implies a read-only check but does not explicitly state safety, rate limits, or prerequisites beyond what is implied by 'check'. Examples of parameters are given but behavioral traits are minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is well-structured with a clear header, explanatory paragraph, and Args section. Slightly verbose but front-loads purpose and provides necessary context without being overly long.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given presence of an output schema (not shown), description does not need to explain return values. It covers when to use, what inputs to provide, and gives a real-world use case. Missing info about output format is acceptable due to output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but description adds significant meaning: examples for api, options for store, and ISO date format for scan_date. This compensates well for the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool checks what an API, permission, or SDK obliges you to declare before writing code. Provides specific examples and distinguishes from sibling preflight_scan, making purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance to use before adding new APIs/permissions/SDKs to avoid rejection. Mentions it is the 'pre-write counterpart to preflight_scan', though could more thoroughly contrast with all siblings like explain_rejection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• explain_rejectionA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It indicates the tool is a lookup/decoding operation (no destructive effects), but does not explicitly confirm read-only behavior, auth requirements, or any rate limits. The description is adequate but could be more explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (two sentences plus an Args block), front-loads the key purpose, and has no unnecessary words. Every sentence serves a purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one required parameter, output schema exists), the description is adequately complete. It explains what the tool does, what input to provide, and how to use it. The output schema presumably documents the return value, so no further detail is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaning beyond the schema by explaining that the parameter can be a specific code (e.g., 'ITMS-91053') or the full email text, clarifying the acceptable input format.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Decode an App Store rejection code (ITMS-91053 etc) into a concrete fix.' It provides a specific verb ('Decode') and resource ('App Store rejection code'), and distinguishes from sibling tools (which focus on API declarations, policies, scans) by context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit usage guidance: 'Paste the code from the App Store Connect email.' It also explains the input format ('code: e.g. "ITMS-91053", or the full email text'). While it doesn't explicitly state when not to use it, the sibling tools are clearly different, making the use case obvious.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• preflight_scanA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that it reads specific files, compares code against declarations, and notes that scan_date affects verdicts. It doesn't mention performance or side effects, but transparency is good overall for a scan tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is reasonably concise with a brief intro and structured Args section. It could be slightly more terse, but all sentences add value. The structure is clear and front-loaded with purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters, no annotations, and presence of output schema, the description covers the main behaviors and parameters. It does not explain return values (but output schema exists), and lacks details on output format. Still, it is sufficiently complete for a scanning tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description compensates fully. Each parameter is explained with additional context: e.g., 'scan_date' can be a future date to preview deadlines, 'store' is 'apple'/'play'/'', 'severity' as min level. This adds significant meaning beyond the schema titles.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool scans a mobile app repo for rejections before submission, specifying files read (Info.plist, etc.) and what it reports. It distinctively covers comprehensive scanning, differentiating from siblings like 'check_api_declaration' or 'explain_rejection'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use before submission but provides no explicit when-to-use, when-not-to-use, or comparison with siblings. Usage context is implied through 'before submission' but lacks explicit guidance on alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_required_declarationA
  4.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It implies a read-only operation by stating 'Get', but does not explicitly disclose side effects, permissions, or limitations. Adequate but not thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, with a single sentence for purpose followed by two bullet-like parameter explanations. No redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (2 parameters with examples) and the existence of an output schema, the description sufficiently covers inputs. The return type is hinted by 'exact declaration and paste-ready snippet', and the output schema handles details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description provides clear examples and default behavior for both parameters (e.g., 'apple.rr.user_defaults' for rule_id, 'CA92.1' for reason_code), adding significant meaning beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it retrieves 'the exact declaration and a paste-ready snippet for a rule,' which is a specific verb and resource. This distinguishes it from sibling tools like check_api_declaration or preflight_scan.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description advises that rule_id comes from preflight_scan output, implying usage after that step. It also explains reason_code tailoring, giving context for use, but does not explicitly compare with alternatives or state when not to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.