Server Quality Checklist

Tool Scores

• explain_preflight_ruleB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered by structured data. The description's 'READ-ONLY' prefix reinforces but does not add to this. It adds minimal behavioral context beyond confirming the rule-explanation output, and does not contradict the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence with zero waste. The 'READ-ONLY' prefix overlaps with annotations but is harmless. Appropriately sized for a simple single-parameter tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a low-complexity tool (1 parameter, no output schema, no nesting), the description is reasonably complete — it states what the returned explanation covers. It partially compensates for the missing output schema by describing the explanation's contents, though it doesn't address error behavior for an invalid rule_id. Adequate given the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% — rule_id is documented with an example ('campaign.daily_volume'). The description adds no parameter-level meaning beyond implying that rule_id selects the rule to explain, which the schema already conveys. Baseline 3 is appropriate since the schema carries the parameter burden.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Explain') and resource ('one rule'), and clarifies what the explanation covers (checks, data needs, configurable options). It implicitly distinguishes from siblings like list_preflight_rules and validate_preflight_config, though it does not name them explicitly. A clear, well-scoped purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus its siblings. The description does not state that this should be used when detailed information about a single rule is needed as opposed to list_preflight_rules for enumeration or validate_preflight_config for config checking. No exclusions or alternative routing provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_preflight_rulesA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, and the description's opening 'READ-ONLY' merely reinforces that. It adds no deeper behavioral detail beyond the filter (which is more parameter-related). No mention of pagination, output format, or rate limits, but for a simple read-only list tool, the annotations carry the transparency burden sufficiently.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that front-loads the read-only nature and the core action ('List'), followed by the only parameter detail. Every word earns its place; there is no redundancy or filler. Ideal conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one optional enum parameter, no output schema, and no nested objects, the description covers the purpose, the filter, and its optionality. It does not mention return format or relationships to sibling tools, but those are not critical for a simple listing operation. The description is sufficiently complete for an agent to call it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage and only one optional enum parameter. The description explicitly lists all valid category values (campaign, contacts, suppression, personalization, copy, schedule, senders) and clarifies the parameter is optional ('optionally filtered'). This fully compensates for the schema's lack of descriptions, giving the agent complete knowledge of the parameter without opening the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb ('List') and resource ('preflight rule catalogue'), and mentions optional filtering. The purpose is clear and not a tautology. It does not explicitly distinguish from siblings like preflight_demo or preflight_files, but the name and description make the function obvious enough that an agent could differentiate it.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Gives context on the optional 'category' filter and lists valid categories, implying when filtering might be useful. However, it provides no explicit guidance on when to use this tool versus alternatives, nor any when-not-to-use statements. The presence of sibling tools like 'validate_preflight_config' suggests related actions, but no comparison is made.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_preflight_configA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already carry the safety profile (readOnlyHint=true, idempotentHint=true, destructiveHint=false), and the leading 'READ-ONLY' merely restates readOnlyHint. The added value is the description of what validation reports (unknown rule ids, unknown options, out-of-range values), which is useful but modest. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tight sentences with no filler. The critical READ-ONLY safety flag is front-loaded, and the second sentence efficiently enumerates what the validation surfaces (unknown rule ids, options, out-of-range values). Slightly redundant with annotations but well-structured and brief.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter, read-only validation tool, this is nearly complete: purpose, scope, safety, and output dimensions are all covered. No output schema exists, but the description names the three classes of findings an agent can expect, which is sufficient for a config-validator of this simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%; the schema already documents config_path as an absolute or relative path to an existing file. The description adds only the contextual hint that the path points to a configuration file, which aligns with but does not materially extend the schema. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Describes a specific verb (validate) applied to a specific resource (rules configuration file). It distinguishes itself from sibling tools like preflight_demo, preflight_files, and preflight_instantly_campaign by explicitly stating it does not run checks, and from list_preflight_rules/explain_preflight_rule by focusing on config validation rather than rule discovery. An agent can pick it apart from siblings without opening the schema.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides a clear context: validate a configuration file before executing checks. The phrase 'without running any checks' implicitly marks off the preflight_* siblings that do execute campaigns, giving a when-not signal. It does not name alternatives explicitly or give exclusion conditions, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• preflight_filesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnly and idempotent annotations, the description adds critical behavioral context: it explicitly states that no directory scanning occurs and that omitted optional inputs are reported as 'unavailable capabilities' rather than clean results. This clarifies how the tool handles missing parameters, which is a significant behavioral nuance beyond the annotations alone.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences with no filler. The first sentence states purpose, the second clarifies input scope, and the third explains behavior on omission. It is front-loaded with the core action and each sentence earns its place by contributing distinct, necessary information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 8 parameters, no output schema, and no explicit enumeration of the checks performed, the description provides sufficient context: it defines the tool's domain (local files), its safety profile, and handling of missing inputs. The output_format parameter handles return shape, and sibling tools list_preflight_rules/explain_preflight_rule can cover rule details. It is complete enough for an agent to use correctly, though slightly more detail on what 'readiness' entails would elevate it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides descriptions for all 8 parameters (100% coverage), so the baseline is 3. The description adds meaning by explaining the collective effect of omitting optional parameters (e.g., suppressions_path yields 'suppression checks did not run'), which is a parameter-level semantic not present in the schema. This justifies a 4 over the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Check a campaign described by local files and return its readiness report') with a clear resource (local files). It distinguishes itself from siblings by emphasizing 'local files' and the scoping rule 'Reads only the paths given here', which sets it apart from tools like preflight_demo or preflight_instantly_campaign that likely handle other sources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for file-based campaigns and clarifies that only explicit paths are read, but it does not explicitly contrast with sibling tools or state when to choose this over preflight_instantly_campaign or preflight_demo. The guidance is implied rather than directive, so it misses the 'when/when-not' bar for a higher score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• preflight_instantly_campaignA
  4.2/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, but the description adds valuable context: every request is checked against a read-only allowlist, and the API key is sourced from INSTANTLY_API_KEY rather than a tool argument. This discloses operational safeguards and security details beyond the annotations, enhancing the agent's understanding of constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise at three sentences, with purpose front-loaded ('READ-ONLY. Inspect...'). However, there is slight redundancy: 'READ-ONLY' is repeated in the allowlist explanation. Despite this, the structure is efficient and each sentence adds relevant information—purpose, scope, and security mechanism.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema, the description should explain what the readiness report contains, but it only says 'readiness report' without detailing its format or contents. It does mention the data sources (leads, accounts, block list) but not the report's structure. Given the absence of output schema, this is a notable gap. The description covers the operational context well (read-only, API key), but the return value remains underspecified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 75% (campaign_id, config_path, max_samples have descriptions; lead_limit has only min/max). The tool description does not elaborate on any parameter semantics. Per the rubric, with high schema coverage, a baseline of 3 is appropriate. The lack of compensation for the undocumented lead_limit is a minor gap, but overall the schema carries most of the parameter information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action and target: 'Inspect a live Instantly campaign and return its readiness report.' This is a specific verb-resource combination and explicitly distinguishes it from sibling tools (preflight_demo, preflight_files, list_preflight_rules, etc.) by the 'live Instantly campaign' scope. The mention of reading leads, sending accounts, and block list further clarifies the exact domain.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains the scenario (inspecting a live campaign) and the read-only nature, but does not explicitly compare with alternatives or state when not to use it. It provides clear context for when this tool is appropriate (live campaign inspection) without listing excluded cases or directing to other preflight tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• preflight_demoA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark readOnlyHint=true, destructiveHint=false, and idempotentHint=true, so the description's 'READ-ONLY' is redundant. However, it adds value by stating no network calls and no credentials, and that it uses synthetic data shipped with the package—behavioral details not present in annotations. No contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three short sentences, each earning its place: the first states the action and safety profile, the second gives key behavioral context, and the third explains practical utility. It is front-loaded with READ-ONLY and has no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple demo tool with one optional parameter and no required inputs, the description fully covers what the agent needs: what it does, that it's safe (read-only, no network/credentials), and when it's useful (showing a report). No output schema exists, but the description doesn't need to detail report structure for a demo. Complete for its purpose.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%—the parameter max_samples is described as 'Affected records shown per finding.' The tool description does not mention the parameter at all, but since the schema fully covers it, the baseline of 3 applies. No additional semantic value added beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs a bundled synthetic demo campaign and returns a readiness report. The word 'demo' and 'synthetic' distinguish it from siblings like preflight_files or preflight_instantly_campaign, which likely operate on real data. This is a specific verb+resource with clear differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly notes it's 'useful for showing what a report looks like,' providing a direct use case. It also hints at no network calls and no credentials, implying it's safe for testing. However, it doesn't explicitly state when to prefer this over siblings, so it lacks explicit exclusions or alternative comparisons.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.