Server Quality Checklist

Tool Scores

• list_scan_rulesA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations (readOnly, idempotent, non-destructive), the description adds genuinely new behavioral context: 'local snapshot of the compiled rule table, no network, no mutation,' the digest computation detail ('sha256 over the RFC 8785 form of {version, rules}'), and the reproducibility caveat that 'a recorded verdict is only reproducible together with this identity.' None of this is derivable from the annotations, and it does not contradict them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, then organized into clearly labeled sections (When to use / When NOT to use / Behaviour / Returns) that make it scannable. Every sentence earns its place — the output content is enumerated once in detail rather than repeated, and the non-use case doubles as sibling differentiation. The minimal redundancy of the example call is negligible.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 1-optional-parameter tool with an output schema present, nothing is missing: return contents, use cases, exclusion cases against siblings, network/side-effect behavior, digest identity, and an invocation example are all covered. An agent has everything needed to decide whether to call it and what to expect.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% — the input schema fully documents include_source, its default, and its effect. The description adds only marginal enrichment by framing include_source=true as enabling 'an independent re-implementation,' which gives the parameter a purpose beyond its mechanics. Baseline 3 is appropriate since the schema carries the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Return the in-force static-scan ruleset' followed by an exact enumeration of contents (version, digest, rule code, severity, tier, surfaces, regex source, guards). It explicitly distinguishes itself from siblings with 'This tool does not scan anything' and names static_scan_tools and vet_mcp_server as the tools that apply the table, so an agent cannot confuse it with them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides labeled 'When to use' guidance (explain a finding code, confirm ruleset v4, re-run a scan with the same table) and an explicit 'When NOT to use' section naming the sibling alternatives (static_scan_tools, vet_mcp_server) and the condition that routes to them. This is exactly the when/when-not/alternatives structure the rubric asks for.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• vet_mcp_serverA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnly/idempotent/destructive annotations, the description reveals local deterministic no-network behavior, built-in threat floor, empty pin store advisory outcomes, origin default, and that it never connects/starts/approves the server. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is organized into labeled sections, front-loads purpose, and uses each sentence for a distinct piece of information (use, exclusions, behavior, example). Despite length, it is dense and scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With a rich input schema, output schema, and annotations present, the description still adds the operational context needed to call correctly: gate order, policy defaults, first-contact behavior, and exclusions. Nothing essential for selection or invocation is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema carries the parameter burden. The description adds a concrete invocation example and notes policy override semantics, but it doesn't need to compensate for missing param docs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description opens with a specific verb+resource: run WARDEN's ordered gate chain over a server identity plus tools/list payload and return a verdict. It also distinguishes itself from siblings by naming what it is not in the 'When NOT to use' section.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Contains explicit 'When to use' and 'When NOT to use' guidance with named alternatives: static_scan_tools, classify_sensitive_tools, check_egress_url, canonicalize_json. It even says to prefer this over calling the four gates manually.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• classify_sensitive_toolsA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Goes well beyond the readOnlyHint/idempotentHint/destructiveHint annotations by disclosing: local glob match with no network access, no persistence, no Warden.vet call, empty patterns array → every tool safe, whole-name matching with '*delete*' hitting create_delete_repo, and that sensitive tools are classified but not run. These are exactly the behavioral traits an agent needs to predict side effects and edge cases.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Organized into labeled sections (When to use / When NOT to use / Behaviour / Returns) that make it scannable, with the core purpose front-loaded in the first sentence. Slightly verbose, but every sentence carries distinct information — no filler, and the section labels justify the length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Complete for a 2-param tool with full schema coverage, rich annotations, and an output schema. The description covers purpose, when-to-use versus three named siblings, edge cases (empty patterns), safety profile (no network, no persistence), matching semantics, and return shape. Nothing an agent needs to invoke it correctly is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds genuine value with a concrete worked example showing the call shape and return value ({ sensitive, safe }), and it reinforces the crucial nuance that descriptions/schemas are ignored in classification and only name is matched. The example clarifies the wildcard semantics beyond what the schema states.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb and resource: 'Split advertised tool names into sensitive vs safe using the operator's case-insensitive * globs'. It names the policy source (policy.sensitiveToolPatterns), explicitly declares what it is not ('This is policy over identifiers, not an injection scan'), and the title itself is an accurate summary. Distinguishes clearly from siblings static_scan_tools, vet_mcp_server, and check_egress_url.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit selection criteria: 'When to use: show the user which names will need confirmation before they approve a server, or to preview a glob set.' Then gives explicit exclusions with named alternatives: 'When NOT to use: scanning descriptions for poisoning (static_scan_tools or vet_mcp_server); checking whether a URL is allowed out (check_egress_url).' An agent can route to the correct sibling without inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• static_scan_toolsA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds significant behavioral detail beyond the read-only, idempotent, non-destructive annotations: local regex+guard evaluation, no network access, no mutation, no server launching, and advisory-tier hits flagged with advisory=true that do not reduce the score. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is longer than average, but each section earns its place: purpose, routing conditions, behavioral caveats, and a concrete example. The When to use / When NOT to use structure is scannable and front-loads the key decisions.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the rich annotations, full input schema, output schema, and sibling context, the description covers everything an agent needs to call correctly: scope, score range, advisory behavior, side-effect guarantees, and alternative routing. Nothing important is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents the tools parameter and its nested properties. The description adds value by instructing the agent to pass descriptions and input schemas through unmodified, warning that rewriting hides injection surfaces, and noting the shape matches vet_mcp_server.tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific action ('Run only the static-scan gate') over a specific resource ('advertised tool names, descriptions, and input schemas'), then identifies the outputs. It clearly distinguishes itself from vet_mcp_server, classify_sensitive_tools, and list_scan_rules via the When NOT to use section.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Contains explicit 'When to use' and 'When NOT to use' sections. It names the triggering condition (having a tools/list dump and wanting injection/credential/hidden-Unicode hits) and lists sibling tools that should be used instead for broader analysis or rule retrieval.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• canonicalize_jsonA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations, the description reveals local/no-network operation, the exact failure mode (isError with CanonicalizationCode, no partial bytes), accepted input forms (parsed value or JSON string parsed with parseJsonStrict), and strict acceptance rules (integers within ±(2^53−1), lone surrogates refused). No annotation contradiction is present.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-organized with clear sections: purpose, when to use, when not to use, behavior, return shape, and example. Every sentence carries useful information, and the core purpose is front-loaded before the usage guidance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool with an output schema, the description fully covers usage context, input constraints, failure behavior, and expected output. Nothing an agent needs to call or interpret the result correctly is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds meaning by specifying that a string input is parsed with parseJsonStrict first, and it provides a concrete example (canonicalize_json({ value: { b: 1, a: 2 } }) → {"a":2,"b":1}) that clarifies input and output shape beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Return'), a specific resource (RFC 8785 JSON Canonicalization Scheme serialization), and the purpose (WARDEN threat-feed signatures, tool-def pins, byte-checking). It also distinguishes the tool from static_scan_tools in the 'When NOT to use' section, so it is not just a restatement of the name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description contains dedicated 'When to use' and 'When NOT to use' sections. It lists concrete use cases (publishing/verifying signed threat feeds, hashing tool defs, comparing JSON documents independent of key order) and names an existing alternative (static_scan_tools) plus an exclusion (pretty-printing for humans).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_egress_urlA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Even with annotations declaring readOnlyHint, idempotentHint, and destructiveHint, the description adds substantial behavioral context: local URL parse, no DNS/HTTP, fail-closed empty allowlist, case-insensitive matching, wildcard semantics, and refusal of unparseable URLs. This fully discloses the tool's operational behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-organized into short labeled sections, front-loads the core purpose, and contains no filler. Each sentence adds meaningful guidance, from scope and exclusions to behavior and return shape.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only, idempotent two-parameter tool, the description is complete: it explains purpose, when to use it, when not to use it, behavior, return value, and an example. The existing output schema covers the detailed return structure, so no necessary information is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents both parameters thoroughly. The description adds value beyond this with a concrete example call, wildcard behavior restated in practical terms, and the fail-closed empty allowlist note, which helps the agent form correct invocations.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Ask EgressGuard whether a URL's hostname is on an operator allowlist.' It clearly distinguishes itself from siblings by naming vet_mcp_server, static_scan_tools, and canonicalize_json as tools not to use for this purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit 'When to use' and 'When NOT to use' sections, naming concrete alternatives and explaining the intended context: checking a host before a tool fetches or posts. It also specifies what the tool does NOT do, such as fetching the URL or inspecting tool text.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.