Server Quality Checklist

Tool Scores

• sign_agent_traceA
  4.3/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond annotations (readOnlyHint=false, destructiveHint=false), the description discloses side effects: writes an envelope file, uses local identity store, hashing, and no network. This provides substantial behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise at five sentences, front-loaded with purpose and usage, followed by side effects and return type. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema, the description appropriately summarizes the return type. The side effects and no-network behavior are clearly stated. Sibling tools include verification counterparts, providing complete context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the description need not add much. It restates the schema's description for trace_path and mentions the default for save_path, but does not add new syntactic details or constraints that would significantly help the agent.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (sign), resource (browser-agent JSONL trace), and method (Ed25519 identity). It specifies the exact trace format (WEB_WIZARD .traces/<runId>.jsonl), distinguishing from sibling tools like sign_action or sign_mcp_manifest, but could be more explicit about differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly states 'Use when a WEB_WIZARD or Steel run completes,' providing contextual triggers. It also notes the absence of network activity. However, it does not mention when not to use the tool or suggest alternatives like verify_agent_trace.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scan_mcp_serverA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnlyHint=true and idempotentHint=true, and the description confirms 'Read-only; no network required when tools is supplied directly.' It adds return format details. No contradictions; the description complements annotations well.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is four sentences, each adding value: core purpose, usage workflow, behavioral notes, return format. No redundancy; front-loaded with the main action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (5 parameters, 1 required) and robust annotations, the description covers purpose, workflow, behavior, and returns. It could mention error handling or edge cases, but overall it is complete enough for an agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description does not add significant new meaning beyond the schema; it mentions the required 'tools' parameter implicitly but does not elaborate on optional parameters' semantic purpose beyond what's in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Fingerprint an MCP server's tool surface into an unsigned ssx360.mcp-manifest.v1.' It uses a specific verb and resource, and distinguishes itself from siblings like sign_mcp_manifest and verify_mcp_manifest by explaining the workflow (use before/after).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use: 'Use before sign_mcp_manifest' and 'Re-scan later and pass results to verify_mcp_manifest with a baseline.' It also provides context about network requirements and read-only nature, helping the agent decide when to invoke.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• statusA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false, and the description reinforces this by stating 'no Git or filesystem writes, no network', adding specific behavioral detail beyond the boolean hints. It also discloses the return tuple structure, providing extra transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact, front-loaded with the main purpose, and each sentence adds value: purpose, usage guidance, read-only disclosure, return format, and parameter note. No redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (1 optional param) with strong annotations and an output schema (also described in text). The description covers what it does, when to use it, what it returns, and how it differs from siblings. Nothing important is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers the single parameter fully with its own description, so the baseline is 3. The description's parameter note ('defaults to detected repo') mirrors the schema's 'Empty auto-detects', adding no new meaning beyond what's already provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Report') and lists exact resources: hook install state, local envelope count, and Matrix Scroll config. It also differentiates from siblings by explicitly excluding signature checks and directing to verify_envelope or verify_pr_range.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use guidance ('Call first in a new repo before any verify/sign tool') and explicit when-not-to-use with named alternatives ('Do not use for signature checks — call verify_envelope or verify_pr_range instead'). This fully covers usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_mcp_manifestA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds valuable context: 'Read-only; no network' and describes the return tuple with conditions for ok false (bad signature or drift). This goes beyond annotations by explaining output semantics and error conditions without contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: first defines the action; second gives usage context and return format. No fluff, front-loaded with the core purpose. Every word adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema, the description still explains return values and key conditions. It covers behavioral constraints (read-only, no network), usage context (CI/trust before upgrade), and parameter roles. The tool is simple (2 params, no enums) and the description fully addresses its complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the description's parameter explanations ('Optional baseline signed manifest for rug-pull drift detection', 'Signed ssx360.mcp-manifest.v1 to verify offline') are identical to the schema's own descriptions. Thus the description adds no new meaning beyond the schema, meeting the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool verifies a signed MCP manifest and optionally diffs against a baseline. It distinguishes from sibling tools like verify_agent_trace and verify_envelope by specifying the resource type (MCP manifest) and the operation (verify with diff). It also suggests a concrete use case (CI or before trusting after upgrade).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly recommends using in CI or before trusting after upgrade. States it is read-only and no network, which reassures safe usage. However, it does not explicitly mention when not to use or compare to alternatives like verify_agent_trace, though the specific resource type implicitly differentiates.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_pr_rangeA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Despite annotations indicating readOnlyHint=false, the description clarifies 'Read-only for Git refs' and discloses side effects: 'hosted mode emits usage to ssx360.com' and requires SSX360_API_KEY. This adds context beyond the annotations without contradicting them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a front-loaded purpose, usage guidance, mode behavior, return shape, and parameter list. However, the parameter list is largely redundant with the schema, adding length without new information, so it scores 4 rather than 5.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (11 params, no required fields, output schema present), the description provides use-case guidance, explains hosted vs offline modes, and summarizes the return format. It omits detailed policy semantics, but those are covered by the schema and output schema, so it is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides 100% coverage with full descriptions for all 11 parameters, so the bar is at baseline 3. The description's parameter block mostly restates schema content with slight grouping (e.g., 'require_actor_types / deny_actor_types: Actor policy lists'), offering minimal additional semantic value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description begins with 'Scroll Gate: verify signed/unsigned commits across a PR commit range,' providing a specific verb and resource scope. It clearly distinguishes the tool from siblings like verify_envelope and audit_export by naming them explicitly, so the agent knows what this tool uniquely does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states 'Use for merge gates and PR review (many commits),' and then provides alternative tools for different conditions: 'Prefer verify_envelope for one commit offline' and 'Prefer audit_export when auditors need bundles.' This is clear when-to-use guidance with named alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• connect_cardA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Despite annotations (readOnlyHint=false, destructiveHint=false), the description adds valuable context: 'opens a short-lived serial session; does not export private keys' and 'No SSX360_API_KEY required.' It also discloses the return shape, covering side effects and auth requirements beyond what annotations provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: purpose sentence, usage guidance, side effects, return, then parameters. It is front-loaded with the core purpose. Slightly verbose due to repeated parameter details already in schema, but every sentence serves a functional role and the organization aids comprehension.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (hardware interaction, side effects, alternatives) and the presence of an output schema, the description covers all essential aspects: prerequisites (probe before signing), exclusions, auth requirements, side effects, return format, and parameter guidance. The agent has enough context to invoke it correctly and understand outcomes.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters. The description restates parameter meanings (e.g., 'reader_name: Serial port, e.g. COM3 or /dev/ttyACM0') but adds no new semantic information beyond the schema. It also includes the same security hint about MATRIXSCROLL_PIV_PIN. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource: 'Probe AP2 Vault Card / SE050 USB CDC hardware signing bridge availability.' It explicitly distinguishes itself from siblings by stating 'Use create_envelope for signing... Do not use for verification — call verify_envelope... Prefer status to check local identity without opening USB.' This clearly identifies the tool's unique role.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use context ('Use before hardware signing to confirm the reader responds on the serial port') and clear exclusions ('Do not use for verification'). It names alternative tools (create_envelope, verify_envelope, status) with reasons, giving an agent strong decision support.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• publish_notesA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With minimal annotation hints (all false), the description discloses side effects clearly: 'updates the local git notes ref only; push refs/notes/matrixscroll to remote separately.' It also specifies the return tuple, adding meaningful behavioral context beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear lead sentence, usage block, side-effect note, and parameter list. It is slightly redundant with the schema's parameter docs, but each section earns its place and is efficiently written.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers purpose, usage alternatives, side effects, and return format. With output schema present, it still provides adequate context for a 4-parameter tool, making it fully self-contained.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with detailed parameter descriptions. The description's parameter section largely repeats the schema (e.g., 'Range start ref (exclusive)'), adding no new semantic information. Baseline 3 is appropriate since the schema already carries the parameter meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource+context: 'Publish local signed envelopes to git notes for CI Scroll Gate verification.' It clearly distinguishes from siblings by naming the CI workflow and separate tools for offline checks and hosted audit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit usage direction is provided: 'Use after create_envelope' and 'Do not use for offline single-commit checks — call verify_envelope' and 'Do not use for hosted org audit — call audit_export'. This covers both when and when-not to use, naming explicit alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_envelopeA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Despite minimal annotations (all false), the description details side effects: 'may write .matrixscroll/envelopes/<sha>.json when save is true.' It also states prerequisites (Git repo, Matrix Scroll identity store), network requirement (no network), and return format. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with clear sections: purpose, usage, side effects, parameters list. Each sentence adds value, though the parameter list largely mirrors schema information. Could be slightly more concise but remains efficient for a 7-parameter tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 7 optional parameters and an output schema not shown, the description covers the return format ('{ok, sha, envelope, path, error?}'), side effects, and prerequisites. It is adequate for an agent to understand the tool's behavior, though the exact output schema details are not included (but output schema exists).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value by providing examples for `actor_type` ('e.g. agent, human, ci') and clarifying defaults. It also groups parameters logically and adds context beyond schema descriptions, but some entries are close to schema text.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Create a signed Git commit envelope with Ed25519 provenance metadata.' This is a specific verb+resource combination. It distinguishes from sibling tools like 'sign_action' (non-Git evidence) and 'verify_envelope' (verification only).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use: 'Use after staging changes and before or after git commit.' Provides alternatives: 'Prefer sign_action for non-Git evidence.' Clearly excludes verification use: 'Do not use for verification — call verify_envelope or verify_pr_range instead.'

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_envelopesA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false. The description adds that the tool is 'Read-only: no local Git writes' and specifies the return shape ('Returns {ok, envelopes, total?, error?}'). This adds useful context beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: it opens with the main purpose, then provides usage guidelines, behavioral notes, and a bulleted parameter listing. Every sentence serves a clear purpose with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is complete given the context: annotations cover safety, output schema exists (not shown but acknowledged), input schema is fully documented, and the description covers usage, behavior, and parameters comprehensively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the description does not need to fully explain parameters. However, the description adds context like 'paginated audit review in agent workflows' for limit and offset, and explains the format for signer_filter. This adds value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'List commit envelopes stored on ssx360.com for the authenticated organization.' It uses specific verb ('List'), resource ('commit envelopes'), and scope ('for the authenticated organization'), and distinguishes from siblings like 'audit_export' and 'verify_envelope'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use ('paginated org triage and agent memory') and when-not-to-use ('Do not use for offline Git repos — call status and verify_envelope. Do not use for bulk export — call audit_export instead.') with alternative tools named.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sign_mcp_manifestA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond annotations (which only indicate non-read-only and non-destructive), the description discloses side effects: 'may write save_path; uses local identity store. No network.' It also describes the return format. This fully informs the agent of behavioral traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences: main action, usage guidance, side effects/return format. Front-loaded with purpose. No wasted words. Efficient and clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers purpose, usage context, behavioral side effects, and return format. Even though an output schema exists, the description still provides a high-level return structure. Given the tool's simplicity (2 params, 1 required), this is complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds minimal extra meaning: it specifies the manifest is an 'Unsigned ssx360.mcp-manifest.v1 document from scan_mcp_server' and save_path is an 'Optional file path.' This is helpful but not significantly beyond the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool's action: 'Ed25519-sign an MCP tool-surface manifest for offline install verification.' It specifies the output format and differentiates from siblings by mentioning 'Use after scan_mcp_server' and 'Prefer verify_mcp_manifest for checks.'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit usage guidance: 'Use after scan_mcp_server. Prefer verify_mcp_manifest for checks.' Also notes side effects and what the tool does not do (no network). This clearly helps an agent decide when to invoke this tool versus alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_agent_traceA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark readOnlyHint and destructiveHint. Description adds offline operation, no network, return format with optional fields. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short sentences: function, usage, return. Front-loaded with core purpose. No extraneous text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Complete for a verification tool: purpose, usage context, offline nature, parameter role, return shape. Output schema covers detailed return. No gaps given complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers both parameters with descriptions. Description adds meaning by explaining trace_path as 'optional trace byte check' and return structure. Adds value beyond schema for the optional parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear specific action: verifying a signed agent trace envelope offline. Distinct from sibling verification tools targeting different resources (e.g., verify_envelope, verify_mcp_manifest).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly recommends use in CI or auditor handoff. States read-only and no-network constraints. Does not explicitly exclude scenarios, but context is sufficient for appropriate selection among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_envelopeA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark readOnly/idempotent/destructive false, but the description adds valuable context: 'Read-only: no network or SSX360_API_KEY required. Does not modify Git state.' It also details the return tuple and error semantics: 'ok is false on signature, policy, expiry, or missing-envelope errors.' No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-organized: purpose sentence, usage guidance, behavioral notes, return format, then parameter summary. It is concise but comprehensive for a 10-parameter tool, with no redundant filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 10 parameters and complex policy options, but the description covers purpose, usage vs alternatives, behavioral constraints, return format, and parameter semantics. It is fully sufficient for an agent to select and invoke the tool correctly, especially given the output schema exists and the description already explains return values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description's parameter section duplicates schema information without adding substantive meaning; for instance, grouping aliases like 'envelope / envelope_path' and 'trusted_keys / trusted_keys_file' is already stated in the schema as alias relationships.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Verify one signed commit envelope offline against RFC 8032 Ed25519 rules.' It explicitly distinguishes from siblings by directing users to verify_pr_range for ranges, audit_export for bundles, and status for hook status, making the tool's unique scope clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use guidance: 'Use for a single commit SHA or explicit envelope JSON file.' It also gives concrete alternatives and exclusions: 'Prefer verify_pr_range for PR/branch ranges and audit_export for procurement bundles' and 'Do not use when you only need hook status — call status instead.'

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• audit_exportA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses side effects (writes files under output_dir), auth requirement (SSX360_API_KEY), and mode-dependent behavior (hosted vs local fallback). These details go beyond the annotations (readOnlyHint=false, destructiveHint=false) and provide essential operational transparency. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the purpose and usage context. It uses bullet-like lines and a compact return type summary. Every sentence earns its place; there is no fluff or redundancy despite covering multiple modes and parameter groups.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 9 parameters, no required fields, and an output schema, the description fully covers hosted vs local modes, authentication, side effects, and return types. It also points to sibling tools for adjacent needs, making it complete for an AI agent to decide when to use it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3, but the description adds value by grouping parameters (hosted filter vs local range) and summarizing each parameter's role (e.g., signer_id filters by device_id or fingerprint). It doesn't merely repeat the schema; it clarifies which parameters apply in which mode.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool exports compliance or procurement audit bundles, naming specific formats (JSON, GUAC JSONL, evidence-pack). It distinguishes from siblings by explicitly naming verify_pr_range and list_envelopes as alternatives, making the purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use guidance: 'Use when auditors need envelope bundles'. Gives clear when-not-to-use: 'Prefer verify_pr_range for merge-gate pass/fail' and 'Prefer list_envelopes to browse hosted metadata'. This directly addresses tool selection versus sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sign_actionA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses side effects: writes save_path, uses local identity store, no network. Describes return structure. No contradiction with annotations (readOnlyHint=false, destructiveHint=false).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise, front-loaded with purpose and usage, structured into clear paragraphs with parameter bullet points. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 4 parameters, output schema exists, and annotations provide basics, description covers side effects, return format, and usage guidance comprehensively. No gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but description adds extra context: action_type enumerates valid labels, payload clarifies no signature block, key_path is for CI ephemeral keys, save_path for file output. Adds value beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Sign a universal provenance action envelope with the active Ed25519 identity.' Lists specific use cases (CI steps, IaC changes, etc.) and explicitly distinguishes from siblings create_envelope and verify_envelope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use (for signing actions like CI, migrations, deployments) and when not to use (for verification, prefer create_envelope for Git commits). Provides clear alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.