Server Quality Checklist

Tool Scores

• proofrelay.describe_cli_sdk_helperA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, destructiveHint, idempotentHint. Description adds that it requires no authentication, mutates nothing, and returns JSON text. No contradiction, and extra context is provided.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, front-loading the main purpose and using multiple short sentences. It avoids unnecessary details, though slightly more structure could improve scannability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters and an output schema present, the description is complete. It covers the tool's role, usage context, and return content (statuses, endpoint, flow). No gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist; schema coverage is 100%. The description correctly omits parameter details and focuses on the tool's purpose and output. Baseline for 0 parameters is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool as an integration guide for the ProofRelay helper pattern, specifying it is read-only and returns JSON text. It distinguishes from sibling recommend_checkpoint by naming it explicitly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for when to use (understanding how to wrap actions) and suggests an alternative (recommend_checkpoint). However, it does not discuss exclusions or differentiate from all siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• proofrelay.get_verifier_statusA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Description adds beyond annotations: 'performs no network call from this local wrapper' and 'returns JSON text'. Annotations already cover readOnly, destructive, idempotent hints, so description enriches with important behavioral details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with purpose, no unnecessary words. Every sentence adds value, efficient and clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters and existing output schema, description covers all needed context: purpose, usage order, side effects, authentication, and return type. Complete for this simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Tool has zero parameters, so description does not need to add parameter info. Baseline 4 for 0 parameters is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it is a read-only status lookup for public ProofRelay discovery metadata, specifying exactly what it retrieves (endpoint, URLs, counts, trust boundary). It is distinct from sibling tools which focus on helper, recommendation, risk scanning, or verification.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use first to confirm' indicating it should be used before other tools. However, it does not explicitly state when not to use or name alternative tools, but the context of siblings provides implicit guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• proofrelay.recommend_checkpointA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds value by stating it requires no authentication, mutates nothing, and returns JSON with status, checkpoint, and reason fields. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with purpose and usage guidelines. Every sentence earns its place with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists, the description covers purpose, usage, safety, and parameters completely. No additional information needed for this 2-param tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description mentions typical values for action_class and explains the has_authority_envelope parameter, but the schema already provides adequate descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly defines the tool as a read-only checkpoint recommendation for high-level agent actions, specifying the verb 'recommend' and resource 'checkpoint'. It distinguishes from siblings by referencing use cases like 'get_verifier_status' for discovery and 'verify_bundle' after evidence exists.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use (before relied-upon actions like paid_tool_call) and when to use alternative tools (get_verifier_status for discovery, verify_bundle after evidence). Provides clear context and exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• proofrelay.scan_mcp_riskA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond annotations (readOnlyHint, idempotentHint), description adds 'does not fetch server_url', 'mutates nothing', 'is not a security certification', and details return fields. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four sentences, each serving a distinct purpose: purpose, usage, constraints, output. No redundant or missing information. Efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero required parameters, simple input, and presence of output schema, description is thorough: covers behavior, constraints, alternatives, and output structure. No gaps identified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with baseline 3. Description adds nuance for 'server_url' parameter (clarifying it is not fetched), but does not add new info for 'descriptor' parameter. Overall, good supplementary context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Read-only advisory scan of caller-supplied public MCP descriptor metadata', specifying verb, resource, and scope. It distinguishes from sibling 'verify_bundle' by explicitly mentioning it as an alternative.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit usage context: 'Use before registering or trusting an MCP server' and lists specific flags (mutating tools, payment language, etc.). Also names alternative tool 'verify_bundle' and states that no authentication is required.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• proofrelay.verify_bundleA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds beyond that: 'does not certify real-world facts' and 'returns JSON text with status, checks, problems, verifier, and hosted endpoint fields,' which neither contradicts annotations nor repeats them. It gives the agent concrete behavioral expectations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no fluff. First sentence delivers purpose and usage; second adds behavioral constraints and output format. Every phrase earns its place, and critical info (safety, authentication, alternative) is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has one parameter (nested object), output schema exists, and annotations are rich, the description still adds value: it explains what the tool does not certify, gives output field names, and distinguishes from siblings. No gaps remain for an agent to make incorrect assumptions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter 'bundle' has schema coverage 100% with a description that lists what to include ('Synthetic or non-confidential evidence bundle metadata') and explicitly warns what NOT to include ('Do not include secrets, prompts, raw logs, source code, customer files, wallet keys, payment credentials, or tenant traces'). The tool description reinforces this, adding no redundancy.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with 'Read-only public-safe bundle shape check for synthetic or non-confidential evidence metadata,' clearly stating the verb (check), resource (bundle), and scope (synthetic or non-confidential). It also distinguishes from sibling 'recommend_checkpoint' by name, ensuring no ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool: 'when you already have a bundle fixture and need a local pass/fail boundary check before sharing or routing to hosted verification' and provides a clear alternative: 'use recommend_checkpoint when deciding whether a checkpoint is needed.' It also notes 'Requires no authentication, mutates nothing,' guiding safe usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.