Server Quality Checklist

Tool Scores

• create_secretA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds valuable behavioral context beyond annotations: it explains the self-destructing nature, end-to-end encryption, server's inability to see plaintext, and how secrets can be read from various sources. While annotations cover basic safety (readOnlyHint=false, destructiveHint=false), the description provides important operational details about security and data handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured with three sentences that each add distinct value: purpose statement, security explanation, and usage guidance. It's front-loaded with the core functionality and contains no redundant information. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a creation tool with good annotations and comprehensive schema coverage, the description provides solid context about security, self-destruction, and input methods. The main gap is the lack of output schema, so the description doesn't explain what gets returned (e.g., a shareable link). However, given the tool's complexity and the schema's thoroughness, it's mostly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the schema already documents all parameters thoroughly. The description mentions 'label' and implies content handling from various sources, but doesn't add significant semantic meaning beyond what's in the schema. The baseline of 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('create a secure, self-destructing link') and resource ('secret'), and distinguishes it from siblings by focusing on creation rather than checking, listing, or viewing secrets. It provides concrete examples of what can be shared (passwords, API keys, credentials).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about when to use this tool ('for sharing sensitive data') and mentions alternative input methods ('reading secrets from environment variables, files, or .env files'). However, it doesn't explicitly state when NOT to use it or directly compare it to sibling tools like list_secrets or view_secret.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_secretsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, destructiveHint=false, and idempotentHint=true, covering safety and idempotency. The description adds useful behavioral context beyond annotations by specifying what information is returned (view counts, expiry, consumption status), which helps the agent understand the output format and scope. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action ('List previously shared secrets') and adds essential details ('current status — view counts, expiry, and whether they've been consumed') without waste. Every word contributes to understanding the tool's function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (0 parameters, no output schema) and rich annotations (readOnly, non-destructive, idempotent), the description is complete enough for an agent to use it correctly. It explains what the tool does and what information it provides, though it could mention pagination or sorting if relevant, but isn't required for basic understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description appropriately doesn't discuss parameters, focusing on the tool's purpose and output. Baseline is 4 for zero parameters, as it avoids unnecessary details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('previously shared secrets') with specific scope details ('view counts, expiry, and whether they've been consumed'). It distinguishes from sibling tools like 'create_secret' (creation) and 'view_secret' (single secret) by emphasizing listing multiple secrets with status information.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for viewing status of multiple secrets, suggesting when to use it (to see list with status details). However, it doesn't explicitly state when NOT to use it or mention alternatives like 'view_secret' for single secrets or 'check_status' for status checks without listing. The context is clear but lacks explicit exclusions or comparisons.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• view_secretA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds significant behavioral context beyond annotations: it discloses that secrets may have view limits and will be destroyed after maximum views (explaining destructiveHint), specifies default browser behavior for security, and describes output_mode options. This enriches the agent's understanding of side effects and operational details that annotations alone don't cover.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured in two sentences: the first states the core purpose and key behavioral traits (view limits, destruction), and the second explains the default and alternative output modes. Every sentence adds essential information with zero waste or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (destructive operation, multiple parameters, no output schema), the description is largely complete: it covers purpose, key behaviors, and output options. However, it lacks details on error handling, response format for 'direct' mode, or how view limits are tracked, leaving minor gaps for the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the input schema already documents all parameters thoroughly. The description adds minimal value beyond the schema, mentioning output_mode options briefly but not providing additional semantics or usage nuances for parameters like url vs. secret_id/encryption_key alternatives.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('retrieve and decrypt') and resource ('a secret from a Vaulted secure link'), distinguishing it from siblings like 'create_secret' (creation) and 'list_secrets' (listing). It goes beyond the name/title by specifying the decryption aspect and Vaulted source.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for usage ('retrieve and decrypt a secret') and mentions default behavior (opens in browser) and alternatives via output_mode. However, it doesn't explicitly state when to use this tool versus siblings like 'check_status' or 'list_secrets', nor does it provide exclusions or prerequisites for usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_statusA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read-only, non-destructive, and idempotent behavior. The description adds valuable context beyond this by specifying that it 'does not consume a view' (a key behavioral trait not covered by annotations) and explains the effect of the 'previousViews' parameter on response messaging. No contradictions with annotations are present.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured in two sentences: the first covers the core purpose and key details, and the second explains optional parameter usage. Every sentence adds value without redundancy, making it front-loaded and appropriately concise for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (4 parameters, no output schema), the description is mostly complete: it covers purpose, usage, and key behavioral traits. However, it lacks details on return values (e.g., what specific data is returned about views, active status, expiration) and does not fully explain all parameters, leaving some gaps in contextual understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With only 25% schema description coverage (only 'previousViews' has a description), the description compensates by explaining the purpose of 'previousViews' ('to detect new views since last check'), adding meaningful context. However, it does not clarify the semantics of 'url', 'secret_id', or 'status_token', leaving some parameters under-explained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('check the status') and resource ('previously shared secret'), with detailed scope including view count, active status, and expiration. It explicitly distinguishes from sibling 'view_secret' by noting 'does not consume a view', making the purpose unambiguous and differentiated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: it specifies when to use this tool (to check status without consuming a view) versus alternatives (implied that 'view_secret' would consume a view). It also mentions optional use of 'previous_views' parameter for detecting new views, giving clear context for parameter application.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.