Server Quality Checklist

Tool Scores

• passkey_statusA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of transparency. It explicitly states it 'analyzes configs for Claude, Gemini, VS Code, Cursor, OpenCode, Windsurf, Cline, and Zed,' and that it identifies 'secrets secured via passkey' versus 'credentials exposed in plaintext.' It also discloses the return structure. This provides meaningful insight beyond a simple 'status' label, though it does not mention side effects like whether it modifies configs or requires special permissions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The first sentence is a crisp summary, followed by a short list of the analyzed tools and a 'Returns:' section. This is well-structured and front-loaded, though the tool list could arguably be condensed without losing value. Overall, every sentence contributes useful information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only status tool with no parameters and no output schema, the description is quite complete: it states the purpose, the scope (which tools are covered), the kind of analysis performed (security of secrets), and the return format. It does not explain use cases or alternatives, but the core functionality is fully covered.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema description coverage is 100%. Per instructions, the baseline is 4 when there are no parameters. The description adds no parameter details (none exist), and no compensation is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with 'Show security status of MCP servers across all detected tools,' which is a specific verb+resource combination. It clearly distinguishes itself from sibling tools like passkey_list (list) and passkey_doctor (diagnostics) by focusing on security status rather than general listing or troubleshooting.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance is given about when to use this tool versus alternatives. It does not mention passkey_doctor, passkey_list, or other siblings, nor does it state any exclusions or prerequisites. The usage context is only implied by the description of what it does.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• passkey_doctorA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does enumerate the diagnostic checks performed and states the return type, which is useful. However, it does not explicitly state whether the tool is read-only or whether it requires special permissions, leaving some ambiguity about side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear opening line, bulleted list of checks, and separate Args/Returns sections. It is somewhat redundant (the deep parameter is mentioned twice) but remains efficient and readable, with no wasted sentences.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with a single optional parameter and no output schema, the description covers the main tool purpose, the specific checks performed, the parameter semantics, and the return type. It lacks detail on error conditions or how to interpret the results dictionary, but overall it provides solid context for a diagnostic tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It explains the sole parameter `deep` both in the bullet list ('(deep=True) entry age analysis and bundle file permissions') and in the Args section ('deep: Include extended checks'). This adds meaningful context beyond the bare boolean schema, though it could be slightly more detailed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it 'Run diagnostics on passkey and MCP configurations' with a specific verb and resource, and lists distinct checks (config file validity, command availability, keychain permissions, MCP server issues). This differentiates it from sibling tools like passkey_list, passkey_fields, passkey_status, and passkey_wrap_server, which focus on other operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool (when you need to diagnose passkey/MCP configuration issues) by enumerating the common issues it checks. However, it does not explicitly state when not to use it or name alternative tools, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• passkey_wrap_serverA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses that it updates config files, returns lists of updated/skipped configs and errors, and notes the default behavior of detecting all configs. This adds transparency, though it doesn't elaborate on side effects like backup or reversibility.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact, starting with a one-sentence summary, then prerequisites, args, and returns. Every section adds useful information with no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has just 2 simple parameters and no output schema, and the description provides a clear return structure and prerequisite. It is complete enough for an agent to select and invoke it, though it assumes some domain knowledge about passkey wrappers.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has zero descriptions, so the description must compensate. It explains server_name must match passkey entry and config_paths defaults to all detected configs, adding meaning beyond type/name. It doesn't specify path formats but adequately clarifies both parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Update MCP config files to use passkey wrapper for a server,' which is a specific verb+resource and clearly distinguishes from siblings like passkey_list or passkey_status. It clarifies the tool's unique purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states the prerequisite that the entry must exist in passkey keychain, giving context on when to use. However, it does not explicitly name alternatives or exclusions, so it stops short of full 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• passkey_listA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the transparency burden. It communicates a read-only listing operation ('List', 'Returns') and adds context about the data model (multiple key-value pairs per entry), but does not mention permissions, error conditions, or empty-list behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact, front-loaded with the core purpose, and includes a helpful return format plus example. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the zero-parameter complexity and the presence of an output schema, the description fully covers the tool's behavior. It states the return type, provides an example, and includes relevant context about entry contents.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has no parameters, so the description does not need to explain any. The baseline for zero-parameter tools is 4, and the description correctly focuses on output rather than inputs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb ('List') and resource ('available passkey entries'). It also clarifies it returns names of secret entries from the system keychain, distinguishing it from sibling tools like passkey_fields and passkey_status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when the agent needs to enumerate all passkey entries, but does not explicitly contrast with sibling tools. It provides a clear context for the operation, though no when-not or alternative guidance is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• passkey_fieldsA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explicitly states that it does not reveal secret values (a safety-related behavior), describes the return format, and documents exception conditions (entry not found, keychain access failure). This is comprehensive for a read-only tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-organized into Args, Returns, and Raises sections, making it easy to scan. Every sentence provides useful information, with no filler or redundancy. It is appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description fully covers the tool's purpose, usage context, parameter meaning, return value behavior, and error conditions. With a single required parameter and an output schema, this description leaves no critical gaps for an agent to misuse the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description must compensate. It does so by providing a clear, contextual definition for entry_name, including an example ('slack') and a realistic return example. This adds meaningful semantic information beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'List field names in a passkey entry' with a specific verb and resource, and also emphasizes the key behavior of not revealing values. This distinguishes it from sibling tools like passkey_list (which likely lists entries) and passkey_status (which likely checks status).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains that this tool is useful for 'discovering what secrets are available without exposing the actual secret values,' which implies when to use it. However, it does not explicitly state when not to use it or name alternative tools, so it lacks explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.