Server Quality Checklist

Tool Scores

• check_dependenciesA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses the behavioral scope by listing the checks performed (uptime, SSL validity, blacklist status, trust score), which gives useful context. However, with no annotations provided, it does not explicitly state whether the tool makes outbound network requests, is read-only, or has any side effects, leaving gaps in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that effectively communicates purpose and key outputs without unnecessary words. Every phrase adds value, making it an excellent model of concision.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, output schema provided), the description is largely complete: it explains what the tool does and the nature of the checks. It lacks explicit usage guidance and operational details like network behavior, but the availability of the output schema mitigates the need to describe return values, so overall it is complete for a tool of this complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema description covers 100% of the parameter 'endpoints' with a clear explanation and example. The tool description adds minimal extra meaning by framing them as 'external dependencies', but does not introduce constraints, format details, or parameter-specific behavior beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Check') and resource ('external dependencies'), and enumerates the aspects of health checked: uptime, SSL validity, blacklist status, and trust score. This distinguishes it from sibling tools like network_scan by focusing on dependency-specific health rather than general scanning.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It neither mentions exclusions nor suggests appropriate scenarios, leaving usage inference entirely to the tool name and sibling context. There is no explicit 'use this when...' or contrast with network_scan or scan_skill.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• network_scanA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively communicates that the scan is active, autonomous, and potentially subject to legal restrictions, which is critical context for a security tool. It doesn't mention potential network disruption, but the schema's scan_type descriptions add some duration and depth context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: two sentences, with the primary purpose front-loaded and the authorization warning logically appended. Every sentence adds value, with no unnecessary elaboration or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has an output schema and a thoroughly described input schema, the description covers the essential purpose, authorization constraint, and the active, autonomous nature of the scan. The only notable gap is the lack of explicit differentiation from sibling tools, but overall the description is sufficiently complete for a tool with this level of structured metadata.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with detailed parameter descriptions for target, scan_type, and authorized. The tool description adds only a general reference to the autonomous agent, which complements the scan_type description ('agent loop') but provides no new parameter-level details beyond what the schema already offers.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that this tool performs an active security scan of a device or host, with an autonomous AI agent deciding follow-up probes. It does not explicitly reference sibling tools, but the unique 'ActiveScanner' behavior and target scope sufficiently distinguish it from typical scanning utilities.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear authorization prerequisite ('Only scan targets you own or have permission to test') and implies use for active network scans. However, it lacks explicit guidance on when to use this tool versus sibling tools like scan_secrets or full_stack_audit, nor does it state when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• full_stack_auditA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description bears full responsibility. It discloses key behaviors: correlation into one report and graceful skipping of missing sections. However, it omits important safety context such as network scanning requiring explicit authorization and potential risks of analyzing untrusted skills. This is a meaningful gap for a security tool, though the schema partially compensates via network_authorized.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two well-structured sentences. It front-loads the purpose, enumerates the audit areas in a parallel list, and ends with the graceful-skipping behavior. Every word earns its place, with zero redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is moderately complex with six optional parameters and an output schema. The description explains the composite nature, auto-skipping, and report generation. It lacks explicit return-value details, but the output schema covers that. The main gap is the missing security authorization caveat, which slightly reduces completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so each parameter is individually documented. The description adds high-level context ('pass only what you have') but does not enrich individual parameter semantics beyond the schema. This matches the baseline for well-covered schemas.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs a comprehensive security audit by listing five specific actions (audits MCP config, checks dependencies, analyzes skills, scans network, scans for secrets/PII) and culminates in a single prioritized report. The verb 'audits' plus enumerated sub-audits fully specifies functionality and distinguishes it from the granular sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: use this for a complete security posture in one call, and it gracefully skips sections when inputs are missing. It implicitly suggests partial inputs are acceptable. However, it does not explicitly contrast with sibling tools (e.g., saying 'for individual checks use scan_skill'), so it lacks formal alternatives/exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scan_skillA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden of behavioral disclosure. It does list the types of analysis performed, which adds value, but it does not explicitly state whether the tool is read-only, requires network access, or how it handles a potentially malicious skill. The description implies safety by recommending use before installation, but it is not explicit about side effects or permissions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the verb and resource, then specifies the exact categories of analysis and the usage timing. Every word adds value with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With an output schema present and a single well-documented parameter, the description provides sufficient context for most use cases. It lacks explicit mention of prerequisites like network access or authentication, but these are minor gaps given the output schema and clear purpose.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage with a clear description for skill_url (including an example). The tool description adds no additional parameter semantics beyond the schema, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: analyzing an AI agent skill for specific security issues (prompt injection, malware, OWASP LLM Top 10). It uses a specific verb ('Analyze') and resource ('AI agent skill') and distinguishes itself from sibling tools like network_scan or scan_secrets by focusing on skills.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'BEFORE installing it' provides clear temporal guidance for when to use this tool. It does not explicitly name alternative tools or state when not to use it, but the context of sibling tools and the explicit pre-installation timing gives adequate context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• audit_mcp_server_configA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It discloses offline operation and no external service requirement, indicating no data exfiltration. It also lists detection categories, but doesn't explicitly confirm read-only behavior or potential false positives.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one dense sentence that front-loads the action ('Audit') and then lists specific risk categories in a scannable way. While the list is long, each item adds distinct value and the overall length is justified.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter tool with an output schema, the description adequately covers what the tool does and its offline nature. It doesn't explain output format, but that's covered by the schema. Minor gap: no explicit use-case example, but overall sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter config_json is fully documented in the schema with 100% coverage, including instructions to paste full content rather than a path. The description adds no additional parameter context, so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Audit') with a clear resource ('MCP client configuration') and enumerates concrete risk categories it detects, making it unmistakably distinct from sibling tools like scan_secrets or check_dependencies.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides useful context that the tool works offline and requires the config JSON, which implies when to use it. However, it does not explicitly state when NOT to use it or mention alternative sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scan_secretsA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully carries the burden and does so excellently. It discloses that detection runs in-process via regex + Luhn, operates offline without an API key, and optionally routes to a backend which persists an audit record – far beyond typical descriptions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise yet rich, with the core purpose in the first sentence and supporting technical detail in a second paragraph. Every sentence adds value, including the unexpected offline capability, without unnecessary filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The output schema exists, so return values are covered structurally. The description covers purpose, usage context, offline/online behavior, and audit implications, making it complete for a simple two-parameter tool. An agent has enough information to invoke it safely and effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for both parameters, so the baseline is 3. The description adds examples of payload types (prompt, API body, file contents) that map to the 'text' parameter, but does not go beyond the schema for the 'redact' parameter. This is adequate given the schema already documents both parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's specific purpose: scanning text for secrets and PII and returning a redacted copy. It distinguishes itself from sibling tools like scan_skill and network_scan by focusing on data leakage prevention, making its function unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly frames when to use the tool ('BEFORE it leaves for an LLM or external API') and provides context about offline vs. backend-routed scanning. It does not explicitly name alternatives, but the use case is clear enough that an agent can select it appropriately.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.