Server Quality Checklist

Tool Scores

• classify_vulnerabilityA
  4.1/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully discloses side effects (none, read-only, stateless), authentication needs (none for basic, pro requires API key), rate limits (free 10/day, pro unlimited), error handling (structured errors), idempotency, and data privacy. This is comprehensive and exceeds requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections and front-loaded purpose. However, there is redundancy between the 'Behavior' and 'Behavioral Transparency' sections, and the length could be trimmed without losing meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite covering purpose, parameters, and behavior thoroughly, the description lacks explicit output specification (e.g., what the classification returns). With no output schema, this is a notable gap. Overall, it provides a good foundation but misses key details for a complete understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description lists all parameters except api_key (present in schema) with brief descriptions, partially compensating for 0% schema coverage. However, meanings for boolean fields are not elaborated, and one parameter is missing from the textual description. It adds some clarity but not enough to fully compensate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'classify' and resource 'vulnerability', specifying classification by type, severity, and OWASP category. It distinguishes the tool from siblings like lookup_cve and analyze_password_strength by focusing on classification rather than lookup or analysis of other aspects.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes explicit 'When to use' and 'When NOT to use' sections, providing context for security assessment and cautioning against sole reliance for production decisions. However, it does not compare directly with sibling tools to guide selection, missing a chance to differentiate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lookup_cveA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Comprehensive disclosure of side effects (read-only), authentication (no auth basic, API key for pro), rate limits (10/day free, unlimited pro), error handling, idempotency, and data privacy. No annotations provided, so description carries full burden and exceeds expectations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with clear sections (Args, Behavior, When to use, etc.). Some redundancy in behavioral details, but overall organized and front-loaded with purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers purpose, parameters, behavior, and limitations. Lacks description of output format (no output schema), but mentions structured error objects. Sufficient for a look-up tool with behavioral details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%. Description lists three parameters with examples but omits 'api_key' parameter (though mentions API key in behavioral section). Partially compensates but missing parameter reduces clarity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Look up CVE details' with specific verb and resource. Distinct from sibling tools like analyze_password_strength, check_security_headers, classify_vulnerability, generate_threat_model.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit 'When to use' and 'When NOT to use' sections provide clear guidance on appropriate contexts and limitations, e.g., 'Do not rely solely on this tool for production security decisions.'

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_security_headersA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses read-only, stateless, idempotent nature, rate limits (free vs pro), and authentication requirements, offering comprehensive behavioral insight.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-organized into clear sections (Args, Behavior, When to use/not), each sentence is purposeful and concise without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers behavior, rate limits, and usage warnings well, but omits output format/return value details (no output schema provided) and error handling, a minor gap for a simple analysis tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds value for the 'headers' parameter with an example and format, but fails to mention the 'api_key' parameter, leaving part of the schema uncovered despite low schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool analyzes HTTP security headers against best practices, a specific verb-resource pair that distinguishes it from sibling tools like analyze_password_strength or classify_vulnerability.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit 'When to use' and 'When NOT to use' sections provide clear context for appropriate usage and cautions against sole reliance for production decisions, guiding the agent effectively.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_threat_modelA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It details side effects (read-only, no modifications), authentication (basic usage no auth, pro requires API key), rate limits (10/day free tier), error handling (structured error objects), idempotency, and data privacy. Very comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with labeled sections (Args, Behavior, When to use, When NOT to use, Behavioral Transparency) and is front-loaded with purpose. It contains multiple sections that add value, though could be slightly trimmed without losing substance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema exists, but the description mentions structured output without side effects, rate limit headers, and error objects. It covers authentication, rate limits, error handling, idempotency, and data privacy. Explicit output format details would increase completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description adds meaning to each of the 7 parameters by listing them with explanations and examples (e.g., 'components: System components (e.g. web_app, api_server)'). It also mentions defaults for authentication_method and deployment, and notes required fields.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool generates a STRIDE-based threat model for a system, specifying the type of output. It distinguishes from sibling tools like analyze_password_strength and lookup_cve, which focus on different security tasks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit 'When to use' and 'When NOT to use' sections provide clear guidance: use for security assessment, threat detection, vulnerability analysis; but not solely for production decisions, always combine with manual review.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_password_strengthA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully discloses behavioral traits: read-only and stateless, idempotent, rate limits (free 10/day, pro unlimited), authentication needs, error handling, and data privacy (local processing only). This addresses all key transparency concerns.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (Args, Behavior, When to use, etc.), but it is somewhat verbose. However, every sentence contributes value and the structure aids readability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one required parameter, no output schema), the description covers all necessary context: purpose, parameters, behavioral traits, error handling, and rate limits. An agent can confidently invoke this tool based on the description alone.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, but the description adds essential meaning: password is processed locally and never stored (privacy), and api_key is referenced in authentication context (MEOK_API_KEY environment variable). This compensates fully for the schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Analyze password strength and provide improvement suggestions.' This specific verb+resource combination, along with sibling tool names (security headers, vulnerability classification, threat model, CVE lookup), distinguishes it effectively.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit 'When to use' and 'When NOT to use' sections provide clear context for tool invocation, including that it should not be used as the sole basis for production security decisions. This guides the agent on appropriate usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.