Server Quality Checklist

Tool Scores

• scan_secretsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. While it states what the tool scans for, it doesn't describe how it behaves: whether it's read-only or modifies files, what permissions it requires, how it handles errors, whether it has rate limits, or what the output format looks like. For a security scanning tool with zero annotation coverage, this is a significant gap in behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that clearly states the tool's purpose. It's front-loaded with the main action and includes specific examples of what it scans for. There's no wasted language or unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a security scanning tool, no annotations, and no output schema, the description is incomplete. It doesn't explain what happens after scanning: whether it returns a list, generates a report, logs findings, or requires follow-up actions. For a tool that likely produces important security findings, more context about output and behavior is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents both parameters ('projectPath' and 'patterns') adequately. The description doesn't add any parameter-specific information beyond what's in the schema, such as examples of patterns or clarification on path formats. With high schema coverage, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: scanning source code for exposed secrets like API keys, tokens, passwords, private keys, and database connection strings. It specifies the verb 'escanea' (scans) and resource 'código fuente' (source code), but doesn't explicitly differentiate from sibling tools like 'scan_code_vulnerabilities' or 'scan_dependencies', which likely focus on different security aspects.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'generate_security_report' or explain how this scan differs from 'scan_code_vulnerabilities' or 'scan_dependencies'. There's no context about prerequisites, when-not-to-use scenarios, or typical use cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scan_code_vulnerabilitiesA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes the scanning behavior and return format (vulnerabilities with location and solutions), but lacks details on performance, error handling, authentication needs, or rate limits. It adequately covers the core operation but misses operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that efficiently conveys the tool's action, target, examples, and output. Every element earns its place with no redundant or verbose phrasing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a code vulnerability scanner with no annotations and no output schema, the description covers the core purpose and output format but lacks details on error cases, performance expectations, or integration context. It's minimally viable but has clear gaps for operational use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters thoroughly. The description doesn't add any parameter-specific information beyond what's in the schema, such as examples or constraints. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('escanea', 'buscando') and resources ('código fuente del proyecto'), listing concrete vulnerability patterns (XSS, eval(), innerHTML, tokens). It distinguishes from sibling tools by focusing on code patterns rather than dependencies, secrets, or report generation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for scanning source code for specific vulnerability patterns, but doesn't explicitly state when to use this tool versus alternatives like scan_dependencies or scan_secrets. No guidance on prerequisites, exclusions, or comparative contexts is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_security_reportA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It describes what the tool does (generates comprehensive security reports, saves to Markdown, prioritizes by severity) but doesn't mention important behavioral aspects like execution time, resource requirements, error handling, or whether it modifies the project. It adds useful context about the report format and prioritization but misses operational details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded with the main purpose. Every sentence adds value: the first states the core function, the second specifies content areas, and the third describes output characteristics. However, it could be slightly more structured by separating the 'what' from the 'how' aspects.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description provides adequate coverage for a tool that generates reports. It explains what the tool produces (comprehensive security report in Markdown with prioritization) but doesn't describe the return value format or error conditions. For a tool with 3 parameters and no structured output documentation, it meets minimum requirements but could benefit from more operational context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters thoroughly. The description doesn't add any parameter-specific information beyond what's in the schema descriptions. It mentions the report format and content but doesn't provide additional context about parameter usage, relationships, or constraints beyond the structured schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Genera un reporte completo de seguridad') and resources ('vulnerabilidades en dependencias, vulnerabilidades en código fuente y secrets expuestos'), distinguishing it from sibling tools that focus on individual scan types. It specifies the output format ('Markdown') and prioritization method ('priorizadas por severidad'), making it distinct from more targeted scanning tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool (for comprehensive security reports combining multiple scan types) versus siblings (which handle specific vulnerability types), but doesn't explicitly state exclusions or alternatives. It provides clear context about generating complete reports but lacks explicit 'when-not-to-use' guidance or named alternative tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scan_dependenciesA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses that the tool performs scanning, returns vulnerabilities sorted by severity with recommended solutions, but doesn't mention execution time, error handling, permission requirements, or rate limits. It adequately describes the core behavior but lacks operational details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise with two sentences that each earn their place: the first states the action and technology, the second describes the output format. No wasted words, well-structured and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a scanning tool with 2 parameters, 100% schema coverage, and no output schema, the description provides good context about what gets scanned and what's returned. However, without annotations or output schema, it could benefit from more detail about error cases or output structure beyond 'listado ordenado por severidad'.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters thoroughly. The description doesn't add any parameter-specific information beyond what's in the schema, maintaining the baseline score of 3 when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('escanea vulnerabilidades', 'retorna un listado') and resources ('dependencias del package.json', 'npm audit'). It distinguishes from siblings by focusing on dependency scanning rather than code vulnerabilities, secrets, or report generation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for dependency vulnerability scanning with npm audit, but doesn't explicitly state when to use this tool versus alternatives like scan_code_vulnerabilities or generate_security_report. No exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.