Server Quality Checklist

Tool Scores

• shieldapi.check_ipA
  3.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations cover safety (readOnly, non-destructive, idempotent) and scope (openWorld). The description adds valuable behavioral specifics by listing the exact reputation checks performed (blacklist status, Tor exit node classification, reverse DNS), which helps agents understand the tool's coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise at nine words. The colon structure front-loads the action ('Check IP reputation') and efficiently lists the specific detection capabilities without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While appropriate for a single-parameter lookup tool, the description omits the output format/reputation scoring system. Given the absence of an output schema, the lack of return value documentation leaves a gap in contextual completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage for the single 'ip' parameter (including format example), the schema carries the semantic load. The description does not add parameter-specific guidance beyond the schema, warranting the baseline score of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verbs ('Check') and resources ('IP reputation') and enumerates the specific checks performed (blacklists, Tor detection, reverse DNS). However, it does not explicitly differentiate from siblings like 'check_domain' or 'full_scan' that might overlap in functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. Given the presence of siblings like 'check_domain', 'full_scan', and other check utilities, there is a notable absence of selection criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• shieldapi.full_scanA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations fully cover the safety profile (readOnly, non-destructive, idempotent). The description adds the behavioral scope ('all security checks'), indicating comprehensiveness, but omits operational details like scan duration, external API calls, or rate limit implications that openWorldHint suggests.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two efficient sentences with zero waste. The first establishes the action and target types; the second differentiates from siblings. Information is front-loaded with the operative verb 'Run' followed immediately by scope.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the single parameter, comprehensive schema coverage, and rich annotations, the description successfully establishes the tool's role in the ecosystem as the broad-scan option versus targeted siblings. No critical gaps remain for invocation, though output expectations could be mentioned.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the structured data already documents the target parameter fully. The description mirrors this information without adding format examples, validation patterns, or semantic constraints beyond what the schema provides, meeting the baseline expectation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific resource 'security checks' and distinguishes from siblings by stating 'all' and 'most comprehensive scan', implying this aggregates the specific check_* tools. However, it could be elevated to a 5 by explicitly naming sibling alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'Most comprehensive scan' implies when to use this tool (when breadth is needed over specificity), but lacks explicit when-not guidance or named alternatives like 'use check_domain for domain-only verification'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• shieldapi.check_passwordA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations comprehensively cover safety properties (readOnly, destructive, idempotent, openWorld). The description adds valuable context about HIBP (Have I Been Pwned) and breach checking, but omits behavioral details like the k-anonymity model, rate limits, or response format (boolean vs. count).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence of 14 words that is perfectly front-loaded with the action ('Check') and packs essential qualifiers (SHA-1, HIBP, data breaches) without redundancy. Every word contributes necessary specificity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the single parameter, comprehensive annotations covering all behavioral hints, and schema coverage, the description is complete for tool selection. It could improve by describing the return value semantics (exposed vs. not exposed) given the lack of output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage ('SHA-1 hash of the password (40 hex chars)'), the structured schema fully documents the parameter. The description provides equivalent information without adding syntax details or examples, meeting the baseline for high-coverage schemas.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description provides a specific verb ('Check'), identifies the resource ('password hash'), specifies the format ('SHA-1'), and states the data source ('via HIBP'/known data breaches), clearly distinguishing it from generic validation tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The mention of 'SHA-1' and 'HIBP' implies the tool requires pre-hashed passwords and queries an external breach database, but it lacks explicit guidance on when to use this versus the sibling 'check_password_range' tool or prerequisites like hash generation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• shieldapi.check_password_rangeA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations fully cover the safety profile (readOnly, idempotent, openWorld). The description adds valuable context that this queries the external HIBP (Have I Been Pwned) database and uses the k-Anonymity privacy model. It does not mention rate limits, authentication requirements, or what data is returned.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, 10 words. Front-loaded with the action verb. Every word serves a purpose (identifies the operation, the specific hash type, the parameter format, and the target database). Zero waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the single parameter with 100% schema coverage and complete annotations, the description is appropriately complete. The mention of 'HIBP' and 'k-Anonymity' provides sufficient domain context. Minor gap: no output schema exists and the description doesn't describe the return value (list of hash suffixes), though this is somewhat implied by the lookup operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, fully documenting the 'prefix' parameter as the first 5 characters of a SHA-1 hash. The description references 'SHA-1 hash prefix', reinforcing the schema semantics, and adds the HIBP context to explain why this specific format is required. Baseline 3 is appropriate given schema completeness.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Look up') and resource ('SHA-1 hash prefix' in 'HIBP k-Anonymity database'). It distinguishes from sibling 'check_password' by explicitly mentioning 'hash prefix' and 'k-Anonymity', signaling this is for range-based privacy queries rather than full hash checks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage through technical specificity ('k-Anonymity', 'prefix'), suggesting when to use this range-based approach versus full password checks. However, it lacks explicit when-to-use guidance or named alternatives (e.g., when to prefer check_password over this tool).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• shieldapi.check_domainA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations cover safety profile (readOnly, non-destructive, idempotent). Description adds valuable behavioral context by enumerating exactly what 'reputation' entails (specific blacklist providers, DNS, SSL) that annotations don't capture.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single dense sentence with purpose front-loaded before colon-separated specific checks. No wasted words; every term specifies a distinct reputation vector.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Adequate for a simple read-only tool with 100% schema coverage and rich annotations, but lacks description of return values or output structure despite having no output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% ('Domain name to check (e.g. example.com)'), providing complete parameter documentation. Description focuses on functionality rather than parameter semantics, adding no additional-guidance beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear specific verb ('Check') and resource ('domain reputation'), distinguishing from siblings like check_email/check_ip by specifying exact inspection targets: DNS records, specific blacklist providers (Spamhaus, SpamCop, SORBS), SPF/DMARC, and SSL.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies usage through specificity of checks listed, but provides no explicit when-to-use guidance versus siblings like check_url (full URLs) or full_scan (comprehensive multi-vector scanning), nor mentions prerequisites like valid domain format.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• shieldapi.check_emailA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare safety profile (readOnly, non-destructive, idempotent) and external dependency (openWorldHint). The description adds valuable context by specifying 'via HIBP' (Have I Been Pwned), clarifying which external service is queried. It does not contradict annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single 11-word sentence with zero redundancy. Front-loaded with action verb, immediately communicates resource and data source. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple single-parameter input, comprehensive schema coverage, and rich annotations covering safety traits, the description is complete. The mention of HIBP adequately explains the external dependency implied by openWorldHint.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage ('Email address to check'), the baseline is 3. The description mentions 'email address' but does not add format constraints, validation rules, or semantic details beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verb 'Check' with explicit resource 'email address' and scope 'known data breaches'. It clearly distinguishes from siblings like check_domain, check_ip, and check_password by specifying the exact entity being validated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The specificity of 'email address' implies when to use this tool (when targeting emails vs domains/IPs), but there is no explicit guidance on choosing between this and full_scan or scan_skill for comprehensive checks, nor when-not-to-use advice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• shieldapi.check_urlA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations cover the safety profile (readOnly, idempotent), so the description appropriately focuses on functional behavior by disclosing threat detection scope (malware/phishing) and underlying data sources (URLhaus + heuristic analysis), adding meaningful context beyond the structured hints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tightly constructed sentences with zero waste: first establishes functionality, second establishes methodology. Every word earns its place; appropriately front-loaded with the core action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter lookup tool with comprehensive annotations and high schema coverage, the description is sufficient. It appropriately omits redundant output schema details while covering the essential 'what' and 'how' of the threat detection.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the parameter is well-documented in the schema itself. The description does not add additional parameter semantics (format constraints, normalization rules) beyond the schema's example, warranting the baseline score of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Check') and resource ('URL') with explicit threat categories ('malware, phishing, and other threats'). It clearly distinguishes from siblings like check_domain or check_email by specifying 'URL' as the target resource type.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implicit usage guidance through specificity ('URL' vs siblings' domain/email/IP), but lacks explicit when-to-use guidance or distinctions from similar tools like check_domain or full_scan. It does not mention prerequisites or rate limits.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• shieldapi.scan_skillA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Strong supplement to annotations: confirms safety profile ('Static analysis only — no code execution' aligns with readOnlyHint/destructiveHint), details the 8 specific risk categories checked, and describes return format (score 0-100, findings, summary) compensating for missing output schema. Could mention performance or size limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three dense sentences with zero waste. Front-loaded action, middle lists specific categories logically, end covers methodology and output. Every clause earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Comprehensive for a scanning tool: explains what gets analyzed, methodology constraints, risk taxonomy, and return structure despite no output schema. Annotations cover safety/idempotency. No gaps given tool complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with clear descriptions ('Raw SKILL.md content', 'max 20 files'). Description aligns with schema but adds no additional parameter syntax detail; baseline 3 appropriate when schema carries full load.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Excellent specific verb ('Scan') + resource ('AI agent skill/plugin') plus specific taxonomy ('Snyk ToxicSkills'). Clearly distinguishes from siblings like check_domain or check_url by specifying the skill/plugin target and the 8-category static analysis approach.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear scope (skills/plugins vs single IoCs) implying when to use, but lacks explicit 'when-not-to-use' or comparison to sibling 'full_scan' or 'check_prompt' tools. Static analysis constraint noted but no workflow guidance provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• shieldapi.check_promptA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Substantial disclosure beyond annotations: specifies 4 detection categories, 200+ patterns, detailed return schema (boolean verdict, confidence 0-1, matched patterns, decoded content), and performance SLA (<100ms p95). Annotations cover safety profile (readOnly/idempotent), description covers functional behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three information-dense sentences with zero waste. Front-loaded with purpose (sentence 1), usage context (sentence 2), and output/performance characteristics (sentence 3). No redundancy with structured fields.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Comprehensive for a detection tool. Compensates for missing output_schema by detailing return structure (boolean, confidence, patterns, decoded content) and performance characteristics. Combined with rich annotations and 100% param coverage, description provides complete invocation context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with complete property descriptions. Description references 'text' which aligns with the 'prompt' parameter, but adds no syntax, validation rules, or format details beyond what's already documented in the schema. Baseline 3 appropriate when schema carries full documentation burden.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Opens with specific verb 'Detect' and clear resource 'prompt injection in text'. Distinguishes sharply from siblings (check_domain, check_email, etc.) by specifying text/prompt analysis across 4 unique categories (direct injection, encoding tricks, exfiltration, indirect injection).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit usage context: 'Designed for real-time inline usage before processing untrusted user input.' Gives clear operational guidance (inline, before processing). Lacks explicit 'when not to use' or comparison to siblings, though domain is distinct.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.