Server Quality Checklist

Tool Scores

• get_patternsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes the tool's behavior by specifying the return format (dictionary with total pattern count and categories breakdown) and listing example categories (e.g., prompt_injection, jailbreak). It also mentions 'No API key required,' which adds useful context about authentication needs. However, it lacks details on rate limits or potential errors.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, starting with the core purpose and followed by organized details (categories, return format, authentication note). Every sentence adds value without redundancy, making it efficient and easy to parse for an AI agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (0 parameters, no annotations, but with an output schema), the description is complete enough. It explains what the tool does, provides example categories, specifies the return structure, and notes authentication details. Since an output schema exists, the description does not need to exhaustively explain return values, and it covers all necessary context for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so the schema already fully documents the inputs. The description does not add parameter information, which is appropriate, but it compensates by explaining the output semantics (e.g., returns a dictionary with specific fields). This goes beyond the input schema, warranting a score above the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('List all available') and resource ('ClawGuard detection patterns'), distinguishing it from sibling tools like get_usage, health_check, scan_batch, and scan_text which have different functions. It provides concrete details about what is returned (42+ security detection patterns organized by category).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by stating 'No API key required,' suggesting this tool is accessible without authentication, but it does not explicitly state when to use this tool versus alternatives like scan_text or scan_batch. There is no guidance on prerequisites or exclusions, leaving usage context somewhat vague.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scan_batchA
  4.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It reveals that the tool processes texts individually and returns all results, which is useful context beyond basic functionality. However, it doesn't disclose important behavioral traits like rate limits, authentication requirements, error handling, or what constitutes 'security threats' in the results.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly structured and concise: it starts with the core purpose, explains the batch behavior, provides usage context, then documents parameters and returns in a clean format. Every sentence earns its place with no wasted words, and key information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (batch processing), no annotations, and the presence of an output schema (which handles return value documentation), the description is quite complete. It covers purpose, usage guidelines, parameter constraints, and behavioral context. The main gap is the lack of security context about what threats are detected, but the output schema likely addresses result structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds significant meaning beyond the input schema, which has 0% description coverage. It explains that 'texts' is a 'List of texts to scan' and specifies a practical constraint ('max 10 per call') that isn't in the schema. This compensates well for the schema's lack of documentation, though it doesn't detail the format or content expectations for the texts.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Scan multiple texts for security threats') and distinguishes it from the sibling 'scan_text' by emphasizing batch processing ('multiple texts', 'each text individually', 'multiple user inputs, chat messages, or document sections in one call'). It explicitly differentiates from the single-text scanning alternative.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: it states when to use this tool ('Useful for checking multiple... in one call') and implicitly when not to use it (for single texts, use 'scan_text' instead). It names the alternative tool by context and specifies the optimal use case scenario.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_usageA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes the tool's behavior by specifying what data is shown (e.g., tier, daily limits, usage count) and the requirement of an API key, which adds context beyond the input schema. It does not mention potential side effects, rate limits beyond status, or error handling, but for a read-only tool, this is reasonably comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose in the first sentence, followed by specific details in bullet-like clarity, and ends with prerequisites and return information. Every sentence adds value without redundancy, making it efficient and well-structured for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (0 parameters, read-only operation) and the presence of an output schema (which handles return values), the description is complete. It covers the purpose, what data is retrieved, prerequisites, and hints at the return structure, providing sufficient context without overloading information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and the input schema has 100% description coverage (though empty). The description does not need to add parameter details, so it appropriately focuses on the tool's purpose and output. It implies no input is required, which aligns with the schema, earning a baseline score of 4 for zero parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Get API usage statistics') and resource ('for your ClawGuard Shield account'), with explicit details about what information is retrieved (tier, limits, usage count, etc.). It distinguishes itself from sibling tools like get_patterns or scan_text by focusing on account usage metrics rather than pattern retrieval or scanning operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: to check API usage statistics, including tier, limits, and request counts. It mentions the prerequisite ('Requires a valid API key'), which helps guide usage. However, it does not explicitly state when not to use it or name alternatives among siblings, such as health_check for system status instead of usage metrics.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scan_textA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes what the tool does (analyzes text using 42+ detection patterns), what threats it detects, and the structure of the return value. It doesn't mention rate limits, authentication requirements, or performance characteristics, but provides substantial operational context beyond basic functionality.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly structured and economical. It opens with the core purpose, provides implementation details in the second paragraph, documents the return structure clearly, and ends with parameter explanations. Every sentence earns its place with no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity, no annotations, 0% schema description coverage, but with an output schema present, the description provides excellent completeness. It explains the tool's purpose, detection methodology, return value structure, and parameter semantics. The output schema handles return value documentation, allowing the description to focus on operational context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must compensate for the schema's lack of parameter documentation. It clearly explains both parameters: 'text' as 'The text to scan for security threats' and 'source' as 'Optional source identifier for tracking (default: "mcp")'. This adds meaningful semantic context beyond the bare schema, though it doesn't elaborate on source format constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Scan text for prompt injection and security threats'), the resource being acted upon ('text'), and distinguishes it from sibling tools like scan_batch (which handles batch processing) and get_patterns (which retrieves detection patterns). The verb 'scan' combined with the detailed threat detection scope makes the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: when scanning individual text inputs for security threats. It implicitly distinguishes from scan_batch (for batch processing) and get_patterns (for pattern retrieval), but doesn't explicitly state when NOT to use it or name alternatives. The guidance is sufficient but not exhaustive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• health_checkA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does well by disclosing key behavioral traits: 'No API key required' (authentication needs), 'Returns the service status, API version, number of active detection patterns, and response time' (output content), and 'Check if...is healthy and responding' (read-only nature). It doesn't mention rate limits or error behavior, keeping it from a perfect score.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly structured and concise: first sentence states purpose, second provides authentication context, third lists return values, fourth gives usage guidance, and fifth summarizes returns. Every sentence earns its place with zero wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (zero parameters, has output schema), the description is complete. It explains purpose, authentication requirements, return values, and usage context. The output schema will handle detailed return structure, so the description doesn't need to duplicate that information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters (schema coverage 100%), so the baseline is 4. The description appropriately doesn't discuss parameters since none exist, focusing instead on what the tool does and returns.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('check if...is healthy and responding') and identifies the target resource ('ClawGuard Shield API'). It distinguishes from siblings by focusing on system health rather than pattern retrieval, usage data, or scanning operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance is provided: 'Use this to verify connectivity before running scans.' This tells the agent exactly when to invoke this tool (pre-scan verification) and implicitly suggests alternatives (use other tools for actual scanning operations).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.