scan_tests

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It does so effectively: it explains what the tool detects, how detection differs across languages (exact vs. heuristic), and that every finding includes file and line. It also discloses that include_skipped findings are reported at 'info' severity. The one gap is that it does not explicitly state the tool is read-only (no file modifications), though 'scan' strongly implies it. Overall, it is far more transparent than typical tool descriptions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured and front-loaded: the core purpose is stated in the first sentence, followed by detailed detection rules, then language-specific behavior, and finally parameter details. Each paragraph adds distinct value without repetition or fluff. The format is scannable and efficient for an agent.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the presence of an output schema, the description need not enumerate return fields, but it still provides rich context: what the tool detects, severity semantics, language-specific accuracy, and parameter behavior. An agent has all necessary information to invoke the tool correctly, choose appropriate inputs, and interpret results. It is complete for a scanning tool of this complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate, and it does. The 'Args' section fully explains each parameter: path (file/dir, auto-skipped dirs), include_skipped (why include, severity implications), and max_findings (cap behavior, summary counts all). This goes beyond the schema by providing behavioral context, making parameter semantics extremely clear.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Scan a file or directory for tests that cannot fail.' It then enumerates concrete categories of such tests (true-by-construction assertions, no assertions, empty bodies, skipped). The purpose is unambiguous and distinct from sibling tools list_rules and explain_rule, which operate on rules rather than tests.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clarifies when to use the tool: it is for scanning files/directories for non-failing tests, and it explicitly notes that build/dependency directories are auto-skipped. It also distinguishes Python analysis (exact via ast) from Rust/JS (heuristic), helping agents set expectations. While it does not name alternative tools for comparison (the siblings are clearly rule-focused, not test scanners), the use case is sufficiently well-scoped. No explicit when-not-to-use guidance, but this is minor given the clear domain.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.