umbra_audit

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations, the description discloses important behaviors: deterministic gap detection, an execution gate that keeps only runnable tests that genuinely fail or error, explicit statement that it 'never edits the test file,' and that the OpenAI backend reads only a locally configured key after confirmation. This provides rich context about side effects and security.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is lengthy but well-structured and front-loaded with the core purpose. Each sentence contributes useful details (capabilities, workflow, constraints). It could be slightly trimmed, but the overall structure is appropriate for the tool's complexity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Even though there is no output schema, the description explains what the tool returns (coverage report, Silence index, proof requests, HTML and Markdown report artifacts) and describes execution side effects and backend behavior. It is complete for a complex tool with no output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema description coverage, the schema already explains all six parameters. The tool description adds workflow context (e.g., using only module_path and tests_path for report generation) but does not significantly enrich the meaning of individual parameters beyond what the schema provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific action: 'Locate test-suite blind spots' and names the exact languages and gap types (unexercised branches, untested input regions, unasserted return contracts, unasserted exception paths). This clearly distinguishes the tool from siblings like umbra_verify_proposal, umbra_write_report, and umbra_glossary.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear context and a workflow: 'After it returns, write and verify a failing-test proof for each returned request, then generate both reports with module_path and tests_path only.' It also provides safety constraints like 'never pass a key as tool input.' However, it does not explicitly name alternative tools for when-not-to-use scenarios, so it falls short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.