PromptBrake Free Tools

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With readOnly/idempotent/destructive annotations already covering the safety profile, the description still adds non-obvious behavior: it does not execute the tests it builds, and its checks cover observed calls and typed arguments rather than backend effects or independent permission enforcement. That last sentence usefully bounds what the tool actually verifies.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Four short sentences, front-loaded with what is produced and then the prerequisites and scope limits; every sentence adds a constraint rather than restating the title. Slightly dense but no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return-shape detail is unnecessary, and the description still names the produced file. Combined with the prerequisite statement and the explicit limits on what is not validated, an agent has enough to decide and invoke; the missing piece is sibling differentiation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Top-level schema description coverage is 0% for the single required `tests` parameter, but the description conveys the cardinality (1-20 checks) and hints at the rule semantics (observed calls vs typed arguments). The nested schema already carries description text for id, name, rule, and prompt, so the description only partially compensates.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific verb+resource with scope ("Build 1-20 tool invocation checks for an authorized staging agent") and names the output artifact (tool-tests.json). It does not, however, distinguish itself from the similarly named sibling build_test_pack, leaving the agent to infer the split from the name alone.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It states preconditions (a one-time dispatcher capture is required, run via the free PromptBrake Action, no license needed) which is real usage context. But it never says when to prefer this tool over build_test_pack or the other siblings, and gives no when-not guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnly, idempotent, non-destructive), yet the description adds substantial context beyond them: no account/API key needed to build, runner + CI access + PB_CUSTOM_TESTS_FILE required to run, checks are case-insensitive text comparisons that do not judge meaning or verify backend actions, invalid packs raise a validation error, and test content is not stored or executed while request metadata is logged.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with purpose and output, then usage, then prerequisites, then check semantics. Every sentence carries content, though the density of constraints makes it longer than strictly necessary for a one-parameter tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description correctly explains the return value (JSON text to save as tests.json) and the failure mode (tool error on invalid packs). Combined with the prerequisites and check semantics, an agent has everything needed to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and there is a single parameter, so the schema already documents items, required fields, maxItems, and case-insensitive comparison. The description restates the 1-20 bound and the case-insensitive/whitespace behavior but adds no syntax or format detail the schema lacks.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Validate 1-20 response tests') plus the concrete output ('return JSON text to save as tests.json for PromptBrake CI'). It explicitly distinguishes itself from the sibling map_owasp_llm_risk, so an agent can route correctly without opening schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Names the triggering condition ('when the user has prompts and expected or forbidden response text') and the alternative for the other case ('use map_owasp_llm_risk for risk-based test ideas'). It also clarifies the no-account path versus the separate execution prerequisites, which is real routing information.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover read-only, idempotent, closed-world, non-destructive behavior. The description adds genuinely new context: no account or API key is required, no target requests or scans are executed, and the returned instructions are test data rather than instructions to follow — a critical framing for injection content. It stops short of describing the returned text's structure or volume, but with a text payload that gap is minor.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, zero waste, leading with what is retrieved before the authorization caveat and the sibling routing. Every clause earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-enum-parameter read tool with no output schema, the description covers purpose, safety framing, auth preconditions, execution scope, and alternative tools. Nothing an agent needs in order to select or invoke it is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% with a fully enumerated category parameter, so the schema already carries the semantics. The description only restates 'one attack category' without adding format or selection guidance beyond the enum, which is the expected baseline when the schema does the lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource (retrieve bundled prompt-injection test inputs as text) scoped to one attack category, and explicitly differentiates itself from siblings map_owasp_llm_risk and build_test_pack. An agent can distinguish all three without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Names the alternatives and the condition that selects each: map_owasp_llm_risk for risk-based guidance, build_test_pack for packaging response checks. It also states the authorization precondition (a chatbot/API the user owns or is authorized to test), removing any ambiguity about when this is appropriate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/idempotent/destructive=false, but the description adds behavior those hints cannot convey: no account or API key required, it reads bundled guidance without contacting a target or running tests, and it explicitly disclaims security/compliance certification. That is real scope and limitation disclosure beyond structured fields.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Three compact sentences, front-loaded with purpose, then routing, then constraints. Every clause carries information an agent needs; there is no boilerplate or repetition of the title.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description compensates by enumerating what the returned text report contains (risk explanation, labeled prompts, signals to inspect, suggested role, PromptBrake coverage). For a single-enum, read-only tool, an agent has everything needed to select and invoke it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the single enum parameter is fully documented in the schema, including lowercase-only and 'ID not title' guidance, so the description adds little. The description re-states 'one OWASP LLM risk ID' but does not enumerate or explain the llm01-llm10 values beyond what the enum already provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (Map), a specific resource (one OWASP LLM risk ID), and the exact artifact produced (bundled test guidance: explanation, prompts, signals, role, coverage). It is distinguishable from all three siblings without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an explicit trigger ('when planning tests for a risk category') and routes two alternatives by name with their selecting conditions (get_prompt_injection_payloads for attack payloads, build_test_pack for a CI response-check pack). Nothing is left to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish readOnly, idempotent, non-destructive, closed-world behavior. The description adds meaningful boundary context beyond them: no account or API key needed, records supplied decisions only, and does not run tests, deploy, or establish security/compliance. It stops short of describing how the completeness score is computed or output size limits.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, output list front-loaded, then usage routing, then boundary caveats — no filler. It is dense, packing four outputs and two exclusions into one sentence, but every clause earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description carries the return-value burden and does so by enumerating the four returned artifacts. Given 12 required parameters and complex release semantics, the coverage of inputs rests on a fully described schema, so the description is nearly complete, only lacking scoring/format details.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% across 16 parameters, with enums and constraints documented inline, so the description need not restate them. It adds no format or syntax guidance for the parameters it names, so baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: it turns release decisions into four named artifacts (Markdown plan, completeness score, open decisions, PromptBrake gate policy YAML) returned as text. It explicitly differentiates from the sibling build_test_pack, so an agent can select it without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an explicit when: 'Use before validating a release to identify missing decisions,' plus a named alternative, 'use build_test_pack for executable response-check definitions.' This routes the agent between the two planning/validation tools without inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.