opa-mcp-server
Server Quality Checklist
Latest release: v0.6.0
- Disambiguation3/5
Most tools have detailed descriptions and distinct resource/action targets, but there are several close pairs: opa_config/opa_status return essentially the same config document, rego_eval_with_explain/rego_explain_decision both produce full traces, and rego_compile_query/opa_compile_query are the same operation via different engines. The descriptions help, but the overlap is more than a stray pair, so agents will need care to avoid misselection.
Naming Consistency4/5Names consistently use snake_case prefix_verb_noun (opa_/rego_/conftest_), and most tools follow an action-first pattern. Deviations like opa_status, opa_config, rego_deps, rego_coverage_gaps, and mcp_server_info break the otherwise predictable scheme, and the opa_/rego_ prefix split doesn't consistently distinguish server vs CLI operations.
Tool Count2/552 tools is far beyond the typical well-scoped MCP surface, even for the broad OPA/Rego/Conftest domain. Several variants (rego_eval_with_*, rego_format/rego_format_write, opa_config/opa_status) could be consolidated, and the set would be better split into separate servers. It feels overloaded rather than curated.
Completeness5/5The surface is exceptionally thorough: policy/data CRUD, query/eval/explain, testing/coverage, bundle build/sign/verify, schema validation, linting/fix, and Conftest pull/push are all covered. I don't see obvious dead-end workflows or missing lifecycle operations for the stated OPA/Rego/Conftest domain.
Average 4.3/5 across 52 of 52 tools scored. Lowest: 3.4/5.
See the Tool Scores section below for per-tool breakdowns.
- No community issues in the last 6 months
- 111 commits in the last 12 weeks
- Last stable release on
- No critical vulnerability alerts
- No high-severity vulnerability alerts
- No code scanning findings
- CI is passing
This repository is licensed under MIT License.
This repository includes a README.md file.
No tool usage detected in the last 30 days. Usage tracking helps demonstrate server value.
Tip: use the "Try in Browser" feature on the server page to seed initial usage.
This repository includes a glama.json configuration file.
This server has been verified by its author.
How to sync the server with GitHub?
Servers are automatically synced at least once per day, but you can also sync manually at any time to instantly update the server profile.
To manually sync the server, click the "Sync Server" button in the MCP server admin interface.
How is the quality score calculated?
The overall quality score combines two components: Tool Definition Quality (70%) and Server Coherence (30%).
Tool Definition Quality measures how well each tool describes itself to AI agents. Every tool is scored 1–5 across six dimensions: Purpose Clarity (25%), Usage Guidelines (20%), Behavioral Transparency (20%), Parameter Semantics (15%), Conciseness & Structure (10%), and Contextual Completeness (10%). The server-level definition quality score is calculated as 60% mean TDQS + 40% minimum TDQS, so a single poorly described tool pulls the score down.
Server Coherence evaluates how well the tools work together as a set, scoring four dimensions equally: Disambiguation (can agents tell tools apart?), Naming Consistency, Tool Count Appropriateness, and Completeness (are there gaps in the tool surface?).
Tiers are derived from the overall score: A (≥3.5), B (≥3.0), C (≥2.0), D (≥1.0), F (<1.0). B and above is considered passing.
Tool Scores
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds useful behavioral detail beyond annotations by disclosing the return shape: 'Returns the standard {result: [...]} shape.' However, it does not mention potential side effects from builtins like http.send, the mutual exclusivity of source/paths, or behavior under partial evaluation. The annotations only provide readOnlyHint=false and openWorldHint=true, so more transparency would help.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise: two sentences, with the core purpose and output shape front-loaded. The 'bread-and-butter authoring tool' phrase adds a small amount of usage flavor but is slightly filler; nevertheless, the overall structure is tight and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 8 parameters and no output schema, the description provides the essential top-level contract—evaluation semantics and the standard result shape—but it does not address caveats around partial evaluation, unknowns, or side effects. It also does not guide the agent toward the eval_with_* siblings when specialized output is needed, so completeness is adequate but not strong.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline of 3 applies. The description's phrase 'against a policy and an input document' provides a high-level mapping to the paths/source and input/inputPath parameters, but it does not add meaningful semantics beyond what the input schema already documents.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the verb and resource: it evaluates a Rego query against a policy and input document using `opa eval`, and even states the output shape. It stops short of a 5 because it does not explicitly distinguish itself from sibling tools like rego_eval_with_explain, rego_eval_with_profile, and rego_eval_with_coverage beyond the phrase 'standard'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Calling this 'the bread-and-butter authoring tool' implies it is the default/basic evaluation tool, which offers some usage context. However, it never explicitly states when to prefer this tool over the eval_with_* variants or related tools like rego_check, so the guidance remains implicit rather than explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true, making the delete behavior clear. The description adds minimal extra context ('from the running OPA server') but does not disclose potential side effects, authentication needs, or constraints beyond the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
A single, front-loaded sentence with no wasted words. Every word is necessary and clear.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple one-parameter schema and no output schema, the description is mostly complete. However, it could mention error handling (e.g., policy not found) or that deletion is permanent, which would raise completeness to 5.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with 'Policy ID to delete.' in the parameter description. The tool description adds no further meaning, meeting the baseline of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Delete', the resource 'a policy', and specificity 'by ID from the running OPA server'. This distinguishes it from sibling tools like opa_get_policy or opa_delete_data.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives (e.g., opa_put_policy to update) or prerequisites like ensuring the policy exists. The description only states the action without contextual usage advice.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate destructive and idempotent behavior. The description adds that the policy is uploaded as raw text/plain and parsed server-side, and that it replaces any existing policy with that ID, providing useful behavioral context beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no redundant information. It is concise and front-loaded with the key action. Could potentially be structured as a brief paragraph but still efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with no output schema, the description covers core behavior (replace, raw text). However, it does not mention return values or error conditions, which would be helpful for completeness given the tool's destructive nature.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema descriptions already define 'id' and 'source' adequately. The description adds that the source is raw text/plain, which is helpful but not extensive. With 100% schema coverage, the description does not significantly enhance parameter understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Upload' and resource 'Rego policy' with a given ID. It distinguishes from sibling tools like opa_get_policy and opa_delete_policy by specifying the upload/replace action.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as opa_put_data or opa_bundle_build. There is no mention of prerequisites or context where this tool is preferred.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds that the tool uses 'opa parse' and returns a tree of specific node types, but does not disclose additional behavioral traits like error handling, output format details, or performance characteristics. The added value beyond annotations is modest.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences: the first clearly states the action and implementation, the second provides usage context. No extraneous information; every sentence is purposeful. Highly concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple input (one string parameter), annotations covering safety/idempotency, and no output schema, the description is fairly complete. It explains what the tool does, how it works (opa parse), and the general output structure. Minor gap: no detail on error cases or output format beyond node types, but sufficient for a parse tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a clear description for the 'source' parameter. The tool description adds little beyond the schema, stating it parses Rego to AST and mentioning the output structure, but does not enrich parameter semantics further. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool parses Rego source to a JSON AST using 'opa parse' and lists the AST node types. However, it does not distinguish from sibling tools like rego_check or rego_eval, though the purpose is unique among siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives a use case ('when you need to introspect policy structure programmatically') but does not provide explicit guidance on when not to use this tool or which alternatives exist. It implies usage context but lacks exclusion criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true, readOnlyHint=false, and idempotentHint=true, covering the safety profile. The description adds that the output is a .tar.gz archive and that signing is optional, but it does not disclose additional behavioral details such as overwriting existing outputs or the fact that signing implies bundle mode (though the schema covers the latter). No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences with no wasted words: purpose first, then output format, then supported features. The description is front-loaded and every sentence contributes useful information. The mention of `opa build` clarifies the underlying command without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (16 parameters, 2 required, no output schema), the description is a useful but high-level summary. It does not guide the agent on when to choose this tool over opa_bundle_sign/verify, nor does it highlight important constraints like entrypoints being required for wasm/optimize; the schema covers those details, but the description alone is not fully complete for a tool of this complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 16 parameters. The description adds a high-level summary of key capabilities (optimization, revision strings, WASM target) that maps to parameters, but it does not provide meaningful new meaning beyond what the schema already states. Baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states a specific verb ('Build') and resource ('deployable bundle from policy / data paths') and adds concrete output details (.tar.gz, optional signing, WASM target). This distinguishes it from related siblings like opa_bundle_sign and opa_bundle_verify, which handle signing/verification rather than building.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for building deployable bundles from policy/data paths, but it does not explicitly contrast this with related tools such as opa_bundle_sign, opa_bundle_verify, or rego_eval. There is no explicit when-to-use or when-not-to-use guidance, so the agent must infer the boundary from the description and sibling names.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate this is a destructive, non-idempotent write operation. The description adds useful behavioral context by explaining the operation format and that omitting path and segments patches the root to add a new top-level document. It does not go into further side effects, but the annotation covers the main destructive risk.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences with no wasted words. It front-loads the core action, then gives the operation shape, then handles the important root-patch special case. Each sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a three-parameter tool with annotations covering the destructive nature and a schema covering all parameters, the description provides the remaining key context: how operations are structured and how to target the root. There is no output schema, but return-value details are not critical for invoking this tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description adds value beyond the schema by clarifying the JSON Patch operation shape and the special root-patching behavior when both path and segments are omitted. This is meaningful parameter-level guidance an agent would not get from the schema alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the action: applying an RFC 6902 JSON Patch to the OPA data document. It names a specific verb and resource and is distinct from sibling tools like opa_put_data and opa_delete_data, though it does not explicitly differentiate itself from them in the description.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when this tool is useful by defining it as the JSON Patch mechanism for data, and it gives a concrete usage tip about omitting path/segments to patch the root. However, it does not explicitly state when to use this tool instead of alternatives such as opa_put_data or opa_delete_data.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations provide readOnlyHint=false and openWorldHint=true, so the description carries some burden for behavioral disclosure. It adds useful behavior beyond annotations by specifying `--explain=full` and the structured trace output, but it does not discuss side effects, output size, or error behavior. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with the core behavior front-loaded and the usage context in the second sentence. There is no filler, and every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description should explain return values more concretely; it only says 'structured trace alongside the result' without detailing trace shape or caveats. The 8 parameters are fully schema-documented, so the gap is moderate rather than severe.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the input schema already documents all 8 parameters. The description adds context about the evaluation mode but no parameter-level meaning beyond what the schema provides, so the baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: evaluate Rego with `--explain=full` and return a structured trace alongside the result. It clearly differentiates from plain evaluation via the trace, though it does not explicitly name `rego_eval` as the non-trace sibling.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says to use this tool when an agent needs to see why a rule fired or didn't, and it connects the trace to `rego_explain_decision`. It does not provide when-not-to-use guidance or name alternatives directly, but the intended usage context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint=true and idempotentHint=true. The description adds value by specifying the exact HTTP endpoint and explaining the concept of partial evaluation (substituting knowns). This provides behavioral context beyond annotations without contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences efficiently deliver the action and result. No extraneous text. The first sentence is front-loaded with the verb 'compile' and endpoint, making it immediately actionable.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the tool's core behavior and return value (residual query) without an output schema. It assumes familiarity with OPA concepts but is sufficient for an agent. Could add more on use cases or prerequisites, but is adequate for the complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear descriptions for all three parameters. The tool description reinforces the purpose of partial evaluation but does not add new parameter-specific details beyond the schema. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool sends a query to the OPA server's /v1/compile endpoint for partial evaluation and returns the residual query. This distinguishes it from evaluation tools like rego_eval or opa_query_decision, showing a specific verb and resource.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not explicitly state when to use this tool vs alternatives such as rego_eval or opa_query_decision. It only mentions partial evaluation but gives no guidance on scenarios or exclusions, leaving the agent to infer usage context from the tool name and siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations (readOnlyHint: false, openWorldHint: true) provide minimal behavioral signal, so the description carries much of the burden. It discloses that it runs `opa bench` and returns timing data, but it does not clarify the side-effect profile despite readOnlyHint: false, nor warn that benchmarking repeats query execution and can be expensive. There is no contradiction with the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each earning its place: action, return values, and use case. The primary verb is front-loaded in the first sentence, and there is zero filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a moderate-complexity tool with no output schema, the description covers purpose, key return fields, and intended use case, while the schema handles parameter semantics. It could add a note about execution cost or a fuller description of the result document structure, but nothing critical is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, and the `count` parameter's schema description already explains runs/fastest/raw behavior, so the description correctly avoids repeating parameter details. The phrase 'policy + input' loosely maps to paths/input/inputPath but adds no syntax or format information beyond what the schema already provides, matching the baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb+resource ('Benchmark a Rego query against a policy + input') and names concrete outputs (iterations, ns/op, allocation counts), which separates it from siblings like rego_eval and rego_test. It does not explicitly name a competing sibling such as rego_eval_with_profile, but the statistical-timing return values make the distinction largely clear.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
'Use this to spot slow rules' gives a direct when-to-use signal tied to a concrete goal. It stops short of a 5 because it offers no exclusions or explicit alternatives — for instance, no guidance on when to prefer rego_eval_with_profile or rego_test instead.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and idempotentHint=true, so the safe, read-only nature is clear. The description adds behavioral context by detailing the return data (manifest, namespaces, annotations, signature metadata) and the accepted input types, which goes beyond the annotation flags.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise—two sentences that succinctly state the action and the outputs. Every word adds value; no redundancy or verbosity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one parameter and no output schema, the description adequately covers what the tool does and what it returns. Minor missing details like error behavior or format specifics are not critical given the tool's straightforward nature.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema covers the single 'target' parameter completely with a clear description of possible values. The tool description echoes this but adds no new semantic information beyond the schema. With 100% schema coverage, the baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's action ('Inspect') and resource ('OPA bundle, policy directory, or single Rego file') and lists the specific outputs (manifest, namespaces, rule annotations, signature metadata). It distinguishes from siblings like rego_check or rego_eval by focusing on structural inspection.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not provide explicit guidance on when to use this tool versus alternatives. While the purpose implies it's for inspection of bundle structure, there is no mention of when not to use it or how it differs from similar sibling tools like rego_deps or rego_describe_policy.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already signal readOnlyHint=false and openWorldHint=true, so the description's main added behavioral value is clarifying that coverage data is returned per line. It does not discuss side effects, output format details, or interactions with partial evaluation, but it is not misleading.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two focused sentences convey the core behavior and the intended use case without redundancy. The first sentence states what it does; the second explains why an agent would choose it.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is adequate for a familiar Rego evaluation tool: it names the key output (per-line coverage) and the motivating use case. However, with no output schema, an agent would still lack details about the exact coverage result structure and whether normal query results are also returned.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema documents all 8 parameters. The description adds no parameter-level detail beyond indicating coverage is enabled, but that is acceptable because the schema carries the burden.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Evaluate') with a clear resource ('Rego with --coverage') and states the distinctive output ('per-line coverage data'). This differentiates it from rego_eval and other evaluation siblings by naming the exact coverage behavior.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a clear use case: verifying that tests actually exercise the rules they're meant to. It does not explicitly name alternatives or state when not to use this tool, so it stops short of full routing guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description clearly discloses the key behavioral trait: unknowns defaults to ["input"] and the residual policy encodes a conditional implication. It also explains the open-world framing of 'given input X...' beyond the readOnlyHint/openWorldHint annotations. It does not discuss side effects or error behavior, but nothing contradicts the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences with no filler; the core operation is front-loaded and each sentence adds either behavior or usage guidance. The structure is easy to scan.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description explains the central behavior and use case, but omits how the residual policy is returned/formatted and does not reconcile with the `partial` boolean parameter, which can disable partial evaluation. With no output schema and eight parameters, that leaves some ambiguity for an agent deciding how to call and interpret the result.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already documents all eight parameters. The description adds meaningful semantics by explaining the `unknowns` default and the substitution behavior that makes partial evaluation useful.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific operation, "Run partial evaluation on a query," and an output, "the residual policy," so an agent can understand what the tool does. It does not explicitly distinguish it from the closely named sibling opa_compile_query, though the partial-evaluation framing implies the difference.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives concrete use cases: "offline policy slicing or pre-computing decision sets," which tells an agent when this tool is appropriate. It does not state when not to use it or name alternative siblings, so it falls short of fully explicit routing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and idempotentHint=true, so the description's mention of 'static analysis' adds context but does not disclose additional behavioral traits like performance or side effects beyond what annotations provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence that front-loads the purpose and key details. No redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite no output schema, the description explains what the tool returns (base and virtual document references, transitively). It covers purpose, parameters, and output sufficiently for a static analysis tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, baseline 3. The description adds meaning by explaining the ref format (e.g., 'data.example.allow') and the paths constraint (must be inside allowed root), which adds value beyond the schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool performs static dependency analysis for a Rego reference, specifying the target ref format and what it returns (base and virtual document references). This distinguishes it from sibling tools like rego_check or rego_eval.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for dependency analysis but does not explicitly state when to use this tool versus alternatives like rego_eval or rego_explain_decision. No when-not or alternative guidance is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses the key behavioral output (per-rule timing and evaluation counts), which goes beyond the annotations. However, readOnlyHint is false and the description does not clarify whether evaluation has any side effects or performance costs; the annotation burden is only partially addressed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no filler. The core behavior and output are front-loaded, and the purpose statement efficiently directs the agent to the use case.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 8 parameters and no output schema, the description adequately conveys the return value and typical use. It does not explain edge cases or interactions with options like partial evaluation, but those are documented in the schema and the core purpose is clear.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters. The description adds nothing about parameter usage or constraints, but this is acceptable because the input schema carries the full parameter documentation burden.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Evaluate'), the exact resource ('Rego with --profile'), and what it returns ('per-rule timing and evaluation counts'). It also distinguishes itself from the many sibling eval tools by naming its purpose: finding hot rules in slow policies.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives a clear use case: 'Use this to find hot rules in slow policies.' It implies when this tool is appropriate over other eval variants, though it does not explicitly name alternatives or say when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent, non-destructive. The description adds that it returns one suggestion per diagnostic and confidence levels (high/medium/low). This provides useful behavioral context beyond annotations, though it does not detail the output structure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with purpose, no unnecessary words. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity and no output schema, the description explains input source, output quantity, and confidence levels. It does not describe the suggestion structure, but for a low-complexity tool, this is nearly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for all fields. The description only adds that diagnostics should come from rego_check or rego_lint, which is helpful but minimal. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it maps compile errors and lint findings to fix suggestions, and specifies the source diagnostics. However, it does not explicitly differentiate from sibling tool rego_fix, which may apply fixes, leaving some ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly instructs to pass diagnostics from rego_check or rego_lint, providing clear usage context. Does not mention when not to use or alternatives, but the context is sufficient for an AI agent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations, it discloses exact return behavior (`per-file pass/fail`), the `NO_TESTS_FOUND` case, and the `CONFTEST_NOT_FOUND` error condition plus the binary resolution order. These are important runtime behaviors that the schema and annotations don't express. Nothing here contradicts the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description moves from core behavior to equivalence, return format, and runtime requirements in a few compact sentences with no filler or repeated schema content. Every clause earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with no output schema, the description still covers return format, error conditions, and external dependency requirements. Combined with the fully documented input schema, an agent has enough context to invoke it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already carries the parameter definitions including allowed-root constraints and defaults. The description adds some helpful context by explaining what a policy directory must contain (`test_*` rules in `*_test.rego`), but it doesn't add meaning for `data` or `namespace`.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states a specific verb and resource: it runs `test_*` rules in `*_test.rego` files and frames the purpose as verifying that policies are correct. It also contrasts itself with `opa test`, which clarifies its niche, but it doesn't explicitly distinguish from closely named siblings like `conftest_test` or `rego_test`.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage context is implied by the explicit reference to conftest's policy-loading machinery and the prerequisite that `conftest` must be on PATH or `CONFTEST_BINARY` set. However, the description never names sibling alternatives or states when to prefer this tool over `conftest_test`/`rego_test`.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare readOnlyHint, idempotentHint, and destructiveHint, which the description aligns with by stating it 'parse[s] a Rego policy and return[s] a structured summary.' The description adds useful behavioral detail beyond annotations, such as listing specific output fields (clauseCount, isDefault, etc.).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, consisting of three sentences that flow logically: what the tool does, details about what it returns, and a use case. Each sentence adds value without unnecessary verbosity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having no output schema, the description adequately describes the return value (package, imports, rules, and rule details) and covers the tool's functionality for a single-input, simple tool. It is complete enough for an agent to understand the tool's purpose and output.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 100% coverage for the single parameter 'source' with description 'Rego source to describe.' The tool description does not add significant new meaning, as the schema already explains the parameter adequately. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool parses a Rego policy and returns a structured summary including package, imports, rules, and detailed rule attributes. It distinguishes itself from sibling tools like rego_eval, rego_check, and rego_inspect by focusing purely on structural description.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly suggests using this tool as 'the first step in any 'what does this policy do' workflow,' providing clear context for when to use it. It does not explicitly mention when not to use it or contrast with alternatives, but the guidance is sufficient.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations (readOnlyHint=false, openWorldHint=true), the description discloses the return shape — findings grouped by severity (high/medium) with remediation guidance — and the prerequisite that regal must be installed. It also reveals the rule-selection behavior (correctness rules tied to policy bypasses plus custom security rules). Nothing stated contradicts the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences cover purpose, output format, usage context, and a prerequisite with no filler or repetition. The core action and scope are front-loaded in the first sentence, with the 'Requires regal' caveat appropriately tucked at the end.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description compensates by describing the findings format (grouped by severity, with remediation guidance). The allowed-roots path constraint lives in the schema, and fleet-vs-per-file guidance covers usage context. Minor gaps like exit-code behavior are acceptable for a non-mutating lint tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with paths, configFile, and ignoreFiles each documented, including the allowed-roots constraint on paths and the fleet-roots hint. The description adds no parameter-specific meaning beyond what the schema already provides, so the high-coverage baseline of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific verb and resource: it runs regal lint restricted to the `bugs` category plus custom `security`-category rules across policy directories. This precise rule-subset scope differentiates it from the general sibling rego_lint without needing to open that tool's schema. The action, resource, and scope are all explicit and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit when-to-use guidance: 'Use this for a periodic fleet-wide sweep rather than per-file style review.' This clearly frames the intended context and rules out per-file review, but it stops short of naming a specific alternative tool for that excluded case, which keeps it from a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructive and idempotent hints; the description adds non-obvious runtime behavior: the body is JSON, path separator parsing switches between dots and slashes, and dot-containing keys can be addressed via slash-separated paths. This is meaningful context beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Every sentence earns its place: purpose, body format, separator rule, and segments fallback. The most important verb-first statement is front-loaded and the paragraph is dense without padding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tricky path-encoding behavior is fully explained, and annotations cover the destructive/idempotent safety profile. There is no output schema and no response description, but for a write operation the essential calling requirements are covered.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3, but the description elevates it by explaining how `path` is parsed, why `hosts/example.com` works, and when `segments` is the right parameter. It adds practical meaning not fully present in the schema's field descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action ('Write or replace a value') on a specific resource (OPA data path), which clearly distinguishes it from siblings like opa_patch_data and opa_delete_data. The 'replace' wording communicates full overwrite rather than merge or delete.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives useful in-tool guidance for choosing path versus segments, but it never addresses when to use opa_put_data instead of opa_patch_data or opa_delete_data. Tool-vs-alternative selection is therefore left mostly implicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark this as read-only, idempotent, and non-destructive. The description adds valuable behavioral context by specifying the POST method, the data-path endpoint, and that the response is 'whatever the rule produces'. It does not detail error or undefined-rule behavior, but the annotations lower the burden.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with no wasted words. It front-loads the action and endpoint, then gives a concrete example in the second sentence.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Without an output schema, the phrase 'returns whatever the rule produces' gives useful response expectations, and annotations cover the safety profile. The need to provide a path or segments is implied but not explicit, which is a minor gap given the schema hints.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already documents all five parameters with clear descriptions. The description reinforces the meaning of `path` and `input` through the data.X.allow example, but it does not add significant meaning beyond the schema, so the baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Evaluate'), a specific resource ('the running OPA server'), and the mechanism ('POSTs to the data path'). The quoted example, 'given this input, what does data.X.allow say?', clearly differentiates this from local evaluation siblings like rego_eval.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives clear context for when to use this tool: querying a running OPA server with an input document. It implicitly distinguishes from local rego evaluation tools, but it does not explicitly name alternatives or state when not to use this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds useful behavioral detail beyond the annotations: it states that files are sorted by coverage ascending and that the threshold limits the report. It also transparently states it invokes 'opa test --coverage.' With annotations being minimal and readOnlyHint false, this provides meaningful insight into the tool's execution and output behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences, each with a distinct purpose: the command and output, the purpose, and sorting/threshold behavior. There is no redundancy or filler; every sentence contributes to an agent's understanding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with no output schema, the description adequately explains the return value ('a per-file breakdown of uncovered line ranges') and the filtering/sorting behavior. It could be slightly more explicit about the exact structure of the returned data, but the essential information needed to call and interpret the tool is present.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description repeats the threshold behavior already present in the schema and does not add any additional meaning for paths or runPattern. It adds no new semantic value beyond what the schema already documents.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the exact verb and resource: 'Run opa test --coverage' and the specific output: 'a per-file breakdown of uncovered line ranges.' This distinguishes it from siblings like rego_test or rego_eval_with_coverage, which focus on running tests or evaluating coverage for specific expressions, not on identifying gaps in coverage.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear context for when to use the tool: to identify rules or branches not yet exercised by tests. It also explains how to customize usage via the threshold parameter. However, it does not explicitly mention alternatives or state when not to use this tool, such as when you simply need to run tests without coverage analysis.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With only sparse annotations (readOnlyHint=false, openWorldHint=true), the description adds useful behavioral detail: full tracing, a structured trace result, per-rule fired/not-fired summary, and the agent's expected role in interpreting the output. It does not clarify the side-effect ambiguity from readOnlyHint=false, but it does not contradict the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no filler. The first sentence states the core behavior and output; the second gives the concrete usage context and the agent's role. Every clause earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 8 parameters and no output schema, the description conveys the essential purpose, output shape, and agent workflow. It could be slightly more complete about what the structured trace contains or how partial evaluation behaves, but the schema covers parameters and the use case is well specified.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the input schema already documents all 8 parameters. The description adds no parameter-level meaning beyond confirming that a Rego query is evaluated, which is the expected baseline when the schema carries the parameter documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action and resource: evaluate a Rego query with full tracing and return a structured trace plus per-rule fired/not-fired summary. It clearly differentiates from siblings like rego_eval and rego_explain_undefined by focusing on the 'why was this denied?' diagnostic use case.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says when to use the tool: when you need to answer 'why was this denied?' and want the agent to narrate the trace without re-implementing tracing logic. It does not name alternative tools or explicitly state when not to use it, but the intended context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds useful behavioral context beyond annotations, such as parallel evaluation, shared input/query across both sides, and the return shape. However, with `readOnlyHint: false`, it does not clarify whether the tool has any side effects, and it does not disclose error behavior if one side fails to evaluate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and well structured: purpose first, then return value and execution behavior, then use cases, then parameter modes. Every sentence contributes information an agent needs.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex tool with 8 parameters and no output schema, the description covers the main invocation patterns, return fields, and use cases. It could be more complete with an example or error-handling notes, but the essential context is present.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds meaningful grouping: sourceA/sourceB and pathA/pathB represent the two policy sides, and the same input and query are used for both evaluations. This clarifies the mental model beyond individual property descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states a specific action: evaluate the same query against two Rego policies and compare the results. It also names its unique output (`equal`, raw result per side, `changedPaths`), which distinguishes it from the many single-policy eval siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly gives use cases: verifying a refactor preserves behavior and understanding where two policies diverge. It does not name alternative tools or state when not to use it, but the guidance is clear enough for an agent to select it appropriately.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark the tool as non-read-only and destructive; the description adds meaningful context beyond that: it never handles credentials, registry auth must be pre-configured externally, the policy path must be inside allowed roots, and `conftest` must be on PATH or `CONFTEST_BINARY` set. It could add overwrite/tag-replacement semantics, but the description meaningfully enriches the annotation profile.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences, each load-bearing: the main action, the credential-handling caveat, and the path/binary prerequisites. Information is front-loaded and there is no filler or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive push operation with no output schema, the description covers the command, target registry, credential model, path restrictions, and binary prerequisite. An agent has enough information to decide whether it can invoke the tool and what side effects to expect.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, and both parameter descriptions already document path constraints, fallback behavior, allowed roots, and registry credential requirements. The tool description mostly restates these, adding only environment-level context like `CONFTEST_BINARY` rather than new parameter-level meaning, so the baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific verb and resource pair: package the local Rego policy directory as an OCI artifact and push it to a registry using `conftest push`. This clearly distinguishes it from siblings like `conftest_pull`, `conftest_test`, and the various rego_ inspection tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
It clearly states when to use the tool (when publishing a policy directory as an OCI artifact) and lays out prerequisites: pre-configured registry credentials, the policy path inside OPA_MCP_ALLOWED_PATHS, and `conftest` availability. It does not explicitly name alternatives or exclusions, but the push-scope and inverse sibling `conftest_pull` make the intended use obvious.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, so the safety profile is covered. The description adds valuable behavioral detail about path interpretation: dotted notation versus slash-only separator, and how a key containing a dot can still be addressed. This goes beyond what annotations and schema alone provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three tightly written sentences with no filler. The core action is front-loaded, and the necessary path-format nuances are packed efficiently into the remaining sentences.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a low-complexity read tool with strong annotations, the description is nearly complete. It covers the trickiest part: path formatting and segments selection. A minor gap is that it does not state what happens when neither `path` nor `segments` is provided, even though the schema allows zero required parameters.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage, the baseline is 3, but the description substantially enriches parameter understanding. It clarifies the dotted-path rule, the slash-only fallback, the `example.com` addressing case, and the exact condition for using `segments` instead of `path`. This resolves real ambiguity in how to invoke the tool.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Read a path from OPA's data hierarchy,' which names a specific verb, resource, and scope. This clearly differentiates it from siblings like opa_get_policy (policies) and opa_query_decision (decision evaluation) by targeting the data hierarchy specifically.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives strong internal guidance on when to use `path` vs `segments`, but it never names alternative tools or states when this tool should be preferred over opa_get_policy or opa_query_decision. Tool-selection context is implied by 'data hierarchy' but not made explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover the safe-read profile (readOnlyHint, idempotentHint, non-destructive), so the description's job was to add behavioral depth beyond that, and it delivers: the exact endpoint, 200-vs-otherwise result semantics, the key gotcha that an unactivated bundle yields { healthy: false } rather than a tool error, and the OPA_UNREACHABLE failure mode. These are precisely the interpretation cues an agent needs and cannot derive from annotations or the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, roughly 70 words, with no filler. The endpoint is front-loaded, followed by result interpretation, the unreachable edge case, and finally the flags — a logical order where every sentence earns its place. Nothing is redundant with the schema or annotations.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-required-param, read-only health check with no output schema, the description is fully sufficient: it names the endpoint, defines both success and failure result shapes, covers the edge cases (unactivated bundle, unreachable server), and documents both optional flags. There is nothing an agent needs in order to call this tool correctly that is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% — both bundles and plugins already carry adequate descriptions. The description's phrase 'query flags to require those subsystems to also be healthy' adds a small amount of meaning by tying the booleans to the subsystem-health concept, which aligns with and slightly reinforces the schema. Since the schema does the heavy lifting, the baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
Names the specific resource (`/health` endpoint) with a clear verb ('Hit'), and then defines the expected response semantics. This makes the tool immediately distinguishable from the many siblings in the namespace, especially opa_status and opa_config, without needing to open their schemas.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description conveys useful context about when to use the tool — checking whether the OPA server (and optionally its subsystems) is healthy — and clarifies that an unactivated bundle appears as a health result rather than a tool error, which affects result interpretation. However, it never explicitly names alternatives or gives when-to-use / when-not-to-use conditions, so routing among overlapping siblings like opa_status and opa_config is left to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses meaningful behavior beyond the annotations: it combines plain eval, full-trace eval, and per-condition AST analysis; it distinguishes runtime failures from indexer elimination; and it explains the default rule semantics and the 'queryResult' default reporting. This gives the agent a clear model of how the tool works.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The first sentence front-loads the core purpose, and the rest of the description earns its place by explaining edge cases and the internal approach. It is somewhat dense and uses technical terms like 'indexer elimination', but it remains structured and free of filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity and the absence of an output schema, the description provides a solid picture: it covers the input expected, the failure modes handled, the default-rule special case, and what the return value contains. It does not enumerate exact output fields, but 'structured breakdown' plus the queryResult mention is adequate for an agent to infer the result shape.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description adds value by clarifying the 'query' parameter: it must be a fully-qualified rule reference, matches what would be passed to rego_eval, and defaults are handled specially. This is useful guidance beyond the schema's simple descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a precise verb-resource pairing: 'Diagnose why a fully-qualified Rego query ... produces no value, or falls back to its default.' It gives a concrete example (data.authz.allow) and a clear scope that distinguishes it from eval and decision-explaining siblings by focusing on undefined/default diagnosis.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly states when to use the tool: when a fully-qualified query is undefined or falls back to a default. It also explains how default rules are treated, giving the agent actionable context. It does not explicitly name alternative tools or say when not to use it, so it stops short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description goes well beyond annotations. It discloses the external dependency (`regal` on PATH or `REGAL_BINARY`), the failure mode (`REGAL_NOT_FOUND`), the auto-disabling of path-dependent rules for inline source, and the `<inline>` path substitution. This is excellent behavioral disclosure that cannot be inferred from the schema. The `readOnlyHint: false` is consistent with a lint operation that launches an external process; no contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and information-dense. The first sentence states the core purpose; the second covers dependencies and failure modes; the third explains conditional behavior for inline source. It is front-loaded and every sentence earns its place. A small deduction because the inline-source caveat is long and could be trimmed.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers return categories, dependency requirements, inline-source behavior, and path handling. With 9 parameters and no output schema, the remaining gap is the exact shape of the returned violation objects (e.g., severity codes, rule IDs). Still, for an agent choosing and invoking the tool, the most important operational details are present. Could mention that `paths` must be within allowed roots, but that is already in the schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so every parameter already has a description in the schema. The tool description does not repeat parameter details, which is appropriate. However, it also doesn't add semantic context about how `enable`/`disable`/`enableCategory`/`disableCategory` interact or how `failLevel` maps to exit codes beyond what the schema already says. Baseline 3 is fair because the schema carries the load.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Lint'), a specific resource ('Rego source'), and names the actual linter ('Regal'). It also specifies returns 'categorized violations (style, bugs, idiomatic, performance) with file/line locations', which distinguishes it from other rego_* tools that analyze, transform, or evaluate Rego.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly explains how to invoke it with `paths` or inline `source`, and notes the behavior difference for the inline case. It doesn't explicitly spell out 'use X instead when...' alternatives, but the sibling list is large and the description's focus on inline-source behavioral detail implies the relevant context. Slight gap: no explicit statement about when to prefer rego_check, rego_fix, or rego_security_audit over this linter.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description goes well beyond annotations. It discloses the two-phase mechanism, the 'changed' flag behavior even when errors remain, and the INVALID_REGO fallback with opa's message. The readOnly and idempotent hints are consistent with this read-and-transform tool, and no contradictions exist.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single compact paragraph with phases numbered and edge cases stated. Every sentence contributes value, though it is slightly denser than necessary. The key purpose is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description carries the burden of explaining return behavior. It covers the main output (migrated source, changed flag), the error-handling behavior, and the remaining-issues scenario. It does not spell out the exact JSON response shape, but that is a minor gap given the detail provided.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers the only parameter (source) 100%, including its purpose and behavior. The tool description adds context about phases and return values, but it does not add new meaning about the parameter itself, so the baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Migrate Rego v0 source to Rego v1 syntax' and immediately details the two-phase process. This cleanly distinguishes it from formatting-only tools like rego_format or validation-only tools like rego_check, even though no sibling is explicitly named.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly communicates the intended use case—migrating v0 to v1—and explains the internal pipeline so an agent knows what will happen. However, it does not explicitly mention when to prefer an alternative (e.g., rego_fix or rego_format) or state exclusions, leaving the exclusion guidance implicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=true, destructiveHint=false, etc. The description adds value by specifying what the tool returns (name, version, runtime details) and that it checks binary reachability. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences: first states purpose, second provides usage guidance. No wasted words, front-loaded with key information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers return values (name, version, runtime details, binary status) adequately. No output schema, but the description provides sufficient context for a simple info tool. Minor gap: 'runtime details' is vague, but overall complete enough.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has no parameters, and schema coverage is 100%. The description does not need to add parameter info. Baseline for 0 parameters is 4, and the description adds no unnecessary detail.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool returns name, version, and runtime details of the opa-mcp server. The verb 'Return' and resource 'opa-mcp server instance' are specific. Among siblings which are mostly OPA/Conftest/Rego manipulation tools, this is the only info tool about the server itself, so differentiation is clear.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states two use cases: confirming the version of opa-mcp and verifying reachability of OPA, Regal, and Conftest binaries. While it doesn't mention when not to use it, the context is clear and no alternatives are needed as the tool is unique among siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover the safety profile comprehensively (readOnlyHint=true, idempotentHint=true, openWorldHint=true, destructiveHint=false), so the bar for the description is lower. The description adds genuine behavioral value beyond annotations by disclosing the response-cap behavior: enabling either include flag can cause list responses to exceed the cap. This is exactly the kind of operational trait an agent needs to anticipate failure modes.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, zero filler. The first sentence front-loads the action and return value; the second handles the optional parameters and the reason for the defaults. Every clause earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple listing tool with 0 required parameters, rich annotations, and no output schema, the description is nearly complete: it states the return value at a useful level ('policy IDs and a count') and explains both flags. It could marginally improve by describing the response envelope or ordering, but nothing an agent needs to invoke it correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with unusually rich per-parameter descriptions (size ratios, response-cap warnings, alternative-tool routing), which sets the baseline at 3. The description adds meaning on top by distinguishing the two flags at a semantic level — 'Rego text' vs 'parsed AST' — and stating the shared default-off behavior and its rationale, which is not fully redundant with the schema text.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource — 'List policies registered on the running OPA server' — and goes beyond that to specify the return value ('policy IDs and a count'). The phrase 'running OPA server' clearly differentiates this from the many rego_* sibling tools that operate on static policy files, and from opa_get_policy/opa_put_policy/opa_delete_policy which target individual policies.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context: use this to enumerate registered policies, and the optional include flags are discouraged by default because they 'push a list of any real size past the response cap.' This effectively tells an agent when NOT to set the flags. It does not explicitly name opa_get_policy as the alternative for fetching a single policy's source in the description body — that routing lives in the schema — so it stops just short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare destructiveHint=true and idempotentHint=true. Description adds details: writes to disk, dryRun preview, abort on parse failure. No contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Concise, front-loaded with main action, then key features (dryRun, return value, sibling differentiation, flags, error behavior). Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with 5 params and no output schema, description covers return format, error behavior, version flags, and safety. Could mention idempotency or permissions, but redundant with annotations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%. Description adds meaning: paths must be within allowed root, dryRun for preview, regoV1 adds import rego.v1, v0Compatible/v1Compatible for version-specific formatting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the tool runs `opa fmt --write` to format Rego files in place. Distinguishes from sibling `rego_format` by noting this writes to disk vs returning a string. Lists version flags.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly recommends using `dryRun: true` for preview and distinguishes from `rego_format`. Mentions abort on parse failure. Could explicitly state when not to use, but differentiation is sufficient.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and idempotentHint=true, but the description adds context: uses AST-level analysis via opa parse, no OPA server needed, and accepts inline source, files, or directories. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Every sentence adds value: purpose, method, use cases, input formats. Well-structured with no fluff or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Though no output schema, the tool returns a JSON Schema object which is self-describing. Description covers input modes thoroughly. Slight lack of output format details is acceptable given the tool's nature.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description adds value by explaining mutual exclusivity of source and paths, allowed root constraint for paths, and recursive directory walking, beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it statically analyzes Rego policies to return a JSON Schema of input.* fields. It uses specific verbs ('analyse', 'return') and resource ('input.* fields'), differentiating it from siblings like rego_eval or opa_query_decision.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions it's a 'correct starting point' for integration tests, schema validation, or documenting policy API, implying when to use. It does not explicitly state when not to use or name alternatives, but the context is reasonably clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Rich disclosure well beyond the annotations: the GITHUB_TOKEN environment requirement plus the GITHUB_TOKEN_MISSING failure mode, non-idempotency ('Each call creates a new Gist'), the secret-by-default privacy behavior, and return-field semantics (gistUrl vs rawPolicyUrl with concrete OPA/Conftest usage). All of this is consistent with readOnlyHint=false, openWorldHint=true, and idempotentHint=false — no contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
Roughly 150 words, with the purpose front-loaded and each subsequent sentence carrying distinct information: privacy default, return format, OPA/Conftest integration, metadata bundling, non-idempotency, and auth requirement. Slightly long, but the density is justified given six parameters, external side effects, and an auth dependency — no sentence is wasted.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex externally-visible write tool with no output schema, the description is complete: it states the return shape inline ({ gistUrl, rawPolicyUrl, id, public }), covers the auth prerequisite and its failure mode, discloses the side effect (new Gist per call), and explains the privacy default. An agent has everything it needs to invoke the tool and interpret the result.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3: every parameter (data, input, query, policy, public, description) already has a thorough schema description. The prose adds connective value by explaining that supplying query/input/data triggers metadata.json bundling for reproducibility, but it does not need to and does not meaningfully re-explain individual parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: "Share a Rego policy with teammates or create a reproducible example by publishing it as a GitHub Gist." It states exactly what the tool does and for what ends. It is unambiguously distinct from all 50+ siblings — no other tool publishes to an external service or deals with Gists.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit trigger scenarios: sharing with teammates and creating a reproducible example, including the 'full evaluation context' benefit of bundling query/input/data. It does not name alternatives or state when-not-to-use, but no sibling performs this function, so exclusions would be low-value; the context is clear enough for an agent to select it correctly.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations, the description discloses that the target directory is emptied before the pull, warns that pulled policies are arbitrary executable Rego with exfiltration/network risks, and notes the external binary dependency and path restrictions. This is substantial value added on top of destructiveHint and readOnlyHint.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the main action and each subsequent sentence covers a distinct aspect: use case, prerequisite, path constraint, and security warning. No filler or redundancy; the security warning earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a side-effectful tool with no output schema, the description covers purpose, prerequisites, destructive side effects, security implications, and path constraints. An agent has everything needed to decide whether to call it and to call it safely.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage, the schema already documents the url schemes and the policy directory fallback and emptying behavior. The tool description doesn't add new parameter-level meaning beyond restating that the policy directory must be inside allowed paths, so the baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific action: 'Download Rego policies from an OCI registry or Git repository into a local directory using conftest pull.' It identifies the resource, destination, and direction, and it distinguishes itself from the sibling conftest_push by direction and from conftest_test by sequencing ('hydrate ... before running conftest_test').
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives explicit usage context: 'Use this to hydrate a local policy/ directory before running conftest_test' and states prerequisites (conftest on PATH or CONFTEST_BINARY, OPA_MCP_ALLOWED_PATHS). It does not explicitly name an alternative to use instead, but the download-vs-push contrast and the sequencing with conftest_test provide clear context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond annotations, it discloses the external binary requirement and failure mode (CONFTEST_NOT_FOUND), and warns that policies can call OPA built-ins such as http.send, implying possible side effects. It also states the result granularity. This is substantial behavioral context; no contradiction with readOnlyHint=false/openWorldHint=true.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single dense paragraph with no filler: purpose, formats, output, prerequisite, parameter routing, default, and side-effect warning all appear in order. Every sentence earns its place and the most important scoping info is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For an 11-parameter, no-output-schema, no-required-parameter tool, it covers the essential decision points: how to supply inputs, what happens when policy is omitted, runtime prerequisite, and what results look like. The schema covers remaining parameter details, so nothing needed for correct invocation is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline applies; the description does not need to restate parameter details. It adds a useful high-level grouping (config vs policy, file vs inline) and the default-policy behavior, but those are already reflected in the schema. No additional semantic enrichment beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Evaluate configuration files ... against Rego policies using `conftest test`.' It enumerates supported input formats and clarifies the output is per-file, per-namespace pass/fail/warn results, which distinguishes it from sibling policy-editing and OPA-data tools. This is as strong a purpose statement as an agent could want.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives clear direction for selecting config and policy inputs: 'Provide config via `files` ... or `inlineConfig`', 'Provide policy via `policy` ... or `inlinePolicy`', and explains the default `./policy` directory. It does not explicitly name a sibling alternative or say when not to use the tool, but the context is unambiguous enough to be actionable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses non-obvious behavior well beyond the annotations: OPA drops the `credentials` block, returns `services.*.headers` verbatim, and redacts header values while keeping header names. This is exactly the kind of behavioral context that helps an agent anticipate the returned data and security implications.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the core purpose and followed by a high-value behavioral caveat. Every clause earns its place; there is no redundant or filler content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
This is a simple zero-parameter read operation. The description states what is returned, where it comes from, and the important redaction behavior. Annotations already convey read-only and idempotent safety. No critical information is missing for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters and the schema coverage is 100%, so there are no parameter semantics for the description to clarify. Per the rubric, a zero-parameter tool gets a baseline of 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Return') and a precise resource ('the running OPA server configuration from `GET /v1/config`'). This clearly identifies what the tool does and separates it from sibling tools like opa_status or opa_health, which concern server health rather than configuration.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The use case is implied: if an agent needs the running OPA server configuration, this is the tool. However, the description does not explicitly state when to prefer this over alternatives or mention any exclusions, so guidance is present only by inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare readOnlyHint, destructiveHint, and idempotentHint. The description adds that the tool runs 'opa check' and returns structured diagnostics with locations, providing behavioral context beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the verb and resource. Every sentence adds essential information without waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the return format (valid/errors with diagnostics) and explains the two input modes. No output schema is provided, but the description adequately describes the output.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already documents each parameter. The description adds value by explaining the mutual exclusivity of source/paths and the purpose of maxErrors, going beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description specifies the verb 'type-check' and the resource 'Rego', explicitly invoking 'opa check'. It distinguishes from siblings like rego_lint and rego_test by focusing on type-checking.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly states the two mutually exclusive usage modes: inline source or file/directory paths. It does not explicitly list when not to use this tool vs alternatives, but the context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes beyond annotations by detailing return values (formatted source and changed flag) and warning about a specific OPA bug that can corrupt escape sequences. This level of detail is valuable for agent decision-making.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences: first states core operation, second adds critical edge case. Extremely concise and well front-loaded with no extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite lacking an output schema, the description explains what is returned (formatted source + changed flag). Handles the single parameter fully and addresses version-specific behavior. Complete for its simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a single parameter 'source' having a clear description. The tool description adds no extra parameter context beyond the schema, so baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Format Rego source code using `opa fmt`', specifying the verb (format), resource (Rego source code), and method. Distinguishes from siblings like rego_check and rego_lint which serve different purposes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides clear context for when to use (formatting) and includes important caveats about OPA version and string interpolation bugs. Does not explicitly mention alternatives or when not to use, but given the distinct purpose, it's still effective.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate read-only, idempotent, non-destructive behavior, and the description goes well beyond that by disclosing the temp-file mechanism, the file-by-file verification order, digest-vs-parsed-value differences, failure reason enumerations, and the precise condition for returning `verified: true`. It also surfaces the `verificationKeyId` nuance.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but every sentence carries substantive behavioral or edge-case information for a complex tool. It is front-loaded with the core purpose and implementation, then proceeds into verification details and error conditions. Some schema repetition exists, such as the `scope` instruction, but overall it earns its length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given there is no output schema, the description thoroughly documents return behavior: `INVALID_BUNDLE` with an enumerated `details.reason`, `INVALID_INPUT` for unusable keys or algorithms, and the exclusive condition for `verified: true`. It also covers failure ordering, v0 compatibility, directory verification convention, and key-ID behavior, making the tool self-sufficient for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so a baseline of 3 applies, but the description adds meaningful semantics: `scope` is emphasized as needing to match the signing value exactly, `v0Compatible` is tied to post-signature failure behavior, and `verificationKeyId` is explained as rarely needing to be set with a single key. This enriches the schema without being redundant.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The opening sentence names a specific verb and resource: 'Verify the signature of a signed bundle directory or `.tar.gz` archive with the public key.' It also clarifies the implementation mechanism and the matching relationship to `opa_bundle_sign`, which distinguishes it from general Rego verification siblings like `rego_verify` and `conftest_verify`.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives strong usage context: it explains that OPA has no standalone verify command, how the verification is performed, what inputs are required, and how `scope` must exactly match the signing value. It does not explicitly name alternative tools or say when not to use this tool, but the guidance is clear enough to invoke correctly.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly/idempotent/non-destructive, and the description adds substantial context beyond them: mismatches surface as rego_type_error diagnostics with file/line locations, the exact return shape `{ valid: true, errors: [] }` / `{ valid: false, errors: [...] }` is provided (important since there is no output schema), and the quirk that schemaPath-as-directory only works via `schemas:` annotations is disclosed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The purpose is front-loaded in the first sentence, and each subsequent sentence covers a distinct aspect: diagnostic behavior, return shape, schema input modes, and source modes. It is dense at roughly 110 words and mildly overlaps with the input schema's parameter descriptions, but nothing is filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema and 5 optional params spanning two mutually exclusive pairs, the description covers everything needed to invoke correctly: diagnostic type, return shape, all three schema-input modes, and both Rego-source modes. An agent can select inputs and interpret results without additional lookups.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description adds value beyond the schema by tying inlineSchema to the `schema` field of rego_infer_input_schema's output and by explaining that schemaPath accepts either a file or a directory with directory reads only through `schemas:` annotations. It reinforces the source/paths split, while strict is left to the schema, which already documents it.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Validate that a Rego policy's input.* field references are consistent with a JSON Schema using opa check --schema.' This clearly differentiates it from siblings like rego_check (generic checking without a schema) and rego_lint, and explicitly positions it as a consumer of rego_infer_input_schema's output.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives clear context for both input-mode pairs: inlineSchema vs schemaPath, with the directory mode conditioned on the policy declaring `schemas:` annotations, and inline Rego vs paths for the policy source. It names rego_infer_input_schema as a workflow upstream, but it never explicitly says when to prefer rego_check or rego_lint instead, so no exclusions or alternatives are stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes far beyond the sparse annotations (readOnlyHint: false, openWorldHint: true). It discloses subtle OPA behaviors: `errored` tests are neither passes nor failures and any one fails the suite; `test_`-prefixed rules and `*_test.rego` files are auto-discovered; the `test_x[case]` parameterized form collapses to one reported test unless `parameterizedGroups`/`caseCounts` are inspected; and enabling `coverage`/`threshold` switches output mode, disabling per-test counts. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
At roughly 230 words for a 12-parameter tool with no output schema, the length is justified and the core purpose is front-loaded. The downside is a single dense paragraph with no bullet or section structure, making the parameter guidance harder to scan, though every sentence carries real information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Because there is no output schema, the description carries the full burden of explaining return behavior, and it delivers: aggregate counts, errored semantics, COVERAGE_BELOW_THRESHOLD, coverage/coveragePct fields, trace population, parameterizedGroups/caseCounts, and the no-match error hint. Combined with 100% schema coverage on the input side, an agent has everything needed to invoke and interpret this tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3; the description earns extra credit by adding cross-parameter and output-mode knowledge beyond the schema: `varValues` only matters with `verbose: true`, `threshold` implicitly enables coverage-report mode, and coverage mode disables per-test counts. It does not restate every parameter (count, explain, v1Compatible are left to the schema), which is acceptable given the schema's thoroughness.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
Opens with a precise verb+resource pair ('Run Rego unit tests') and names the underlying implementation (`opa test`), plus the return shape (aggregate pass/fail/skip/error counts and per-test records). This distinguishes it from sibling tools like rego_bench (benchmarking), rego_check (static analysis), and conftest_test (Conftest tests).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides rich contextual guidance for when to use its options: `bundle: true` for bundle-structured directories, `threshold` for coverage gating, `timeout` for tests exceeding OPA's 5s default, and `ignorePatterns` for generated/fixture files. However, it never explicitly routes to or excludes sibling alternatives (e.g., rego_test_multiroot for multiple roots, rego_bench for performance runs), so tool selection must be inferred from the purpose statement.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly/idempotent/non-destructive, so the bar for added context is lower. The description adds real behavior: the tool returns Rego source, omits the AST by default, explains the size tradeoff, and notes the includeAst alternative. No contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, all substantive, with the main purpose in the first clause. No filler or duplication of schema fields.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-required-parameter read tool, the description covers what the caller gets (Rego source), the optional behavior (includeAst), and the alternative for AST. Annotations cover safety and idempotency, and schema covers parameters, so nothing essential is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so parameters are fully documented; the description adds little beyond what the schema already provides. The mention that AST is omitted 'unless asked for' aligns with includeAst's schema description, so no additional compensation is needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
Description opens with a specific verb+resource: 'Fetch a single policy by ID from the running OPA server.' It clearly scopes to one policy, distinguishes from list/put/delete siblings, and differentiates from rego_parse_ast by stating this returns Rego source and AST is optional.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states that the AST is omitted unless asked and directs the agent to use `rego_parse_ast` when an AST is wanted, giving a concrete when-not. It also implies the primary use case—getting the Rego source for one policy—without ambiguity.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already signal destructiveness and non-readonly, but the description adds valuable behavioral detail: it modifies files, can be previewed with dryRun, directory-package-mismatch moves files, uncommitted changes require force, and the regal binary must be installed. This goes well beyond what annotations alone convey.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and front-loaded: the main action and rule list appear first, followed by the most important usage warnings and prerequisites. Every sentence conveys distinct operational information without repetition or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive file-modifying tool with nine parameters and no output schema, the description covers the critical operational context: prerequisites, preview capability, file-moving side effects, and force requirements. The full parameter detail is already handled by the 100% schema coverage, so nothing essential for correct invocation is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description adds meaningful parameter context by enumerating the exact rule names available to enable/disable, clarifying that directory-package-mismatch physically moves files, and explaining the force requirement in terms of uncommitted git changes. This exceeds the schema's minimal parameter notes.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the specific action ('Run regal fix to automatically apply mechanical fixes') and the exact resource scope by listing the five supported rules. This makes it immediately understandable and distinguishable from siblings like rego_lint or rego_suggest_fix, even without naming them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives practical context for when to use dryRun, force, and disable for directory-package-mismatch, and it explicitly frames the tool as applying mechanical fixes. It does not name sibling alternatives such as rego_suggest_fix, so exclusion guidance is missing, but the conditions provided are clear enough for an agent to select it properly.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description goes well beyond the readOnly/idempotent annotations by disclosing that existing test_* and todo_test_* rules are skipped, that computed value rules get todo_test_ stubs, that opa test reports these as skipped, and how tableStyle changes the generated loop. It also explains AST-driven input-shape inference and the inferredInputShape response field.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense but purposeful. Every sentence adds a distinct behavioral or usage detail, and the core purpose is front-loaded before implementation specifics.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description still covers the key return information by naming the inferredInputShape response field and describing what the generated stubs look like. The tool has only two simple parameters, and the description gives enough detail for an agent to call it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already documents both parameters at 100% coverage. The description adds practical value by explaining the benefit of tableStyle (adding multiple input/expected pairs without duplicating assertion code) and by reinforcing that source is policy source. This is a slight but meaningful improvement over the schema alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: "Generate a `*_test.rego` skeleton from a policy." It then explains the process (parse AST, find non-test rules, emit one stub per rule), which clearly distinguishes this tool from siblings like rego_test that run tests rather than generate them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The intended context is clear from the first sentence: use this when you need a test skeleton for a Rego policy. It does not explicitly name alternatives or state when not to use it, but the generation behavior is distinct enough that no exclusion is necessary.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description extensively discloses behaviors beyond the annotations: it returns concrete counterexamples, reports INCONCLUSIVE for specific unsupported constructs, treats absent fields as undefined, and explains the empty-input caveat for always_true. Annotations only declare read-only/idempotent, so this adds substantial value.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but information-dense; every clause contributes a necessary limitation or behavioral detail. It is front-loaded with the core verification promise before moving to edge cases, making it efficiently scannable for an agent.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having no output schema, the description covers all major outcomes: proof, counterexample, INCONCLUSIVE, unsupported language features, and undefined-field semantics. An agent has enough information to invoke the tool correctly and interpret likely results.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description adds semantic depth beyond the schema by clarifying what always_true means for absent fields and how a rule requiring input.x yields the counterexample {}, which enriches the enum definitions in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: formally verify a property about a Rego rule using SMT solving. It clearly distinguishes itself from testing by checking ALL possible inputs and from sibling eval/test tools by emphasizing proof or counterexample output.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives clear context for when to use this tool: when exhaustive verification is desired instead of testing, and it explains when results will be INCONCLUSIVE due to unsupported constructs. It does not explicitly name sibling alternatives, but the 'Unlike testing' contrast and limitation list provide practical usage guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations (destructiveHint=true, idempotentHint=false), the description discloses the 204 success response, the 404-to-DATA_NOT_FOUND mapping, and the intentional root-path exclusion. These behaviors directly inform an agent about success, failure, and edge cases, which is exactly the kind of context annotations alone cannot provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is information-dense but every sentence earns its place: the core action, path syntax rules, fallback parameter guidance, and edge-case behavior are all stated with no filler. The most important context is front-loaded and the formatting is easy to scan.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a two-parameter destructive operation with no output schema, the description is complete: it covers success codes, error mapping, path constraints, and the root-path edge case. The annotations already mark the destructive nature, and the description fills the remaining behavioral gaps an agent would need to safely invoke the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Although the schema already covers 100% of parameters, the description adds crucial semantics: dotted vs slash-only path parsing, how to address keys containing dots or slashes, and when to switch from `path` to `segments`. This resolves ambiguous inputs that the schema descriptions only partially convey.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Remove') with a clear resource ('document from OPA's data store') and the path-based scope. It naturally distinguishes itself from sibling tools like opa_delete_policy by explicitly targeting the data store rather than policy.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear operational context: it explains when to use `path` vs `segments`, notes the root-path deletion exclusion, and requires at least one segment. It does not explicitly compare itself to alternative data-management tools like opa_patch_data or opa_put_data, but the conditional guidance is strong and the tool's purpose is unmistakable among siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description goes well beyond the annotations by disclosing the in-place mutation ('signed in place', `.signatures.json` is written), the naming scheme for recorded files, archive refusal, key-type constraints, and return values. These details align with `destructiveHint=true` and `idempotentHint=true` without contradicting them, and they compensate for the lack of an output schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Although longer than typical tool descriptions, every sentence carries a distinct piece of information: action, side effects, archive exception, key details, claims, and return value. The most decision-relevant fact (archive refusal and `opa_bundle_build` route) is placed after the core mechanics, which is still well front-loaded and dense without padding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 4-parameter tool with no output schema, the description covers the operation, side effects, return payload, parameter specifics, and the important boundary case (archives). The only minor omission is behavior when `.signatures.json` already exists, but the `idempotentHint` annotation covers that, so nothing essential is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already documents all four parameters (100% coverage), so the baseline is 3. The description adds value on top by explaining that RSA/ECDSA keys are PEM files while HMAC algorithms expect a file holding the secret, and by spelling out what `claimsFile` should contain. This moves it to a 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb+resource pair ('Sign a bundle directory with `opa sign`') and immediately differentiates from siblings by explaining it signs directories, not archives, and that signed archives come from `opa_bundle_build`. It also names `opa_bundle_verify` as the counterpart for verification, so an agent can distinguish it among the bundle-related tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states that archives are refused and directs agents to `opa_bundle_build` with `signingKey` when a signed archive is needed. It also tells agents where the signed directory can be verified (`opa_bundle_verify`, `opa build`, `opa run --bundle`), making the tool's place in the workflow clear. No ambiguity about when to use this tool versus its bundle siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the readOnlyHint/idempotentHint annotations, the description discloses meaningful behavioral details: the response reflects startup-reported configuration, includes bundle/decision-log/plugin settings, and service header values are redacted because they may contain API keys. This adds genuine transparency beyond what structured annotations already convey.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each earning its place: the first states the action and endpoint, the second clarifies the difference from a sibling tool, and the third covers response contents and a security-relevant redaction. The description is front-loaded and compact with no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Although there is no output schema, the description compensates by enumerating response categories, clarifying the relationship to opa_config, and warning about redacted header values. For a zero-parameter read-only status tool, this is sufficient context for an agent to invoke it correctly and interpret its result.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are zero parameters, so schema coverage is complete by definition and the description need not explain parameters. It still adds useful context about what the returned configuration document contains and the redaction policy, which is more than the empty schema provides. Baseline 4 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource ('Return the running OPA server configuration via GET /v1/config') and precisely distinguishes this tool from its sibling opa_config by noting the 'status' key presentation and the intent to check 'what is running' vs 'what was configured'. This gives an agent a clear, unambiguous purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly names the alternative tool opa_config and states the selection criterion: use this when the agent wants 'what is running' rather than 'what was the server configured with'. This is direct routing guidance with no ambiguity about when to prefer this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations (readOnlyHint: false, openWorldHint: true), the description discloses key runtime behaviors: each file is evaluated independently, failures appear as error fields instead of result fields, CI gates report failed: true rather than erroring, and the timeout is bounded by the server subprocess. It also flags the decision-name conversion gotcha.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense but every sentence earns its place, from the core function to the batch differentiation, input-mode options, per-file error semantics, and CI-gate behavior. It is front-loaded with the most important purpose and alternative-routing information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 9 parameters, no output schema, and complex mutual exclusions, the description covers the key success criteria: when to use it, how policies and inputs are supplied, how failures are represented, how CI gating works, and the decision-name gotcha. The combination of description and schema gives an agent enough context to invoke it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Although schema coverage is 100%, the description adds substantial meaning: bundle and dataPaths are mutually exclusive, fail gates convert results into CI pass/fail signals, and decision accepts Rego reference forms and converts them because passing them straight through leaves every file undefined. These are critical operational details not evident from the schema alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Evaluate a policy decision against one or more input files using opa exec --format=json.' It also explicitly distinguishes itself from rego_eval, which handles a single input, so an agent can unambiguously identify when this tool applies.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
It states the ideal context ('ideal for CI pipelines that check many config files against a policy in one call'), names the alternative rego_eval, and explains the conditional selection between bundle and dataPaths. It also clarifies when to enable CI-gate flags (fail/failDefined/failNonEmpty), leaving little to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already mark the tool read-only, idempotent, and non-destructive. The description adds valuable behavioral context beyond that: default output shape, exact-match behavior for `builtins`, the `matched`/`missing` fields, response size constraints, and the `OUTPUT_TOO_LARGE` failure mode instead of truncation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is longer than average but every clause earns its place by explaining a distinct behavior or constraint. It front-loads the core purpose, then proceeds through the parameter modes in a logical order, ending with the size-cap caveat. No filler or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of the tool's parameter interactions and the absence of an output schema, the description is remarkably complete. It covers all parameter modes, default behavior, output contents, error behavior, and the prerequisite for `names_only: false`, leaving an agent with enough information to call the tool correctly in any scenario.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Although the schema already covers 100% of parameters, the description adds meaning beyond the schema: it explains the relationship between `names_only`, `builtins`, and the response cap, clarifies the default when `names_only` is omitted, and states that `builtins` requests exact matches with missing names reported separately. This materially helps an agent choose parameter values.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Return OPA capabilities -- the available builtins, future keywords, features, and WASM ABI versions.' This clearly distinguishes the tool from the many rego_* and opa_* siblings by describing exactly what capability metadata it exposes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit conditional usage for each parameter mode: `current: true`, `version: "v1.19.0"`, and neither, plus guidance on `names_only` and `builtins`. It also warns about response size limits and the `OUTPUT_TOO_LARGE` error, so an agent knows when to request only a few builtins versus raising the cap.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
It discloses far more than the annotations provide: the exact leaf-rule heuristic, that `sharedPaths` are added to every invocation yet excluded from discovery, that coverage switches output mode and drops test record counts, that `overallCoveragePct` is the mean across roots with coverage data, and that exceeding `maxRoots` returns INVALID_INPUT. No contradiction with readOnlyHint=false or openWorldHint=true.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense but every sentence earns its place for an 11-parameter, two-mode tool with a subtle leaf rule. It is front-loaded with the core verb and problem, then methodically explains modes, discovery semantics, and coverage aggregation without wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite no output schema, the description names the key result fields (`coverage`, `coveragePct`, `overallCoveragePct`, `thresholdMet`), the error condition (INVALID_INPUT on maxRoots), and the per-mode behavior trade-offs. For a tool this complex with only minimal annotations, nothing an agent needs to invoke it correctly is left unspecified.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, yet the description adds genuine semantic value beyond the schema by explaining parameter interactions: `threshold` implicitly enables coverage mode, `roots` and `scanDir` are mutually exclusive, `sharedPaths` are excluded from auto-discovery, and `maxRoots` enforces a hard cap. It also explains the behavioral consequences of `coverage` on output fields, which the schema alone does not convey.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb-resource pair — "Run `opa test` once per root and aggregate results" — that immediately establishes what the tool does. It further distinguishes itself from the sibling `rego_test` (single root) by naming the package-conflict problem (OPA issue #4724) it uniquely solves.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit mode-selection criteria: use `explicit` when roots are known upfront or scan cannot determine them, and it precisely defines the leaf rule that governs `scan` mode. It frames the motivating scenario (when `opa test .` fails on multi-namespace repos), and the schema enforces mutual exclusion between `roots` and `scanDir`, so an agent can select the correct invocation path without inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
GitHub Badge
Glama performs regular codebase and documentation scans to:
- Confirm that the MCP server is working as expected.
- Confirm that there are no obvious security issues.
- Evaluate tool definition quality.
Our badge communicates server capabilities, safety, and installation instructions.
Card Badge
Copy to your README.md:
Score Badge
Copy to your README.md: