Skip to main content
Glama

Refutation Pass

refutation-pass

Refutation Pass — Hand it one claim about code — "this is safe", "this fails closed" — and it tries to REFUTE the claim with a concrete reproduction instead of reviewing it. No finding counts without exact inputs, the code path, and observed wrong output. Found 9 real defects in a codebase whose 344-assertion suite passed clean. Full input contract: this tool's input_schema. (8 MESH/call, a tool · audit)

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
inputYesCapability-specific payload, e.g. agent-brain: {think:'...'}; agent-memory: {action:'store'|'recall', content|query}

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds meaningful behavioral detail beyond the annotations: it is adversarial rather than neutral, and no finding counts without exact inputs, the code path, and observed wrong output. It also communicates a performance expectation. This does not contradict the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core behavior is front-loaded in the first sentence, followed by the evidence bar, a credibility data point, and the input contract pointer. The 'Found 9 real defects' line is slightly promotional, and the parenthetical is compact. Overall it is well organized and not bloated.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers purpose, evidence standards, and cost, and implicitly defines findings as having exact inputs, code path, and observed wrong output. However, the input schema does not provide a refutation-pass-specific payload example, and there is no output schema or explicit return-format description, leaving some ambiguity for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has 100% description coverage but the description is generic and gives examples for other capabilities, not for this tool. The tool description clarifies that the input should contain a claim about code, but it does not specify a concrete payload shape. This meets the baseline for covered parameters but adds only modest semantic value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action: take one claim about code and attempt to refute it with a concrete reproduction, explicitly contrasting with 'reviewing' it. This clearly differentiates the tool from generic audit or review siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives a clear usage context: use when you have a falsifiable claim about code and want adversarial refutation, with examples like 'this is safe' or 'this fails closed'. It does not explicitly name alternatives or exclusions, but the use case is specific enough to guide selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

TDQS

B3.4/5.0
Disambiguation2/5

Several tools have overlapping purposes: search and mesh_discover both enumerate the catalog, while biz-analyze, task-analysis, and task-orchestrate all produce structured plans from a described situation. This will cause agents to misselect between them despite otherwise distinct tools.

Naming Consistency2/5

Naming is inconsistent: mesh_* tools use snake_case, most capability tools use hyphenated lowercase names, and a few (fetch, search) are bare verbs. There is no single verb-object or noun-verb pattern that holds across the set.

Tool Count3/5

28 tools is on the heavy side, but the marketplace concept justifies including many callable capabilities. However, the mix of platform tools and unrelated utilities makes the surface feel cluttered and hard to navigate.

Completeness4/5

The core marketplace lifecycle is well covered: signup, discover, fetch, publish, delegate, refer, follow, subscribe, and balance. Minor gaps exist (no unpublish or edit for listings), but most agent workflows can proceed without dead ends.