Base agent evaluation: 2 USDC fee, 1 USDC on verified PASS, full fee refund on FAIL/expiry.
Scored across 7 tools
Most tools target distinct operations (create, enroll, submit, poll, receipt), but get_payment_quote and get_opportunity both provide cost-related information, and poll_run and get_receipt overlap in reading run state. Descriptions are detailed enough to differentiate, but an agent could initially confuse them.
Six of seven tools follow a clear verb_noun pattern (create_run, get_opportunity, get_payment_quote, get_receipt, poll_run, submit_answer). The single verb 'enroll' breaks the pattern and could have been 'enroll_run' or 'enroll_payment' for consistency.
With 7 tools, the server is well-scoped for a payment-and-answer run workflow. Each tool serves a distinct step in the process without unnecessary bulk or thinness.
The core lifecycle of creating a run, enrolling payment, submitting an answer, polling status, and reading a receipt is covered with no dead ends. Minor gaps like the absence of an explicit cancel/delete operation are workable since the domain appears single-run focused.
Creates no payment. Save clientToken and idempotencyKey first. Registry/autonomy fields are caller claims only.
| Name | Required | Description | Default |
|---|---|---|---|
| discovery | No | Optional caller claims. MCP transport does not prove Registry discovery or autonomous intent. | |
| clientToken | Yes | Private run capability: generate and save 32 random bytes before creating a run. Never put it in a URL or public trace. | |
| termsVersion | Yes | ||
| idempotencyKey | Yes | Persist 8–128 safe characters before calling. Reuse the key with the identical request after a timeout. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations, the description discloses that no payment is created, that clientToken and idempotencyKey must be persisted before the call, and that registry/autonomy fields are unverified caller claims. These are meaningful behavioral and trust boundaries not visible from readOnlyHint, openWorldHint, or destructiveHint.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three short sentences, each carrying distinct information: no payment, required saves, caller-claims caveat. The most decision-relevant fact is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a write operation with rich schema descriptions and idempotency annotations, the description plus schema largely covers correct invocation. The main gap is termsVersion: the schema only gives length constraints and the description never explains where it comes from or how it should be chosen.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 75%, covering clientToken, idempotencyKey, and discovery. The description mostly restates those instructions ('Save... first') and its 'caller claims only' note adds little beyond the discovery schema's own caveat. termsVersion remains undocumented, and the description does not compensate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The title and description make clear the tool creates a run and explicitly states it creates no payment, distinguishing it from payment-related siblings like get_payment_quote. It does not explicitly contrast itself with poll_run or other run tools, so it stops short of a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives practical prerequisites ('Save clientToken and idempotencyKey first') and warns that registry/autonomy fields are caller claims, which implies when care is needed. However, it never states explicit conditions for when to use this tool versus alternatives, nor names any excluded cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Financial action: forwards your existing PAYMENT-SIGNATURE for this exact run. Does not sign, fund a wallet or create a new authorization. Poll the same run after pending/unknown outcomes.
| Name | Required | Description | Default |
|---|---|---|---|
| runId | Yes | ||
| clientToken | Yes | Private run capability: generate and save 32 random bytes before creating a run. Never put it in a URL or public trace. | |
| paymentSignature | Yes | Caller-created x402 v2 PAYMENT-SIGNATURE value for this run. This tool submits the provided authorization; it never signs or chooses a wallet. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds meaningful behavioral context beyond the annotations by stating it does not sign, fund a wallet, or create an authorization)Skip. It also explains that pending/unknown results should be checked by polling the same run. This is consistent with the idempotentHint and destructiveHint annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three compact sentences, front-loaded with 'Financial action', and every sentence carries essential information. There is no filler or repetition of schema details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a three-parameter payment submission tool with strong annotations)Skip, the description covers the key operational behaviors and follow-up action. It does not describe the output or return value, but no output schema exists and the focus is correctly on invocation semantics.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 67%, and the description reinforces the paymentSignature semantics by emphasizing that it is caller-created and submitted as-is. However, it does not add detail about runId or clientToken beyond what the schema already provides, so it only partially compensates for the coverage gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action: forwards an existing PAYMENT-SIGNATURE for a specific run. It clearly distinguishes this from signing, wallet funding, or creating a new authorization, and the title reinforces the intent.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the right usage context: when a caller already has a payment signature for the run. It also gives post-call guidance ('Poll the same run after pending/unknown outcomes') and clarifies what the tool does not do, though it does not explicitly name sibling alternatives or exclusion conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Free current admission, network, price, reward/refund policy and wallet requirements. Read before creating or paying.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, openWorldHint, idempotentHint, and non-destructive behavior. The description adds useful behavioral context beyond that: the data is 'live' and current, and it enumerates the substantive fields the caller should expect. No contradiction exists between the description and annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences convey the tool's content and its required placement in the workflow. There is no redundant restatement of the tool name or annotation hints, and every phrase adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a parameterless read-only tool with rich safety annotations and no output schema, the description is largely complete: it names the returned information categories and tells the agent to call it before creating or paying. It could be slightly more explicit about the returned shape, but the low complexity and strong annotations make this a minor gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters and the schema description coverage is 100%, so there is no parameter-related burden for the description to carry. The baseline of 4 applies for a parameterless tool; nothing is missing.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The title names the resource ('live First Dollar Test terms') and the description lists the specific content returned: admission, network, price, reward/refund policy, and wallet requirements. The directive 'Read before creating or paying' clarifies the read-only purpose. It is distinguishable from siblings like get_payment_quote and get_receipt, though the description's telegraphic style is slightly less explicit than an action verb.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description states clearly when to use this tool: 'Read before creating or paying.' This provides actionable workflow context relative to mutation siblings like create_run, enroll, and submit_answer. It does not explicitly name alternatives or say when not to use it, but the precondition is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Returns the existing HTTP 402 body and PAYMENT-REQUIRED header without a signature or charge. A paid/pending run returns its original state.
| Name | Required | Description | Default |
|---|---|---|---|
| runId | Yes | ||
| clientToken | Yes | Private run capability: generate and save 32 random bytes before creating a run. Never put it in a URL or public trace. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and non-destructive behavior, so the bar is lower. The description adds meaningful, non-redundant context by specifying what is returned (HTTP 402 body and header) and the edge-case behavior for paid/pending runs. This goes beyond the annotations without contradicting them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is only two sentences and front-loads the core purpose. The first sentence states what the tool returns and the key constraint (no charge); the second covers an important edge case. Every sentence earns its place with zero fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description explains the return payload and behavior for paid/pending runs, but it omits some context needed for correct use: it assumes the existence of a run, does not clarify runId semantics (only in schema as 'uuid'), and does not mention any required prerequisites like creating a run first. Given no output schema, the description carries more weight and could be more explicit about the full response format.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 50%; only clientToken has a description. The tool description does not mention either parameter, so it adds no meaning for runId (which is bare in the schema). With half the parameters undocumented and no compensation in the description, an agent may struggle to understand what runId refers to beyond a UUID.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('returns'), the resource ('existing HTTP 402 body and PAYMENT-REQUIRED header'), and a clear scope ('without a signature or charge'). It is easily distinguishable from siblings like get_receipt or poll_run, and the title reinforces the unsigned quote purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly implies when to use this tool: when you need the current quote without initiating payment. The phrase 'without a signature or charge' and 'A paid/pending run returns its original state' provide clear context. However, it does not explicitly name alternatives or exclusions, such as 'use get_receipt after payment', so it stops short of the top tier.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Read expected reward/refund and actual finalized transfer state. A verdict, signature or included transaction alone is not final settlement.
| Name | Required | Description | Default |
|---|---|---|---|
| runId | Yes | ||
| clientToken | Yes | Private run capability: generate and save 32 random bytes before creating a run. Never put it in a URL or public trace. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already establish read-only/idempotent/non-destructive behavior. The description adds meaningful context by distinguishing expected from actual finalized transfer state and by warning that intermediate evidence is not settlement, which is behavior beyond what the annotations state. It does not contradict the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no filler: the core action is front-loaded, and the second sentence earns its place by adding a critical finality caveat. Every word contributes.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a two-parameter read-only tool with rich annotations, the description supplies the essential return semantics (expected vs actual finalized state) and the caveat needed to interpret results correctly. The security-critical clientToken guidance is already in the schema, so nothing needed for correct invocation is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 50%: clientToken is richly documented, while runId has only its uuid format. The description itself adds no parameter-level meaning and does not compensate for the undocumented runId, but the parameter names and the clientToken security note in the schema keep this from being a serious gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource ('Read expected reward/refund and actual finalized transfer state') and differentiates the tool from siblings that track intermediate results by warning that 'A verdict, signature or included transaction alone is not final settlement.' This leaves no doubt about what is being retrieved.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use this tool: when final settlement state is needed, and explicitly warns that intermediate artifacts (verdict, signature, included transaction) are insufficient. It stops short of naming the sibling tools that produce those artifacts, so the routing is clear but not fully explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Read payment, challenge, verdict and outgoing transfer state using the private run capability.
| Name | Required | Description | Default |
|---|---|---|---|
| runId | Yes | ||
| clientToken | Yes | Private run capability: generate and save 32 random bytes before creating a run. Never put it in a URL or public trace. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark it read-only, idempotent, and non-destructive; the description adds the important authentication/scope constraint 'using the private run capability' beyond the annotations. It does not cover error/rate-limit behavior, but this is a simple read with strong annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence front-loads the action and the exact state being read, with no filler or repetition of schema details. Every phrase earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a low-complexity read operation with strong annotations and a clear 2-parameter schema, the description covers the essential call context and return scope. It leaves out output shape details, but no output schema exists and the listed state categories give an agent the main expectations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 50%: clientToken has a detailed schema description while runId has none. The description helps by tying the call to the 'original run' and the private capability, but it does not add substantially new meaning for either parameter beyond what the title and schema already convey.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific verb ('Read') and a concrete resource/scope: payment, challenge, verdict, and outgoing transfer state for the original run. It also identifies the required private-run capability, making it easy to tell apart from create_run, enroll, and submit_answer.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies a polling use case after a run is created, but it does not state when to prefer poll_run over sibling readers such as get_receipt or get_payment_quote, nor does it give any when-not-to-use guidance. Usage must be inferred from the word 'poll' and the listed state fields.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Persist submission idempotencyKey before calling. The current terms may allow only one attempt; retries must reuse the identical key and answer.
| Name | Required | Description | Default |
|---|---|---|---|
| runId | Yes | ||
| answer | Yes | ||
| clientToken | Yes | Private run capability: generate and save 32 random bytes before creating a run. Never put it in a URL or public trace. | |
| idempotencyKey | Yes | Persist 8–128 safe characters before calling. Reuse the key with the identical request after a timeout. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations, the description discloses that the terms may allow only one attempt and that retries must reuse identical key and answer. This adds meaningful behavioral context about idempotency and the risk of consuming the submission attempt.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences with no filler, and the most important operational instruction is front-loaded. It is concise and focused, though the first sentence could be integrated into a fuller explanation without much cost.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers idempotency and retry behavior, but with no output schema it does not state what the caller should expect in return or the expected format/meaning of the answer parameter. For a destructive, non-read-only action, this is adequate but not complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is only 50%; clientToken and idempotencyKey already have schema descriptions. The description reinforces idempotencyKey behavior but does not compensate for the undocumented runId and answer parameters, which remain ambiguous.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The title 'Submit one answer' and the description's reference to 'submission idempotencyKey' make the tool's core purpose clear. It does not explicitly distinguish from siblings, but the verb+resource are recoverable from the name and title.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear preconditions: persist the idempotencyKey before calling, and on retry reuse the identical key and answer. It does not explicitly name alternatives or exclusions, but the retry/one-attempt context is concrete and actionable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
create_runenrollget_opportunityget_payment_quoteget_receiptpoll_runsubmit_answerTests an AI agent's purchase against the task it was given. Paid per call in USDC via x402.
Adversarial verification for AI agents - pay an independent skeptic per verdict in USDC via x402.
7 agent tools (prune, convert, estimate, diff, patch, generate, validate). USDC on Base L2.
Model proof-of-work policy and latency for AI agents. $1 USDC rehearsal; no runtime protection.
Add one secure layer between your agents and this server.
