Skip to main content
Glama

Evaluation result check

wever_evaluation-result-check
Read-only

Agent Evaluation: submit one evaluation result report: each claim with its text, metric and claimed figure, the subset it comes from and whether the claim names it, the baseline it compares with and whether it says higher, lower or meets a threshold; each measured result with its metric, value, subset, sample size, spread (a standard deviation, a standard error or a 95 percent interval) and run conditions (model, dataset and date); each baseline result; and the subsets the evaluation ran beside the subsets the report shows. The signed report matches every claim to its measured figure and says whether the figure matches, whether each comparison runs the claimed way and clears the noise in both figures, whether a subset figure says so and whether every subset that ran is reported, and names every claim with no result, every figure the results do not show, every comparison with no baseline result, the wrong way round or inside the noise, every threshold claimed and missed, every undisclosed subset and every dropped subset, then every figure behind a claim with no sample size, fewer than 30 tasks, no spread or no run conditions, with the fix for each. This reads one result report’s claims against its own numbers: whether a benchmark claim’s figures appear in the source it cites is wever_benchmark-evidence-check, whether a claim is supported by offered evidence is wever_claim-evidence-check and one paid call’s result against its order is wever_result-validation-check. Verdict claims_unsupported, result_gaps, result_unverified or claims_supported. POST /api/evaluation-result-check. Reads only the material in the call: no network request, fetch, money movement, message, sharing, configuration change or external write. Paid URL: 0.10 USDC per call on Base via x402 exact, with the shared 10 signed-wallet free calls per rolling 30 days. Bind X-Wever-Wallet-Proof to the exact JSON arguments and the tool route. Returns a report with an Ed25519 signed receipt. Contract: /api/agent-evaluation-contract. Human page: https://weverlabs.com/agent-evaluation/.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "claims": "<claims>"
    },
    "name": "wever_evaluation-result-check"
  }
}

Input Schema

NameRequiredDescriptionDefault
labelNoOptional name recorded in the report.
claimsYesEach claim.
resultsNoEach measured result.
subsetsNo
baselinesNoEach baseline result.

Schema Changelog

Changes observed during successful MCP inspections.

  1. Added

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and openWorldHint=false, but the description goes well beyond them: exact route (POST /api/evaluation-result-check), no network fetch, no money movement, no external write, pricing of 0.10 USDC on Base via x402 with 10 free signed-wallet calls per rolling 30 days, the X-Wever-Wallet-Proof binding requirement, and an Ed25519 signed receipt on return. Those are operational facts an agent cannot obtain from the annotation flags.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The routing sentence and the payment/receipt lines are front-loaded and useful, but the payload and output inventory is delivered as one very long run-on sentence that an agent must parse hook by hook. Almost every clause carries information, yet splitting the claim/result/report inventory into distinct sentences would preserve the content with far less cognitive load.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, yet the description enumerates the report contents in detail and names the four possible verdicts (claims_unsupported, result_gaps, result_unverified, claims_supported), so an agent knows what comes back. What remains unstated is error/failure behavior for a paid call (e.g. insufficient payment, invalid wallet proof) and any size or pagination limits, which matters for a nested-schema tool capped at 200 claims and 500 results.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 80%, so the schema already documents most fields and a 3 would be the floor. The description still adds semantic intent the schema does not: that spread may be 'a standard deviation, a standard error or a 95 percent interval', that run conditions mean model, dataset and date, that subsets cover 'the subsets the evaluation ran beside the subsets the report shows', and how each field feeds the comparison logic. It stops short of clarifying optionality or how missing fields are treated, so it is not a 5.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb and resource ('submit one evaluation result report') and states precisely what the tool does with it: 'matches every claim to its measured figure' and reports mismatches. It then explicitly separates itself from three siblings by handing each a different question (benchmark source figures, claim evidence, paid call vs order), so an agent can route without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives explicit boundaries: 'This reads one result report's claims against its own numbers: whether a benchmark claim's figures appear in the source it cites is wever_benchmark-evidence-check, whether a claim is supported by offered evidence is wever_claim-evidence-check and one paid call's result against its order is wever_result-validation-check.' That is a when-to-use-this vs when-to-use-that statement with named alternatives, not an inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources