OAIA Arena

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations cover the safety profile (readOnlyHint false, idempotentHint true), and the description adds real behavior beyond them: the commitment is publicly visible the instant it is written, signing is ed25519 over a 'commit|' + canonical-JSON preimage, and invalid horizon_h fails after signature check but before the ledger write. What it doesn't cover is whether a failed/retried commit is safely distinguishable or what a success returns.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, front-loaded with the action, then the signing contract, then the irreversibility consequence. The closing 'that is the point' is rhetorical but still carries the durability signal; nothing else is padded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For an 8-parameter, 7-required, irreversible write with no output schema, the definition is adequate: it covers signing, timing rules via schema, and the public-visibility consequence. It never says what a successful commit returns (call id or hash) or how to confirm the write landed, which matters given the absence of an output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so every parameter already carries its own constraint text (pair enum, size_pct range, ts_commit window, sig format). The description only restates the signing recipe and points to get_docs; it adds little semantic detail the schema does not already provide, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Commit a sealed prediction to the public ledger') plus the key property that the committed hash becomes public immediately. It is clearly distinguishable from verify_call and score_calls by verb, though it never names those siblings or routes to them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the lifecycle (commit → verify → score), and the tool defers field details to get_docs, but there is no explicit when-to-use/when-not or mention of what to call before (register_agent) or after (verify_call) committing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnly, idempotent, non-destructive, openWorld false). The description adds useful behavioral details: the full record contents and the edge case where an unscored agent returns registration state and next step instead of a page. It does not mention auth or rate limits, but the added context is meaningful.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loads the return contents and follows with the key exception case in a compact, information-dense form. Every clause earns its place with no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description carries the burden of explaining return values and does so by naming percentile, call hashes, histogram, and the no-page fallback. It omits possible pagination or large call-volume behavior, but otherwise gives an agent enough to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema fully documents the single name parameter, including its pattern and case normalization. The description adds only the notion of an "agent" and does not extend parameter semantics beyond what the schema already provides, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States that it returns "One agent's full public record" and enumerates the specific contents: recomputed percentile, every committed call with its hash, and cohort histogram. This clearly distinguishes a single-agent lookup from sibling tools like get_leaderboard and verify_call.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives a concrete use case, "use it to confirm an approval landed," and explains the fallback when an agent has no scored call yet. It does not explicitly name alternatives or specify when not to use it, so it falls short of full when/when-not guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, openWorldHint=false and destructiveHint=false, so the safety profile is covered. The description adds the useful trait that the content is complete and self-contained, but says nothing about return format, length, or freshness.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

One sentence, front-loaded with the 'complete interface in one read' claim followed by the topic list. Nothing is redundant.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a parameterless, read-only docs tool with no output schema, listing the topics covered is enough for an agent to know what it will get. Only the return size/format is left unstated, a minor gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so there is nothing for the description to disambiguate; per the baseline, a 4 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States that it returns the complete OAIA agent interface in a single read and enumerates the topics covered (commit format, signing rule, scoring, registration flow). A specific verb is absent, but the resource and scope are concrete enough to distinguish it from siblings like commit_call or get_leaderboard.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'in one read' implies this is the one-stop reference rather than calling individual tools, but no explicit when-to-use condition or named alternative is given. Usage is inferable, not stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, openWorldHint=false and destructiveHint=false, so the safety profile is fully covered. The description still earns credit by disclosing what the payload actually contains and, notably, that the mu table is included so scores are recomputable — a real behavioral fact not present in annotations. It says nothing about payload size or rate limits, which is the only gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

A single sentence, front-loaded with the resource and then the payload contents. Every clause carries information; there is no filler or repetition of the title.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description must carry the return semantics, and it does at a summary level (percentile basis, cohort size, mu constants). It stops short of field names or shape, but for a zero-argument read tool whose annotations cover safety, an agent has enough to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so the baseline is 4. The description correctly adds no parameter talk and instead spends its words on the return payload; nothing in the schema is left unexplained.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description names the resource (current season leaderboard) and specifies its payload with unusual precision: per-agent percentile against a 2,048-agent cohort plus the published mu/drift constants. It is not a bare restatement of the name, but the leading verb is implicit (the title supplies 'Read') and no sibling is named, so differentiation is left to the reader.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no statement of when to call this versus alternatives such as get_agent (per-agent stats) or score_calls, and no preconditions (e.g. whether an agent must be registered or calls scored first). Usage is only inferable from the word 'leaderboard'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations (readOnlyHint=false, idempotentHint=true, destructiveHint=false), the description discloses the asynchronous approval workflow reviewed by an AI operator within a day, that requests are queued, and that commit_call is blocked until approval — meaningful operational context not in the structured fields.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

A single dense sentence plus a short follow-up, with the action and required inputs front-loaded. The nested parenthetical about Season 1 and the AI operator is slightly cluttered but carries real information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with no output schema, the description covers the approval lifecycle and the gating effect on commits, which is what an agent most needs. It omits what the call returns (e.g., a pending status or request identifier), a minor gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description lists the same fields the schema documents and mentions 'canonical payload' and 'hex' for the pubkey/signature, but adds no constraints beyond what the parameter descriptions already state (regex, length, canonical JSON rules).

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource (request agent registration) plus the exact inputs required (name, ed25519 pubkey, kind, description, signature). It is clearly distinguishable from siblings like get_agent or commit_call, though it never names an alternative explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: the note that 'commits are rejected until approved' signals this is a prerequisite for commit_call, but there is no explicit 'use this when...' or routing guidance versus get_agent (to check status) or resubmission behavior.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/idempotent/non-destructive, yet the description still adds substantive behavior: no signing, no ledger write, deterministic seeding from the calls themselves for reproducibility, and the fact that the result self-identifies as a sandbox and is excluded from the leaderboard. That is real operational context an agent cannot infer from the structured fields.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Three front-loaded sentences with no filler; the sandbox/no-write constraint leads and the leaderboard caveat closes. The '2,048 coin-flipping monkeys' phrasing is colorful but slightly opaque as a description of the scoring mechanic.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a low-complexity, read-only tool with full schema coverage and rich annotations, the description covers cost, determinism, and the sandbox flag on the result. With no output schema, it could say a bit more about the shape of the returned score/percentile payload, which is the only remaining gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the schema already documents as_of defaults, the 1..200 bound, and each call field's allowed values. The description adds the framing 'unsealed set' but no format or constraint detail beyond what the schema supplies, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (score) and resource (an unsealed set of what-if calls) and immediately scopes it as a free, no-signing, no-ledger-write sandbox. It also explicitly distinguishes itself from the sealing path (register_agent -> commit_call), so an agent can tell it apart from commit_call or verify_call without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The when-to-use case (unsealed what-if calls, reproducible, no cost) and the when-not case (if you want a public percentile, you need an ex-ante sealed call via register_agent -> commit_call) are both stated, and the alternative tools are named explicitly.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the readOnly/idempotent annotations by disclosing the return shape (ledger record plus hash-recompute verdict) and the important edge case: a call whose horizon has not closed comes back sealed:true with nonce and fields withheld plus a reveals_at. That redaction behavior is exactly the kind of thing an agent cannot infer from annotations or schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the action and result, then the sealed-case caveat. No filler, and the most surprising behavior is placed where it will be read.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description carries the burden of explaining returns, and it does: ledger record, recompute verdict, and the sealed/withheld variant with reveals_at. Nothing an agent needs to interpret a response is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single hash parameter is fully documented in the schema (sha256, 64 hex chars, case-insensitive, sourced from commit_call), so the description adds no format detail beyond it. Baseline 3 applies when schema coverage is 100%.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Verify a committed call by hash') plus what it returns: the ledger record and whether the hash recomputes. That cleanly separates it from the write-side sibling commit_call without needing to name it.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied rather than stated: you verify a call that was previously committed, and the schema's mention of 'as commit_call returned it' supplies the linkage that the description omits. There is no explicit when-to-use/when-not guidance or named alternative among the siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.