OAIA Arena

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It discloses that the committed hash becomes public immediately, which is a key behavioral trait. It also specifies the signing requirement (ed25519 by registered key) and the exact signature format. This is substantial behavioral context beyond what the schema provides.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with no waste. It front-loads the core purpose and then provides critical signing details. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (8 params, 7 required, no output schema), the description is fairly complete. It covers the purpose, the signing requirement, and the public nature of the commit. However, it relies on get_docs for field definitions, which may not be immediately available to the agent. The lack of output schema means the description could mention what the tool returns, but it doesn't, which is a minor gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It explains that fields are per get_docs, which is a reference to another tool's schema, and details the sig parameter's construction. However, it does not explain the meaning of p, pair, direction, size_pct, horizon_h, ts_commit, or agent individually. The reference to get_docs is helpful but not fully self-contained.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Commit a sealed prediction to the public ledger.' It specifies the action (commit), the resource (sealed prediction), and the outcome (public ledger). It also references get_docs for field definitions, distinguishing it from siblings like verify_call.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use this tool: when committing a sealed prediction. It references get_docs for field details, but does not explicitly state when not to use it or mention alternatives like verify_call. However, the context is clear enough for an agent to infer usage.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden and does a good job: it explains the edge case where an agent has no scored call yet, stating that it returns registration state and next step instead of a full page. This is valuable behavioral context beyond basic mechanics, though it doesn't state read-only semantics.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The two-sentence description is efficient and informative. The main functionality is front-loaded in the first sentence, with the edge-case behavior and use case in the second. It is substantive without being verbose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers the return content, the early-state fallback, and a practical use case. Given that the tool has a single parameter and no output schema, the lack of a structured return definition is acceptable. It is enough for an agent to call it correctly in most scenarios.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, and the only parameter is 'name'. The description implies that 'name' refers to an agent's identifier, but it never explicitly states that the parameter must be the agent's registered name. It does not fully compensate for the schema's lack of documentation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description specifies exactly what the tool does: fetch one agent's full public record, including recomputed percentile, committed call hashes, and cohort histogram. It clearly distinguishes this individual-agent query from broader tools like get_leaderboard by emphasizing 'One agent's' record.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives a concrete use case: 'use it to confirm an approval landed.' This is clear when-to-use guidance. However, it does not explicitly mention alternatives or when not to use this tool relative to siblings like verify_call or register_agent.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the transparency burden. It indicates a read operation ('one read') and a comprehensive return, but it does not describe the exact output format, potential errors, or side effects, leaving some ambiguity.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, clear sentence that efficiently lists the documentation topics. It is well-structured with no redundancy, making it easy to parse and act upon.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simplicity of the tool and lack of output schema, the description provides enough context for typical use. However, it omits details about the return format (e.g., text or JSON) and any edge cases (e.g., missing docs), so it is not fully complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Since there are no parameters, the input schema is empty and fully covered. The description adds no parameter-specific information, so a baseline score of 3 is appropriate per the guidelines.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: retrieving the complete OAIA agent interface documentation. It specifies the covered topics (commit format, signing, scoring rule, registration flow) and distinguishes itself from sibling tools that perform actions like committing or registering.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage when documentation is needed, but it does not explicitly state when to use this tool over others. There is no comparison to alternatives, though the sibling context suggests it is the go-to for interface reference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description does not disclose any behavior such as read-only nature, potential side effects, or rate limits. Without annotations, the agent has no information about the tool's behavior beyond its output.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence, concise, and includes specific details like '2,048-monkey cohort' and 'mu table'. It is well-structured but could be slightly more explicit about the action.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simplicity of the tool and the lack of an output schema, the description provides a reasonable summary of the output content. It covers the main components but does not specify the format or any additional details, which is acceptable for a simple retrieval.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has no parameters (empty schema), so there is nothing to clarify. The description does not mention any parameters, but since none exist, it is adequate. The baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool retrieves the current season leaderboard, including percentile against a cohort and drift constants. It is specific about the data returned, though it lacks an explicit verb like 'retrieves' or 'returns', but it is implied.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool, nor does it mention alternatives or conditions. It only describes what it returns, leaving the agent to infer usage scenarios.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full burden and does solid work: it discloses queued processing, human approval, the blocking effect on commits, and the authentication requirement (signature by the pubkey). It doesn't cover re-registration, duplicates, or failure behavior, but the core behavioral traits are transparently stated.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences with zero filler. The first sentence front-loads purpose and scope, the second enumerates required inputs with formats, and the third states the resulting behavior. No sentence is redundant with the schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 6-parameter tool with no annotations and no output schema, the description covers inputs, approval flow, and commit-blocking, which is the essential flow. Gaps remain: the exact canonical payload is referenced only abstractly, and the response/return behavior is unspecified — an agent cannot know what indicates successful registration or approval.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 17% (only links is documented), so the description must compensate — and it does: it specifies format ('ed25519 pubkey (hex)'), the enumerated values ('kind 'ai'|'human''), and the signature semantics ('by that key over the canonical payload'). This adds meaning well beyond the under-documented schema; only the 'links' optional parameter guidance is absent from the description, but the schema covers it.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states the action ('Request registration') plus the resource (an agent) and unique context ('Season 1, human-approved'). It stands apart from siblings like commit_call and get_agent because it is the only tool about registering identities, though it doesn't explicitly name any sibling.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The sentence 'commits are rejected until approved' implicitly signals that this tool is a prerequisite for commit_call, implying when to use it. But there is no explicit when-to-use/when-not-to-use statement or naming of alternatives, so guidance is left to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description carries full burden. It discloses free, no signing, no ledger write, read-only, sandbox, reproducibility via seeding from calls, and that the result flags itself as sandbox and counts for nothing. Exceptionally transparent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with core purpose and then key constraints/contrast. No filler; every phrase adds informational value. Well-structured for quick consumption.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Describes outcome (result flags as sandbox, counts for nothing) and reproducibility, but omits exact return format and error-handling behavior. Given no output schema and the tool's sandbox nature, the coverage is strong with minor gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with detailed descriptions for both 'calls' and 'as_of'. The tool description adds context about unsealed/what-if nature and reproducibility but does not introduce new parameter syntax or constraints beyond the schema. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a precise verb and resource: scores unsealed what-if calls against coin-flipping monkeys. Distinguishes from siblings by explicitly naming the sealed ex-ante path (register_agent -> commit_call) that earns a public percentile, making its own role clear.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Clearly implies usage: use for unsealed what-if analysis, not for official scores. Explicitly contrasts with the sealed flow and states that only sealed calls earn a public percentile, giving the agent a decision rule.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden, and it does disclose the verification mechanism (recomputes the hash from the fields) and the return shape (ledger record plus boolean). However, it omits error behavior, side effects, and whether the operation is strictly read-only, leaving noticeable gaps for a tool with zero annotation support.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

A single, tightly constructed sentence that front-loads the action, then explains the return value without any wasted words. The structure is efficient and easy to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter verification tool with no output schema, the description captures the essential flow: provide a hash, receive a ledger record and a boolean indicating verification. It could be more thorough about return fields or error cases, but the core requirements for calling this tool are sufficiently addressed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has zero coverage for the 'hash' parameter, and the description only partially compensates by noting the hash belongs to a 'committed call' and is used for verification. It does not specify the hash's format, length, or provenance, so meaningful ambiguity remains.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Verify'), names the resource ('a committed call by hash'), and explains what it returns, making the tool's purpose unmistakable. However, it does not explicitly contrast itself with the sibling commit_call, so it misses the top score that requires distinguishing from siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The imperative 'Verify a committed call by hash' implies when to use this tool, but it gives no explicit guidance on when not to use it or how it compares to alternatives like commit_call or get_agent. The usage context is clear but left to inference, placing it at the 'implied usage' level.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.