Agentic RL: Credit Assignment and CLI Agents

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of disclosing behavior. 'Inspect' strongly implies a read-only, non-destructive operation, but the description does not explicitly state that there are no side effects, nor does it mention authentication or rate limits. It is clearer than a generic 'update' but still lacks explicit behavioral guarantees.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, focused sentence that front-loads the action and resource, then enumerates the specific data categories. No words are wasted, and the enumeration is necessary to convey the scope of the overview.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Since there is no output schema, the description compensates by listing the categories of information the overview will provide. It does not specify the exact structure or formatting of the returned data, but for a read-only overview tool, the listed categories are sufficient for an agent to understand what to expect.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters and the schema is an empty object, so there is no parameter information for the description to add. The baseline for no parameters is high, and the description correctly does not introduce any irrelevant parameter details.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses the specific verb 'Inspect' and identifies the exact resources (ShellOps and ShellOps-Pro) and the data elements covered (task counts, splits, types, schemas, source files, license, citation). It clearly distinguishes the tool as a dataset overview rather than a search, fetch, or listing operation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description does not explicitly state when to use this tool versus the sibling tools, nor does it mention any prerequisites or exclusions. It only describes what the tool does, leaving the agent to infer that it should be used for an overview rather than for searching or fetching specific items.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of disclosing behavior. It states what the fetch returns (section anchor, version, equations, table cells, links, attribution), implying a read-only retrieval. It does not mention error handling or what happens for invalid IDs, but for a simple fetch operation this is adequate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence that states the purpose immediately and includes all necessary details without redundancy. Every clause adds information (fetch, ID source, content components), making it highly efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple fetch with one parameter and no output schema, the description provides sufficient context: it explains what to pass and what to expect in return. It does not cover error scenarios or relationship to other tools beyond search_evidence, but these are not critical for a basic get-by-ID operation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema provides an empty description for evidence_id, and schema description coverage is 0%. The description compensates by specifying that the evidence_id is 'returned from search_evidence,' giving the parameter origin and expected semantic meaning. This is essential for correct invocation but does not include format constraints or examples.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a specific verb ('Fetch') and a specific resource ('complete original evidence block'), and differentiates from search_evidence by specifying that it retrieves the full block using an ID. It also enumerates the returned content (section anchor, version, equations, etc.), adding concreteness beyond a generic fetch.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states that the evidence_id is 'returned from search_evidence,' which gives clear guidance on the correct usage flow (search first, then fetch). However, it does not explicitly mention when not to use this tool or contrast with other sibling tools like dataset_overview or list_sources, so it lacks explicit exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It clearly states the return content (instruction, reward spec, reference answer, metadata, parquet rows, asset links) and explicitly notes that file content is only available at source links and that no execution or verification occurs. This is transparent about what the tool does and does not do, though it could mention potential error cases or permission requirements.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with no waste, front-loading the primary purpose and then detailing the return value and exclusions. Every sentence earns its place, and the structure is efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only inspect tool with no output schema and no annotations, the description is quite complete: it lists all returned data categories, states file content is at links, and clarifies non-actions. Minor gaps like pagination or error handling exist but are not critical for this tool's straightforward purpose.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has zero description coverage for both parameters, so the description fully compensates by defining task_id as 'exact task_id' and partition as restricted to 'shellops' or 'shellops_pro'. This adds essential meaning beyond the empty schema fields, making parameter usage clear.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Inspect') with a clear resource ('published ShellOps or ShellOps-Pro task') and identifies the exact identifiers required (task_id and partition). It distinguishes itself from sibling tools like search_tasks by emphasizing the need for an exact ID, making its purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage context: 'by its exact task_id and partition' signals that this tool is for retrieving a known task, not for searching. It also clarifies what it does not do ('No shell execution or solution verification'), helping the agent decide when to use it. However, it does not explicitly name alternative tools, though the context is clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must carry the burden of behavioral disclosure. The verb 'List' implies a read-only operation, which is reasonable, but the description does not explicitly state that it is non-mutating or disclose any other behavioral traits (e.g., pagination, rate limits, or side effects). It adds minimal value beyond the obvious read nature.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, concise sentence that front-loads the action ('List') and then specifies the exact items to be listed. It contains no filler or redundant wording, making it highly efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has no parameters and no output schema, the description should explain what the tool returns. It states that it lists declared papers, original versions, licensing exclusions, and retrieval scope, which gives the agent a clear idea of the content. However, the term 'retrieval scope' is somewhat ambiguous and could benefit from clarification, but overall it is adequately complete for a simple list tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, and the schema is an empty object (100% coverage). The description does not need to add parameter meaning, and the baseline for 0 parameters is 4. The description focuses on the output content rather than parameters, which is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('List') and identifies a precise resource: the 13 declared papers, indexed original versions, licensing exclusions, and retrieval scope. This clearly distinguishes it from sibling tools like Agentic_RL_get_task or Agentic_RL_search_evidence, which serve different purposes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It does not mention any context, prerequisites, or conditions under which an agent should choose this over siblings like Agentic_RL_dataset_overview or Agentic_RL_fetch_evidence. The agent is left to infer usage from the name and description alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It discloses that the tool returns passages, citations, equations, and table cells, which implies a retrieval operation. However, it does not explicitly state side effects (e.g., read-only), limitations, or error behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise and well-structured within two sentences. It directly states the purpose, gives query syntax, and lists return content without unnecessary filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers the core function, query syntax, and return content, but lacks details about output format, result structure, or edge cases. Given the absence of an output schema and parameter descriptions, more context would be helpful for an agent to fully understand expected results.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Both parameters have empty descriptions in the schema, so the description must compensate. It indirectly explains 'query' by describing keyword syntax, but it does not explicitly define the parameter or the 'limit' parameter beyond its default value. This leaves significant ambiguity.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Search original papers') and the subject domain ('agentic reinforcement learning, credit reinforcement learning and CLI agents'). It does not explicitly distinguish itself from the sibling tool 'search_tasks', which could cause ambiguity, but the focus on 'original papers' provides some differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit usage instructions for query syntax ('Use English keywords (AND), OR and quoted phrases') and describes the expected return content. It does not clarify when to use this tool versus alternatives like 'search_tasks', but the operational guidance is clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does so thoroughly: it discloses case-insensitive literal substring matching, the fields searched, empty-query behavior, ordering by partition then task ID, explicit pagination, no relevance scoring, and the nuance that the train subset is not double-counted. These are non-obvious behavioral traits that help the agent anticipate results.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is tight and front-loaded: the first sentence states the core purpose, followed by clarifying behaviors. Each sentence adds unique value—empty-query behavior, allowed values, ordering/pagination, and the train double-count caveat. No redundancy or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given a search tool with five parameters and no output schema, the description covers the essential operational details: what is searched, how to filter, ordering, pagination, and an edge case about split counts. It implicitly indicates the return type (tasks) and is sufficient for an agent to invoke correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate. It explains the meaning of query (substring), partition ('all', 'shellops', 'shellops_pro'), and split ('all', 'train_src', 'train', 'test'), and implies limit/offset via 'explicit pagination'. It does not explicitly define limit/offset constraints, but given conventional naming, the added meaning is substantial beyond the empty schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description begins with a specific verb ('Find') and a clear resource ('real ShellOps CLI benchmark tasks'), then defines the search scope (substring in complete instruction, task ID, or published task type). This distinguishes it from siblings like Agentic_RL_search_evidence (searches evidence) and Agentic_RL_get_task (retrieves a specific task).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides clear context on how to filter results via partition and split, and explains ordering and pagination behavior. However, it does not explicitly state when to use this tool versus siblings (e.g., 'use get_task when you have a task ID'), though the verb and resource imply the use case. There are no exclusions or alternative routing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.