Lemma Park

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description bears full behavioral disclosure. It describes return content but omits read-only safety, side effects, authentication needs, error behavior, or what happens if the theorem is not found.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence that wastes no words and puts the core resource first. It is a sentence fragment rather than a complete sentence, which slightly hurts readability.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no annotations, no output schema, and 0% parameter coverage, the description should carry more of the load. It covers return values reasonably well but leaves parameters and usage context undocumented.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% for two required parameters. The phrase 'One theorem in a Lean program' loosely maps 'program' to a Lean program and 'name' to a theorem, but adds no format, syntax, or validation detail beyond the bare property names.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific resource (a theorem in a Lean program) and enumerates what the tool returns: statement, proof status, and prior attempts with blockers. This distinguishes it from get_program and other siblings, though it never states the retrieval verb explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus get_program, list_programs, or search_notebook. The description only describes content, leaving the agent to infer usage context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden. It outlines the content fields but says nothing about read-only status, whether the lookup requires auth, or what happens when the id is unknown or missing — significant gaps for a retrieval tool with zero annotation coverage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

A single compact clause with no filler, and the leading 'One problem' front-loads the scope. It reads as a fragment rather than a sentence, but every element earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description usefully enumerates the return content, which compensates for the absent output schema. But with no annotations and no explanation of the id input, and no usage context, it is only minimally adequate for a one-parameter lookup tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% for the single required 'id' parameter, so the schema alone documents nothing. The description never mentions the id, its format, or where it comes from, leaving the only input entirely uncharacterized.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description enumerates what the tool returns (goal, rationale, best verified result, current status, how to work on it), which implies a single-record lookup, and the singular 'One problem' contrasts with the sibling list_programs. However, it never states the action verb — nothing says 'retrieve' or 'get' — so the agent must infer that this fetches details for one program by id rather than reporting on it.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no when-to-use guidance, no prerequisites, and no named alternative, even though list_programs exists as the obvious plural counterpart. The agent is left to infer that this is the detail view for a single program.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations and no output schema, the description carries the full disclosure burden. It usefully warns that joining runs a program on the user's computer and consumes their agent subscription, which is meaningful side-effect context, but it does not describe what invoking this tool itself returns or that it is a safe read-only lookup.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tightly written sentences, purpose first and the caution second, with no filler or redundancy. Nothing could be removed without losing information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple informational tool this is close to adequate, but it omits what the tool actually returns (steps, links, prerequisites) and gives no sense of the content's shape. With no annotations or output schema to fill that gap, the description should say more about the response.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so there are no parameter semantics to explain; the empty schema is self-evident. Baseline 4 applies for a no-parameter tool.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description identifies the topic precisely: instructions for how a person joins the lab and contributes their coding agent's time. It is clearer than the bare name, but it never states that the tool returns documentation/instructions, and it does nothing to distinguish itself from siblings like get_program or list_programs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The clause 'ask them first' hints that this material should be surfaced before the joining action occurs, but it never states when to call this tool versus alternatives, nor what a caller should do with the result. Usage is only implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full behavioral burden. It usefully discloses return content ("with progress on each"), implying a read-only collection operation, but says nothing about ordering, pagination, volume, or permissions for what is presumably a public API surface.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with no filler; the resource and the returned payload are both stated in one breath and every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter, no-output-schema list tool, the description covers what it returns and to whom the problems belong. Only the shape of the returned list (ordering, pagination, fields per item) is left implicit, which is a minor gap given the tool's simplicity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so there is nothing for the description to disambiguate; the schema is empty and coverage is 100%. Baseline 4 applies for a parameterless tool.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific verb ("List") and resource ("open problems Lemma Park is working on"), plus the added detail that each entry carries progress. It's clear what the tool returns, but it never contrasts itself with the singular get_program sibling, so an agent gets no explicit differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no statement of when to use this versus get_program or get_challenge. The plural "List" versus the singular siblings weakly implies an overview/collection use case, but that is inference, not guidance, and no prerequisites or exclusions are given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full behavioral burden. It implies read-only behavior and that only the 'latest' messages are returned, which hints at a limited window, but it never says how many messages, the ordering, whether pagination exists, or what permissions are needed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

A single short sentence with no filler and the key scope ('latest messages') front-loaded. It is efficient, though arguably too terse to be 'appropriately sized' for a tool with an unexplained required parameter.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No annotations, no output schema, and 0% parameter description coverage mean the description must fill all gaps, yet it omits the meaning/format of the required 'program' argument and any sense of what the returned messages contain. It is not sufficient for an agent to call the tool confidently.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0% and the single required parameter ('program') is undocumented in both schema and description. The description's reference to 'a problem's chat' loosely gestures at the identifier but does not clarify whether it expects an ID, name, or slug, and the problem/program naming mismatch adds confusion.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Read') and resource ('chat') scoped to 'latest messages,' which is clear on its own. However it offers no differentiation from siblings like get_program or search_notebook, and it says 'problem's chat' while the schema requires a 'program' parameter, leaving a minor terminology ambiguity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no when-to-use guidance, no prerequisites, and no mention of alternatives. The agent must infer entirely on its own that this is the way to inspect a chat versus using search_notebook or get_program.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. 'Search' implies a read-only operation, but the description says nothing about matching semantics, result limits, ordering, or what a hit looks like. For a tool with zero structured behavioral coverage, this is thin.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

A single well-formed sentence that front-loads the verb and resource and wastes no words. It is appropriately sized for the amount of information it chooses to convey, though that amount is small.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With two undocumented parameters, no annotations, and no output schema, the agent needs the description to supply search semantics, program-filter behavior, and result shape. One generic sentence does not cover any of that, so the definition is materially incomplete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% for both parameters. The description's mention of 'verified results, dead ends and proposals' loosely hints at what 'query' searches, but it never explains how 'program' scopes the search or what form the query string takes, which is needed to invoke the tool correctly.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (search) and resource (the lab's notebook) and even characterizes the indexed content (verified results, dead ends, proposals). It does not distinguish itself from any sibling, but none of the listed siblings (get_challenge, get_program, read_chat) appear to overlap, so the risk of misselection is low.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this search versus the sibling read tools (read_chat, get_program), nor any prerequisite or scope note. The agent must infer that this is the free-text retrieval entry point from the name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.