Server Quality Checklist

Tool Scores

• hypothesis_getA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden of behavioral disclosure. It does not mention any side effects, permissions, rate limits, or potential large data transfer. The only behavioral hint is that it returns a comprehensive object, and the instruction to read it before reviewing implies it is read-only, but this is not explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, consisting of two short sentences. It is front-loaded with the core purpose ('One hypothesis in full') and lists the included content without unnecessary elaboration. There is no fluff or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description provides a good overview of what the tool returns, enumerating the fields included. It also gives usage context. However, it does not mention any limitations or optional behaviors (such as how include_matches affects the response), though the schema handles that. Overall, it is fairly complete for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides clear descriptions for both parameters (hypothesis_id and include_matches). The tool description does not add any additional meaning or context about these parameters; it focuses solely on the output. Since schema coverage is 100%, the baseline is 3, and the description adds no extra value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool returns a comprehensive single hypothesis object including all relevant details. It explicitly mentions the content (statement, detailed text, citations, Elo history, reviews, tournament matches) and provides a clear use case ('Read this before reviewing or ranking it'). This distinguishes it from sibling tools that likely list or retrieve summaries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a direct instruction to read this before reviewing or ranking, indicating when to use the tool. However, it does not explicitly mention alternative tools or specify when not to use it, though the context of 'full hypothesis' implies it is the comprehensive retrieval option.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• session_statusA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden of behavioral transparency. It indicates that the tool returns status information (status, counts, Elo spread, budget, existence of final overview) but does not disclose whether it is read-only (presumably yes), any side effects, or whether it blocks. It provides useful detail on the returned data but could be more explicit about non-mutating behavior, especially given it is a polling tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that efficiently enumerates all key metrics and ends with a clear purpose statement. Every part contributes value, and it is front-loaded with the core purpose. No fluff, perfect length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity (single parameter, no output schema) and the rich list of returned metrics in the description, the description is fairly complete for an agent to understand what to expect. It lacks details on return format or error handling, but for a polling tool, this is sufficient. The provided info covers the essential data points, so a 4 is warranted.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100% (session_id is described as 'Session id, e.g. ses_01J...'). The description adds no additional parameter semantics beyond that. Since the schema already covers the parameter, the baseline is 3, and the description does not add extra meaning, so this score is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: reporting progress for a single session, with specific metrics listed. It distinguishes itself from siblings like session_list (which lists sessions) and overview_get (which gets the final overview) by focusing on session status polling. It could be more explicit about being a read-only status check, but the purpose is clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states 'This is the polling tool,' which is a clear usage guideline for when to use this tool (for polling status). It does not explicitly say when not to use it or name alternatives, but it implies a monitoring context rather than a control context, distinguishing it from session_pause/abort/start. The context is somewhat clear, but exclusions are not spelled out.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• overview_getA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It adds one useful trait (the existence condition), but does not explain what happens if the session hasn't finalized (e.g., error vs. empty response), nor does it confirm read-only semantics or describe the output structure. Some transparency is present, but significant gaps remain.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with two concise clauses. It delivers the essential purpose and a key condition without any fluff. Every word earns its place, making it highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no output schema, the description gives a reasonable high-level idea of the return content ('meta-review synthesis'), but it omits details like the expected structure or format of the overview, and does not clarify behavior when the precondition is unmet. It is adequate but not comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already documents the single parameter (session_id) with an example. The description adds no additional meaning to the parameter. Since schema coverage is 100%, the baseline is 3; the description neither enhances nor detracts from parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'The final research overview for a session: the meta-review synthesis of the ranked hypotheses.' This is specific and distinguishes it from siblings like hypotheses_list (which lists individual hypotheses) and hypothesis_get (fetches a single hypothesis). The resource is explicit, and the verb 'get' is implied by the name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear usage condition: 'Only exists once the session has finalized.' This tells the agent when the tool is appropriate to call. It doesn't explicitly name alternatives or exclusions, but the condition effectively implies it should be used only after finalization. This is sufficient guidance for a simple retrieval tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• session_listA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the burden. It discloses that sessions are listed newest first and includes specific fields, but it does not explicitly state it is read-only or mention any side effects, pagination behavior, or error cases. Basic safety profile is implied by 'List' but not explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the action and details, then a practical usage hint. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with one parameter and no output schema, the description covers the returned fields, ordering, and a use case. It lacks explicit mention of pagination or result limits beyond the limit parameter, but overall it is sufficiently complete for the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers 100% of the single parameter (limit) with a clear description ('Max rows. Default 20.'). The tool description adds no additional parameter semantics, so it meets but does not exceed the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists sessions with specific fields (status, hypothesis count, top Elo, budget spent) and sorting (newest first). It is distinct from siblings like session_status or session_pause, and the usage hint further clarifies its purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use it: 'to find a session id the user refers to by topic.' It does not mention alternative tools for exclusion, but the use case is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• doctorA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It lists what is checked but does not disclose whether the tool has side effects, what the output looks like, or what 'hash fallback' implies. It gives partial transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, with a clear front-loaded action ('Check the install') and a structured list of what is checked. Every sentence earns its place with no waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no parameters and no output schema, the description covers the key context: what it checks and when to run it. The lack of return format details is a minor gap, but it's adequate for selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the empty schema provides full coverage. Baseline 4 is appropriate since there are no parameter semantics to explain.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states what the tool does: it checks the install configuration, specifying backend, credentials, embeddings, and database location. This is a distinct diagnostic function, differentiating it from sibling session/review tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides an explicit trigger: 'Run this first when anything fails.' While it doesn't mention exclusions or alternatives, the directive is clear and actionable for a diagnostic tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• estimate_costA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description must carry the behavioral disclosure burden. It explains scope ('under the current config'), the kind of estimate provided, and the conditional pricing behavior on metered backends. Still, it does not explicitly state whether the operation is read-only, what side effects it has, or how precise the estimate is.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no filler. The key action, output content, conditional behavior, and usage instruction are all front-loaded and clearly expressed.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no params, no annotations, and no output schema, the description covers the essential inputs ('current config'), outputs (calls/tokens/duration/price), and timing ('before session_start'). It is slightly ambiguous what 'current config' refers to, but it is adequate for this tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is no schema coverage gap to compensate for. The description adds meaning by explaining that the estimate is based on 'the current config' and what is being measured.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies a specific verb-like action ('Pre-flight size of a session') and details exactly what it produces: calls, tokens per agent, duration, and price on metered backends. It also distinguishes itself from siblings by explicitly framing it as a pre-`session_start` step.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives explicit guidance on when to use the tool: 'Run this before session_start and show the user.' However, it does not mention when not to use it or name alternatives from the sibling list, so it stops short of full exclusion guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• session_abortA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It adds 'for good' indicating irreversibility and 'at its next check' showing deferred execution, which are useful behavioral details beyond the name alone.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no wasted words. The action is front-loaded, and the second sentence adds a key behavioral detail without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no output schema, the description explains the purpose and the timing of the effect. It does not describe return values, but that is not critical given the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for session_id, and the description does not add any parameter-specific meaning. Baseline 3 applies because the schema already provides adequate documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Abort a session for good', which is a specific verb + resource. The phrase 'for good' distinguishes this from session_pause or session_resume, clarifying a permanent termination.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description conveys that this is for permanent termination and notes the main loop exits at a check, providing context for when to use it. It does not explicitly exclude temporary pauses, but the context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• session_pauseA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral disclosure burden and does so well by explaining what happens: workers drain and the loop sleeps until resumed. It implies reversibility and non-destructiveness, though it does not state error behavior for already-paused sessions or permissions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short, front-loaded sentences. The first states the core purpose, and the second adds essential behavioral detail with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with no annotations and no output schema, the description provides enough context: it names the action, the precondition, and the resulting behavior. It does not explain return values, but the tool's simplicity and clear behavior make the description reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already fully documents the only parameter, session_id, with type, requirement, and example format. The description adds no further parameter-specific meaning, so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool pauses a running session, using a specific verb and resource. It distinguishes itself from sibling tools like session_resume and session_abort by describing a reversible pause with workers draining and the loop sleeping until resumed.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'Pause a running session' provides clear context for when to use the tool, and 'until resumed' implies the resume counterpart without explicitly naming session_resume. It lacks explicit when-not-to-use guidance or alternative tool names, but the intended usage is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• hypothesis_recordA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explains the behavior (enters as draft, gets reviewed, competes in tournament) and the significance of recording (makes it real). It also mentions that it writes to the same table as Generation and Evolution, adding context. Minor omissions like permission requirements or error handling are not critical for this tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured paragraph that front-loads the purpose, then explains lifecycle and key parameter usage. Every sentence adds value, no fluff. It is concise yet sufficiently detailed, earning a high score though not as terse as the calibration example.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 7 parameters (5 required) and no output schema, the description provides enough context for the agent to invoke correctly. It explains the tool's role in the hypothesis workflow, the meaning of key fields, and the distinction between fresh vs evolved hypotheses. The rich schema descriptions complement the text, making the overall documentation complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although the input schema already provides descriptions for all parameters (100% coverage), the tool description adds nuanced guidance beyond the schema: it explains the meaning of parent_ids (derivation vs fresh idea) and strategy (how produced, linking to evolution), and emphasizes that summary is used for dedup/clustering. This enriches parameter understanding significantly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Add a hypothesis to a session' and explains its lifecycle (draft, reviewed, tournament). It also distinguishes from siblings like hypotheses_list and hypothesis_get (read tools) and review_record/match_record (review/match tools). The verb-resource pairing is specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear guidance on when to use parent_ids ('Set parent_ids when it derives from existing hypotheses') and explains the consequence of not recording ('an idea only described in chat is never reviewed'). It implies this is the proper way to register a hypothesis versus just chatting, and mentions the strategy enum values for fresh vs evolution. It lacks explicit 'when not to use' but offers enough context for correct usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• feedback_addA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It transparently states the timing of feedback delivery (next planning step or on resume) and the special effect of kind='pin' or 'rejection' with a target on hypothesis state. This goes beyond the schema. However, it does not mention potential failure modes or prerequisites like session existence, leaving some gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no wasted words. It front-loads the primary purpose and immediately provides the timing and state-change nuance. Every clause adds value, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (session state dependency, hypothesis mutation) and the lack of annotations or output schema, the description covers the key behavioral aspects. It could be more complete by specifying that the session must exist or what happens on invalid input, but it is adequate for a tool with this scope. The absence of return value documentation is acceptable since no output schema is provided.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 75% (three of four parameters have descriptions; kind only has an enum). The description adds semantic value by explaining the interaction between kind and target for moving hypothesis state, which is not present in the schema. This compensates for the missing description of kind, justifying a score above the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool injects researcher feedback into a session, with a specific verb and resource. It differentiates from siblings by focusing on feedback injection rather than session control (like session_pause) or hypothesis recording (like hypothesis_record). The mention of how it affects hypothesis state further clarifies its unique role.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear contextual usage: it explains when the feedback takes effect (running vs paused sessions). It does not explicitly name alternative tools or state when not to use it, but the behavioral context is sufficient for an agent to infer the appropriate use case. Since sibling tools like review_record or match_record are not referenced for exclusion, this is not a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• hypotheses_listA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the ordering by Elo, the exact fields returned, and the deliberate omission of bodies. It does not explicitly mention pagination behavior or read-only status, but these are inferable from the 'List' verb and the limit/offset parameters in the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that states the action, scope, ordering, and fields returned. The following note about omitted bodies and the alternative tool adds essential guidance without waste. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a list tool with four parameters and no output schema, the description is nearly complete: it specifies the return field set, ordering rule, and key omission. It could have explicitly described pagination behavior, but limit/offset parameters in the schema cover that. Overall, it provides sufficient context for an AI agent to invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description does not add further meaning to individual parameters like session_id, limit, state, or offset beyond what the schema already provides. It only restates the overall purpose rather than detailing parameter nuances.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists a session's hypotheses ranked by Elo, enumerating specific fields (title, summary, state, matches played, dedup cluster). It distinguishes itself from hypothesis_get by noting bodies are omitted, and from session_list by focusing on hypotheses within a session.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides an explicit alternative: 'Bodies are omitted; use hypothesis_get.' This communicates a clear when-not scenario (when bodies are needed) and points to the correct sibling tool. The context implies this is the go-to for summary listings of hypotheses per session.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• session_resumeA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that the engine reclaims expired task leases and continues where it stopped, and that it returns immediately, requiring polling of session_status. This gives insight into behavior, though it lacks details on error handling or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise—two sentences—and directly conveys the essential information without unnecessary filler. It is well-structured and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of an output schema, the description explains the immediate return and the need to poll session_status. It also explains the lease reclaiming behavior. It could be more complete about success/failure conditions, but for this simple tool it is adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter 'session_id' is well-described in the schema with an example, so the schema coverage is high. The tool description does not add extra information, but it is not needed given the clear schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool resumes a paused or interrupted session. It is specific and distinguishes this tool from siblings like session_start, session_pause, and session_abort.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description indicates when to use it (for resuming sessions) but does not explicitly mention alternatives or when not to use it. It does provide a hint that it returns immediately and to poll session_status, which implies a usage pattern.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• session_startA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It states the detached process, early return after session id exists, and duration of minutes to hours. This is valuable context. It does not cover failure modes or edge cases, but for a session start, the disclosed behavior is sufficient to set expectations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded. It opens with the core purpose, then explains the asynchronous behavior, and finishes with practical instructions. Every sentence contributes value; no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity—a long-running asynchronous process with multiple internal stages—the description adequately covers the key context: detached execution, early return, polling, and cost estimation. It does not describe the output or error handling, but since there is no output schema and the tool is a starter, the provided context is sufficient for typical usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% (all 6 parameters have descriptions), so the schema already provides parameter semantics. The tool description does not add extra details about parameters beyond what the schema offers. Per the baseline rule for high coverage, a score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb and resource: 'Start a new research session' from a natural-language goal. It distinguishes from siblings like session_status and session_resume by focusing on the initiation action and describing the long-running process. The purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly directs the agent to poll session_status for progress and to call estimate_cost to agree on scale before starting. It also implicitly distinguishes from session_resume by focusing on new sessions. These explicit references to sibling tools provide strong usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• match_recordA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses key side effects: 'apply the Elo update' and idempotency per (pair, round_id). Since no annotations are provided, the description carries the full burden, and it does so well by noting repeat recordings move no Elo. It doesn't mention return format, but that's not a critical gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with the main purpose, followed by criteria guidance and idempotency note. Every sentence earns its place with no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having no annotations and no output schema, the description covers purpose, comparison criteria, rationale expectations, and idempotency behavior. The mode parameter is documented in the schema, so not repeating it is fine. This is sufficient for an agent to invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds value beyond the schema by specifying what 'rationale' should contain (criteria-based reasoning) and clarifying that round_id controls idempotency/rematching. This extra context lifts it to 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with a specific verb and resource: 'Record the outcome of a head-to-head comparison between two hypotheses and apply the Elo update.' This clearly distinguishes the tool from sibling record tools like hypothesis_record and review_record, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The context is clear: use this when you need to record a comparison between two hypotheses and update their Elo ratings. It doesn't explicitly name alternatives or state when not to use it, but the head-to-head comparison context is distinct enough. The rationale guidance ('in the terms the criteria are stated in') also adds practical usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• review_recordA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that evidence requires a url and excerpt, that a 'full' review promotes a draft to reviewed, and the consequence of not recording first. These are behavioral traits beyond a simple 'record' action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with purpose, then critical constraints. No fluff, each sentence adds essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 8 params, nested objects, and no output schema, the description covers the key edge cases: evidence provenance, promotion logic, and ordering. It gives enough context for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 75%, so baseline is 3. The description adds value by explaining evidence requirements (url+excerpt you actually read) and the effect of 'full' kind. This goes beyond the schema's per-field descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States exactly what it does: 'Record a review against a hypothesis'. Specifies verb+resource, and adds context about format and side-effect (promotes to reviewed). Clearly distinct from siblings like 'hypothesis_record' which record hypotheses, not reviews.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context: use it when you have a review to record, and gives a critical ordering constraint ('Do not report a review to the user without recording it first'). Does not explicitly name alternatives, but the purpose is self-evident and the constraint is practical.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.