Server Quality Checklist

Tool Scores

• list_rabbitholesA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided; description carries full burden. States sorting order and return fields but does not explicitly declare read-only nature or any other behavioral traits. Adequate for a simple list tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with purpose, no wasted words. Efficiently conveys essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Sufficient for a zero-parameter tool with no output schema. Mentions return fields and ordering; missing pagination details but likely unnecessary given simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, so baseline is 4. Description adds no parameter information, but none is needed as schema coverage is 100%.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists saved Rabbitholes sorted by last updated, specifies return fields (id, title, last-updated time, node count), and distinguishes from sibling open_rabbithole by indicating it enables resumption.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context: use to list rabbitholes for subsequent resume via open_rabbithole. Does not explicitly state when not to use or compare to other siblings, but the purpose is straightforward.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ingest_pdfA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description explains output formats (PNG naming), opportunistic embedded rasters, and the agent's role in composing markdown. Lacks mention of error handling or prerequisites.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is detailed but each sentence adds value. It front-loads the core purpose and avoids redundancy with the schema. Could be slightly more concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers input, output naming, usage patterns, and alternatives. Lacks explicit return value structure (beyond ingest_id), but given no output schema, it is mostly complete for an extraction tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline 3. The description adds context on default pages range, hole_id usage, and includes a note on arXiv preference, enhancing parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool extracts a local PDF into image assets and per-page text, with specific naming conventions. It is distinct from siblings like answer_branch or open_rabbithole.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance on using open_rabbithole with the returned ingest_id or passing hole_id for direct attachment, and advises against PDF ingestion for arXiv links. Does not exhaustively cover all scenarios but is helpful.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• answer_branchA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description fully discloses behavior: streaming chunks, final blocking call, concatenation rules, authoring vocabulary, asset handling, and timeout recovery. It covers safety (don't re-send content) and formatting expectations comprehensively.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is lengthy but well-structured with clear sections (authoring vocabulary, streaming choreography). It is front-loaded with purpose. Could be slightly more concise, but the detail is warranted given tool complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description covers all aspects: usage triggers, parameter details, streaming protocol, error handling, formatting rules, and asset management. No gaps for agent execution.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 100% coverage but description adds deep semantics: explains streaming protocol, chunk vs final, asset referencing, title requirement on final call, and content formatting. Goes far beyond schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Answer one pending branch request from an open Rabbithole' and specifies the trigger condition. It differentiates from sibling tools (ingest_pdf, list_rabbitholes, open_rabbithole) by focusing on answering branch requests.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to call this tool (after open_rabbithole or answer_branch returns status='branch_request') and provides handling for various scenarios (empty selected_text, lens, keep_listening status, timeout). No ambiguity about usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• open_rabbitholeA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses behavior: the tool blocks until human action, returns statuses like branch_request, keep_listening, and session_closed, and explains how to handle timeouts by re-calling with hole_id. It details branch_request semantics including selected_text, lens, and rehydration.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is lengthy but necessary due to the tool's complexity. It is well-structured: starts with purpose, then parameter usage, then return behavior. Every sentence adds value, though a minor trim could improve conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the 7 parameters, no output schema, and complex interaction patterns, the description covers all scenarios: new hole vs resume, branch_request handling, keep_listening, timeout recovery, and rehydration. It is fully complete for an AI agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds significant context beyond parameter names: e.g., base_url resolves relative links, assets are referenced as asset:name.png, and ingest_id is for staged PDF assets. This enriches the schema's baseline descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool opens a document on an infinite canvas for reading and exploration. It distinguishes between starting a new hole (with title and content or file_path) and resuming a saved hole (with hole_id), which differentiates it from sibling tools like list_rabbitholes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use each parameter combination, such as using base_url for URL-sourced content, assets for local images, and ingest_id for PDFs. It also explains when to use list_rabbitholes to find saved holes and how to handle branch requests and keep_listening responses.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.