Server Quality Checklist

Tool Scores

• paper_batch_searchC
  2.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false and openWorldHint=false, so the safety profile is fully covered without the description. The description adds one behavioral trait beyond that: batching multiple questions into a single round trip. It says nothing about partial results, failure handling per question, or how "adaptively bounded" retrieval behaves.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single front-loaded sentence with no padding or restatement of the tool title. It is appropriately sized, though "adaptively bounded" is unexplained shorthand that spends words without adding meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with no output schema, the description should at least sketch what comes back per question; it does not, so return format and result structure must be discovered by calling. The input side is fully covered by the schema, leaving the definition minimally adequate rather than complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and includes cache behavior for path, the path-vs-paper_id exclusion, and the 32-item maxItems bound. The description only echoes the 32-question limit already enforced by the schema, adding no syntax, format, or selection nuance. Baseline 3 is correct when the schema carries the load.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description pairs a clear verb ("Retrieve") with a resource ("evidence") and states a scope (up to 32 questions in one call), which is enough to tell it is a batch variant. However, "adaptively bounded evidence" is undefined jargon, and no sibling (paper_search, paper_ingest) is named to anchor the distinction. An agent can guess the purpose but not confidently differentiate it from paper_search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no when-to-use guidance, no statement of when paper_search or paper_ingest is the better choice, and no preconditions (e.g. that a paper must be ingested first). The batch framing implies a use case but nothing is stated explicitly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• paper_searchC
  2.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare a safe, idempotent, closed-world read operation, so the safety profile is covered. The description adds useful behavioral context about the return format (Markdown, ranked, page hints, linked images), but says nothing about ranking behavior, limits, or how empty queries behave.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single front-loaded sentence with no filler. It is efficient, though arguably too terse to carry the missing usage and parameter context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description does describe the return shape, which partially compensates. But for a 3-parameter search tool it omits when to choose it over siblings, the meaning of max_results, and any ranking or scope details an agent would need to invoke it confidently.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 67%: query and paper_id are documented (including the 'empty returns the opening chunks' behavior), while max_results has no description. The description adds no parameter-level meaning beyond the schema, so the mid-range baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Return') and a resource ('compact, ranked Markdown evidence with page hints and linked image paths'), which tells the agent what comes back. However, it frames the tool as an output producer rather than stating that it searches a paper's content for a query, and it does not distinguish itself from siblings paper_batch_search or paper_visual.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus the clearly related siblings (paper_batch_search for multiple papers, paper_visual for visual content). The mention that paper_id comes from paper_ingest appears only in the schema, not the description, so no explicit sequencing advice is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• paper_visualB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is covered and the description need not repeat it. The description adds that output is rendered on demand ('on-demand original PDF page/crop'), which hints at server-side rendering, but it omits cost/latency implications, and the phrase 'exactly one' sits awkwardly against the schema's contact-sheet mode for 2-6 evidence IDs. With annotations covering the behavioral bar, a 3 is appropriate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single front-loaded sentence with no filler or redundancy; the core promise (what is returned) leads immediately. It is efficiently sized, though its brevity veers toward under-specification for a 7-parameter tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description partially carries the return-value burden by naming the returned artifact (image, PDF page, or crop), which is useful. However, for a 7-parameter tool with three distinct output modes (single image, PDF page, contact sheet), it never explains how modes are selected or what the contact-sheet return looks like, leaving real gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so every parameter (bbox scaling, asset path, scale cap, page_number, evidence_id, evidence_ids) is already documented in the schema itself. The description adds no parameter-level meaning beyond what the schema provides, which is the correct baseline of 3 when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description gives a specific verb ('Return') and a concrete resource (an extracted image, or an on-demand original PDF page/crop), which is clearer than a bare image-fetch restatement. It does not, however, distinguish the tool from its siblings (paper_ingest, paper_search, paper_batch_search) or state why one would fetch a visual rather than search. Purpose is clear but sibling differentiation is absent.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no when-to-use or when-not-to-use guidance in the description; the only routing hints ('Prefer this for figures', 'preferred over repeated visual calls') live in the schema property descriptions, not the tool description. An agent gets no explicit condition for choosing this tool over paper_search or paper_ingest. Usage is only implied by the tool name.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• paper_ingestB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, so the safety profile is covered. The description adds that the tool materializes cached evidence and images, which is useful outcome context, but it does not explain re-ingest behavior, cache location, or cost beyond what the 'force' parameter text says.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single front-loaded sentence naming the input, the action, and the two outputs with zero filler. Nothing could be removed without losing information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description usefully names the two artifacts produced, but it omits how this tool relates to the search/visual siblings and gives no sense of cost or latency for a parsing operation. Adequate but with clear gaps for a tool sitting in a four-tool family.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so both 'path' (absolute local path with accepted extensions) and 'force' (expensive cache invalidation) are fully documented in the schema. The description adds nothing beyond that, which is the correct baseline when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb (parse), a specific resource (one research file), and the outcome (cached evidence plus referenced images). The word 'one' implicitly scopes it to single-file ingestion, contrasting with paper_batch_search, though no sibling is named explicitly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no explicit when-to-use guidance: nothing says when to ingest versus paper_search, paper_batch_search, or paper_visual, nor whether ingestion is a prerequisite for the other tools. Only the implicit single-file scope hints at usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.