Server Quality Checklist

Tool Scores

• get_cachedB
  3.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and openWorldHint=false, so the description is not required to repeat that. It adds valuable context: the no-network guarantee and the effect of the detail and preferred_only parameters on the response shape. This goes beyond the annotations without contradicting them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the main purpose, followed by parameter-specific notes. No wasted words, but the structure could be improved by grouping more parameter information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 8 parameters, one required, no output schema, and the presence of a nested strategy object, the description is insufficient. It does not describe the return format, the meaning of modes, strategy, max_tokens, or download_dir, leaving an agent with significant gaps in understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must explain parameters. It only explains detail and preferred_only, leaving six other parameters (doi, modes, strategy, max_tokens, download_dir, include_refs) undocumented. The nested strategy object is particularly complex and unexplained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool looks up DOI-proven cached files within a download_dir and explicitly mentions 'without touching the network', which differentiates it from fetching tools. However, it does not name a specific sibling tool, so it stops short of the strongest differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage context is implied: it is for looking up cached files without network access. But there is no explicit statement about when to use this vs. fetching tools or any exclusions/alternatives, so guidance is not fully provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• browser_preflightA
  3.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description goes beyond annotations by disclosing that the tool opens publisher pages and may update filtered storage-state, and by listing behaviors it never performs. This is valuable context for a tool with readOnlyHint=false and idempotentHint=false, though 'filtered storage-state' is somewhat vague.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no filler. The primary purpose is front-loaded, and the second sentence adds behavioral caveats efficiently. Every phrase contributes useful information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 7 optional parameters, no output schema, and low schema description coverage, the description is not complete enough for an agent to invoke the tool confidently. It omits return value/output expectations, explains no parameter semantics beyond provider and storage-state, and the side effects on storage-state are left underspecified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for parameter meaning. It only implicitly clarifies that provider may select one or all providers and that storage-state may be updated, but it says nothing about detail, test_url, timeout_ms, browser_user_agent, save_storage_state, or storage_state_path.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb ('live-check'), a specific resource ('shared browser HTML path'), and scope ('one provider or all browser providers'). It is distinguishable from siblings like fetch_paper or provider_status by its focus on the browser HTML path, though it does not name any sibling directly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It clearly conveys when this tool is appropriate: when a live preflight of the browser HTML path is needed, for one or all providers. It also gives exclusions ('never runs PDF fallback, automatic authentication, or runtime installation'), which helps agents avoid misusing it, though it does not explicitly point to alternative sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• batch_fetchA
  3.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses remote service access, shared cache/artifact/Markdown writes, atomic batch_results writes, and the bounded text-sample behavior of detail=bounded. This meaningfully enriches the annotations, which only indicate non-read-only, non-idempotent, open-world behavior. These disclosures align with the annotations and provide important side-effect context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact: four sentences front-load the core action and then add side effects, a special detail mode, and a runtime prerequisite without repetition. It is appropriately sized given the tool's complexity, though the terminology is dense. Every sentence contributes meaningful information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For an 18-parameter tool with no output schema and no per-property descriptions, this definition leaves many invocation decisions unexplained, such as modes, strategy fields, max_tokens, download behavior, artifact modes, and markdown output settings. It captures high-level behavior and key defaults but is not complete enough for an agent to confidently build all valid requests. The lack of return-value documentation further compounds the gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate, but it only explains detail, batch_results, and overwrite. The remaining 15+ parameters, including modes, strategy, max_tokens, concurrency, no_download, artifact_mode, and content_max_chars, are left to raw titles and defaults. The few explanations are useful but insufficient for such a large nested schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: fetching 1–50 papers under bounded concurrency, producing input-ordered manifest/acceptance results. This clearly positions it as a batch fetch operation and references fetch_paper to anchor its behavior. It does not explicitly contrast with siblings like batch_check, so differentiation is good but not maximal.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description establishes clear usage context: use this tool to fetch multiple papers rather than a single one, with effects shared with fetch_paper. It also gives a concrete prerequisite for browser routes and notes the overwrite default. It does not explicitly state when not to use it or name alternatives such as batch_check, but the context is reasonably clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• resolve_paperA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations already provide readOnlyHint and openWorldHint, so the safety profile is covered. The description adds useful behavioral context by stating that the result is a 'candidate' that is 'normalized', implying the tool may not always return a definitive match and that output is standardized. No contradiction exists.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads the verb, the resource, the input forms, and the output form. Every word earns its place, with no filler or repetition of schema field names.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With four optional parameters, no parameter descriptions, no output schema, and no usage guidance, this description is too thin to fully equip an agent. It does not state whether at least one parameter is required, what happens if multiple forms are provided, what a 'normalized paper candidate' contains, or how resolution failures should be handled.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description carries the full burden for parameter semantics. It mentions 'DOI, URL, or title query' but does not map these onto the schema's query, title, authors, or year parameters. The authors and year fields are left entirely unexplained, and no guidance is given on how the parameters combine or which are mutually exclusive.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific action ('Resolve'), a resource ('paper'), and the accepted inputs ('DOI, URL, or title query'), and clarifies the output ('normalized paper candidate'). This clearly distinguishes it from siblings like fetch_paper and batch_resolve, which involve retrieval and batching rather than resolution.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied by the purpose: use this tool when you need to turn an identifier or title into a normalized paper candidate. However, there is no explicit guidance about when to prefer this over fetch_paper, has_fulltext, or batch_resolve, nor any mention of exclusions or workflow sequencing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• batch_resolveA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and openWorldHint=true, so safety is covered. The description adds behavioral context by mentioning 'shared transport reuse' and 'optional cross-host concurrency,' which are not present in the schema. It does not disclose failure behavior or response shape, but those are not required by the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one dense sentence with no filler, front-loaded with the core action and batch scope. Every clause contributes either resource identification or behavioral context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool accepting up to 50 queries with no output schema, the description leaves out return format, per-item failure semantics, and any distinction from batch siblings. It is adequate for understanding the basic operation but not fully complete for invoking it safely at scale.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaning by explaining that queries can be DOIs, URLs, or titles and that concurrency is optional and cross-host. However, it omits the array-size limits and the concurrency range (1–8), leaving the schema to fill those gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb ('Resolve') and a clear resource ('multiple DOI, URL, or title queries'), and explicitly signals batch scope with 'multiple' and 'cross-host concurrency.' This distinguishes it from single-resolution siblings like resolve_paper.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The word 'multiple' implies the tool is for batch resolution, but the description never states when to prefer it over sibling tools such as resolve_paper or batch_fetch, nor does it provide any exclusions or when-not-to-use guidance. Usage is implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_cachedA
  4.2/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The readOnlyHint annotation is reinforced and expanded with meaningful behavioral detail: it never touches the network and never scans loose files, and it operates only on the current cache index for DOI-proven entries. This goes well beyond what the annotation alone provides.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single front-loaded sentence with no filler. The core action, target resource, and key behavioral constraints are all delivered compactly and in logical order.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only list operation with one optional parameter, the description is mostly adequate and the absence of an output schema is mitigated by the clear 'List... downloads' phrasing. However, the role of download_dir is completely undocumented, leaving ambiguity about whether a custom path changes the index or selects a different cache.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description never mentions the optional download_dir parameter. While the parameter name hints at a directory, the description does not explain how it relates to the 'current cache index' or what null/default behavior means, so the low-coverage burden is not met.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List'), a specific resource ('DOI-proven cached downloads from the current cache index'), and adds explicit exclusions ('without touching the network or scanning loose files'). This clearly separates it from fetching, resolving, and single-item retrieval siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly conveys when to use it: when the agent wants cached, DOI-proven downloads without network access or loose-file scanning. It does not explicitly name alternatives or state when not to use it, but the context is strong enough to guide selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• provider_statusA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark this as read-only, and the description adds useful behavioral context by clarifying that no browser or publisher pages are opened. It does not fully explain what 'local dependency readiness' entails, but it does not contradict the annotations and adds meaningful safety-relevant detail.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with zero filler. The core purpose is front-loaded, and the exclusion of browser behavior plus the sibling pointer are both essential and efficiently packed.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the tool's role and its key limitation well, but with no output schema and no parameter-level detail, the agent still has gaps around return shape and the meaning of 'dependency readiness.' It is adequate for tool selection but not fully complete for invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, yet the description provides no explanation of the group, detail, or provider parameters. An agent can read the enum values but cannot know what 'compact' vs 'full' returns or what each group filter semantically selects.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Inspect') on a clear resource ('filtered static provider configuration and local dependency readiness') and explicitly distinguishes itself from browser_preflight. An agent can immediately tell what this tool does and what it does not do.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives explicit when-to-use guidance by stating 'never opens Camoufox or publisher pages' and directs the agent to the correct alternative: 'use browser_preflight for live health.' This leaves no ambiguity about when to select this tool versus its sibling.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fetch_paperA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Even though annotations indicate a non-read-only, side-effect-capable tool, the description adds substantial behavioral detail: remote service access, provider artifact writes, MCP cache sidecar, no_download suppression, save_markdown path return, artifact_mode cache retention, asset_profile ImageContent bounds, and access-control boundaries. It goes well beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but every sentence adds meaningful information. It front-loads the core action and defaults, then layers side-effect semantics, cache behavior, and resource pointers. For a tool with 12 parameters and nested options, this is appropriately sized rather than bloated.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity, 0% schema coverage, and no output schema, the description covers the key aspects needed for correct invocation: required query, default modes, side-effect control, cache matching semantics, output forms including path and ImageContent, and a pointer to the provider catalog for environment-specific facts.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description carries the parameter-meaning burden. It explains the purpose and defaults for many parameters, including modes, include_refs, max_tokens, prefer_cache, no_download, save_markdown, artifact_mode, and asset_profile. Some parameters such as download_dir, markdown_filename, preferred_providers, and inline_image_budget are not explained, so it is not a 5.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Fetch one paper as structured article, Markdown, and/or metadata...'. It clearly identifies the core output modes and distinguishes itself from siblings like batch_fetch and get_cached by emphasizing single-paper fetch.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit operational guidance: defaults, fallback behavior, no_download=true suppressing writes, save_markdown=true as an explicit write, artifact_mode semantics, and the warning to use prefer_cache=true only with matching request semantics. It does not explicitly name alternatives like batch_fetch, so it is not a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• batch_checkA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and openWorldHint, and the description adds concrete behavioral context: it never fetches full bodies, never writes downloads, returns input-ordered results, and exposes probe_state, evidence, and errors. This goes well beyond the structured annotations and accurately characterizes the operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but every sentence earns its place. It front-loads the core purpose, then adds mode constraints, result semantics, safety guarantees, and the sibling alternative in a compact form without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only probe tool with no output schema, the description is remarkably complete: it explains what signals are used, how to read results, what not to expect, and how to select the sibling tool for real fetching. The agent has enough to invoke it correctly and interpret its output.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must carry semantic weight. It does clarify mode ('defaults to metadata and accepts only metadata'), queries ('input-ordered results', results[0] for one query), and concurrency ('optional cross-host concurrency'). Minor gaps remain: it doesn't explain concurrency's upper bound or default behavior, though the schema provides those constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb and resource: 'Probe one or more papers using cheap metadata and landing-page signals.' It also distinguishes itself from batch_fetch by explicitly stating that this tool never fetches full bodies or writes downloads, so an agent can tell them apart without opening schemas.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear guidance: use batch_check for probing and batch_fetch for real fetching. It also explicitly states that mode only accepts metadata and that article mode and has_fulltext are removed, preventing agents from attempting unsupported modes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.