Server Quality Checklist

Tool Scores

• paper_get_pdfA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds behavioral details beyond annotations: it explains the source priority order, what the returned JSON contains (pdf_url, source, title, year, browser_excerpt), and that errors are returned as strings. Annotations already indicate read-only, idempotent, and open-world behavior, so the description complements well without contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with three main sections: purpose, source priority, and return format. It front-loads the core purpose and avoids unnecessary details. Minor redundancy exists in describing the return format twice (JSON fields and str type), but overall it is efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and a clear output, the description covers the key aspects: input, processing logic (source priority), and output format. It also notes the possibility of browser_excerpt in fallback. It is thorough enough for an agent to use effectively, though could have mentioned what happens when no PDF is found beyond returning an error.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers 100% of parameters with descriptions (paper_title). The description restates the parameter structure but adds no new semantic meaning beyond what the schema provides. Given high schema coverage, a score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Find the best open-access PDF URL for a paper by title.' It specifies the action (find best PDF URL) and the resource (paper by title). This distinguishes it from sibling tools like paper_get_fulltext or paper_get_metadata, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  While the description provides a source priority chain (Semantic Scholar OA → arXiv PDF → Unpaywall → gomcp browser), it does not explicitly state when to use this tool versus its siblings. The usage context is implied by the tool's name and purpose, but explicit guidance on edge cases (e.g., when to prefer other tools) is missing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• paper_get_referencesA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds that it returns JSON with specific fields (paper_title, total_references, returned, references) and an error object on failure. This provides behavioral context beyond annotations, such as output structure and error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (4 sentences), front-loaded with purpose, and structured with clear sections for return fields, arguments, and return format. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity (one parameter), the presence of annotations and output schema, the description is complete. It covers purpose, output structure, input requirements, and error handling, leaving no apparent gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a detailed description of 'paper_title'. The description only repeats 'paper_title: str' without adding new meaning, so baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Return the bibliography of a paper — the papers it cites (backward citations)', which is a specific verb+resource. It implicitly distinguishes from sibling tool 'paper_get_citations' (forward citations) by specifying 'backward citations'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool vs alternatives. The term 'backward citations' implies use for getting cited papers, but no exclusion or alternative mention. This is implied usage, fitting the score 3.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• doi_get_metadataA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, covering safety. The description adds valuable behavioral detail: it queries three sources in parallel (Crossref, Semantic Scholar, Unpaywall) and merges results. It also describes the output fields and error format, going beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is somewhat long but well-structured with bullet points for sources and a clear list of returned fields. Each part adds value for a complex tool. Could be slightly more concise, but the structure helps readability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (multiple data sources, merged output), the description is thorough. It explains the parameter format, the sources queried, the full list of output fields, and error handling. With annotations covering safety and output schema existing, it is completely adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with doi parameter already documented in schema. The description repeats the accepted formats (bare DOI or URL) but does not add significant new meaning beyond what the schema provides. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Resolve a DOI to comprehensive paper metadata' and explicitly differentiates from sibling tool paper_get_metadata by stating 'This is the reverse of paper_get_metadata'. It specifies the verb (resolve), resource (DOI to metadata), and scope (multiple sources).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains that this tool is for when you already know the DOI, and that it is the reverse of paper_get_metadata (which likely uses title). It implicitly guides when to use this vs. siblings, but does not explicitly state when not to use it or list alternative tools for different scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• paper_get_citationsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, destructiveHint=false, and idempotentHint=true. The description adds valuable behavioral context: a limit of 100 papers, the specific output fields (title, authors, year, etc.), and potential error response. This goes beyond the annotations, though it does not address rate limits or authentication.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the purpose, then details output structure. It is efficient with no extraneous information, though the Args/Returns section partly duplicates the initial description. Still, it is well-structured and concise, earning a 4.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, list output) and the presence of an output schema, the description is complete. It covers purpose, parameters, and return format thoroughly, with no gaps. Annotations further cover safety and idempotency, making this fully informative.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the description's parameter section merely repeats the schema. The description does not add new meaning beyond what the schema already provides (paper_title with min/max length). Baseline of 3 is appropriate as the description adds no extra semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Return up to 100 papers that cite the given paper (forward citations).' This uses a specific verb ('return') and resource ('papers that cite'), and explicitly mentions 'forward citations' to distinguish it from the sibling tool paper_get_references.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for obtaining forward citations, but does not explicitly state when to use this vs. alternatives like paper_get_references. However, the mention of 'forward citations' and the sibling context (paper_get_references for backward) provides clear contextual guidance. A slight lack of explicit exclusions or when-not-to-use prevents a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• paper_get_fulltextA
  4.2/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses the fetch order (arXiv HTML5 → gomcp browser → abstract-only fallback), character limit, return fields (title, abstract, tldr, fulltext, etc.), and source, adding significant value beyond the annotations which only state readOnly, idempotent, and openWorld hints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, and the fetch order and return structure are clearly presented. It is not excessively long, though it could be slightly more concise by removing the 'Args' and 'Returns' lines that duplicate schema info.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the existence of an output schema, the description adequately explains the return fields and error format. It covers the fetch order and fallback behavior, making the tool's behavior complete for an agent to invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 100% coverage for the single parameter, with a detailed description and example in the schema. The tool description only restates the parameter without adding new semantics, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it retrieves the full text of a paper by title, with a character limit. It lists the specific verb-resource pattern and distinguishes itself from siblings like paper_get_pdf and paper_get_metadata by focusing on full text.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not explicitly state when to use this tool versus siblings. It implies usage for full-text retrieval but provides no guidance on when not to use it or alternatives for PDF or metadata.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• paper_get_metadataA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate read-only, idempotent, non-destructive. Description adds value by detailing the two-step query strategy (Semantic Scholar then arXiv) and listing return fields, providing behavioral insight beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise and well-structured: opening sentence, source explanation, enumerated return fields, explicit args and returns. No unnecessary text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Description covers purpose, behavior, return format, and parameters. With annotations and implied output schema, it provides sufficient context for an agent to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with full parameter description. Description restates the parameter structure but does not add significant new meaning; baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it returns comprehensive metadata for an academic paper by title, specifies sources (Semantic Scholar first, arXiv fallback), and distinguishes from sibling tools that focus on citations, fulltext, or PDF.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description implies use when you have a paper title and need metadata, but does not explicitly state when not to use or compare with alternatives like doi_get_metadata. The fallback behavior provides useful context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.