Server Quality Checklist

Tool Scores

• paper_detailsA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses meaningful behavior: Semantic Scholar is tried first, with OpenAlex/Crossref as fallback, and it lists the specific fields returned. It does not mention rate limits or behavior for nonexistent IDs, but for a read-only lookup the disclosed behavior is solid.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded with the core purpose. Every sentence adds value: the first states the operation, the second covers accepted inputs, and the third summarizes the fallback behavior and output fields. There is no fluff or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter read tool with an output schema, the description is largely complete it covers input formats, source preference, and returned fields. It misses explicit guidance on when to use siblings and what happens when an identifier is not found, but these are minor gaps given the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema only defines doi_or_id as a string with no further description, so the description must compensate. It does: it explains the three accepted identifier types with concrete examples for DOI, arXiv ID (bare, prefixed, and URL), and OpenAlex ID. This is complete parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get full metadata for a paper') and specifies the resource: paper metadata retrieved by DOI, arXiv ID, or OpenAlex ID. It is distinct from the sibling tools such as search_papers or read_paper_full_text, though it does not explicitly call out those alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool: when you already have an identifier for a paper and want full metadata. It gives accepted ID formats and examples, but it does not explicitly say 'use search_papers when you lack an identifier' or otherwise state when not to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• venue_proceedingsA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations available, the description carries the full transparency burden, and it does well: it discloses the data source (DBLP), the exact return fields (title, year, authors, DOI), and the caching behavior ('Results are cached'). It does not cover failure modes, rate limits, or data freshness, but the core behavioral profile is clearly communicated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded: the first sentence states the core purpose, followed by compact parameter explanations and a useful caching note. It is slightly longer than strictly necessary due to the inline examples, but every element earns its place and none is redundant.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that an output schema exists and there are no annotations, the description covers the essential call context: purpose, source, return fields, parameter semantics, and caching. The main gap is the undocumented 'limit' parameter, which may affect result completeness for large proceedings. Overall it is sufficiently complete for an agent to select and invoke the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does for most parameters: venue is explained with both a display name and a DBLP stream key, year is explained as an optional filter, and query is explained as a keyword filter with examples. However, the 'limit' parameter is absent from the description entirely, so one of the four parameters remains semantically underspecified.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'List papers published in a venue's proceedings (via DBLP)'. This clearly differentiates the tool from siblings like search_papers, paper_details, or get_bibtex by bounding it to a venue-scoped survey operation, and it even gives a concrete usage scenario ('ACM e-Energy').

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context by explicitly framing the tool for surveying what a conference 'actually accepts', which tells an agent when this tool is useful. However, it does not explicitly mention alternative tools or state when not to use this one, so it stops short of full when/when-not guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_papersA
  4.3/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explains the automatic fallback from Semantic Scholar to OpenAlex, on-disk caching behavior, DBLP routing for venue queries, and the structured result format. This gives the agent a strong mental model of the tool's behavior beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the tool's purpose and return format, and the subsequent sections for sources and caching are informative rather than padded. It is somewhat long, but every sentence adds useful behavioral or selection detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers return shape, source behavior, caching, and venue routing, which is substantial for a search tool. The main omissions are the limit and year-filter parameters, and there is no explicit guidance on when to use sibling tools instead, but the core invocation path is well supported.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description compensates for the 0% schema coverage by explaining the query, venue, and source parameters in detail, including source-specific syntax. However, it does not describe the limit, year_start, or year_end parameters, leaving a meaningful gap for a tool with six parameters and no schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Search academic papers by keyword,' and enumerates the exact return fields (title, year, venue, authors, DOI, abstract, citation count, PDF link). This clearly distinguishes it from sibling tools like paper_details, get_citation_graph, and read_paper_full_text.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives actionable guidance for source selection, such as marking arXiv as 'best for finding specific recent papers by exact name' and explaining that venue-scoped queries are routed through DBLP automatically. It does not explicitly contrast this tool with sibling tools, but it provides clear context for when to use it and how to choose among internal alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_bibtexA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden. It discloses useful traits beyond the name: Crossref lookup, fallback to OpenAlex metadata, local generation, and caching. This gives the agent meaningful expectations about execution and repeat calls.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences with no filler. The primary action is front-loaded, fallback behavior is clearly parenthesized, and caching is stated in a standalone sentence. Every clause adds information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (one parameter, no annotations, output schema present), and the description covers data sources, fallback behavior, and caching. It could add polish by mentioning failure behavior for unknown DOIs, but the core information an agent needs to select and invoke it is present.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage for the single parameter is 0%, so the description must compensate. It does so by explaining that the paper is identified 'by DOI', giving semantic meaning to the only required parameter. It lacks format/example guidance, but for a single string parameter this is sufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb ('Generate') and a specific output resource ('BibTeX citation entry for a paper by DOI'). It also names the data sources (Crossref, with OpenAlex fallback), which makes it easy to distinguish from siblings like paper_details and get_citation_graph.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the clear use case: obtain a BibTeX citation for a known DOI. It does not explicitly name alternatives or exclusion conditions, but the purpose is specific enough that an agent can infer when this tool is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_citation_graphA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It reveals that results are cached, that the response includes the data source used, and defines the traversal directions. It could additionally mention pagination or limit behavior, but the current disclosure is solid for a read-only lookup tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded, with the core purpose in the first clause and each subsequent sentence carrying useful information. There is no filler or repetition of schema details that are already visible.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For this tool's complexity, the description covers the essential invocation details: ID formats, direction semantics, result nature, and caching. Since an output schema is present, the description does not need to explain return fields, and no critical calling information is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds real meaning by enumerating paper_id formats (DOI, OpenAlex ID, Semantic Scholar ID) and explaining direction values. The limit parameter is left to schema/default semantics, which is a minor gap given its simplicity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb and resource ('Follow a paper's citation graph') and defines direction semantically. It is easily distinguishable from sibling tools like search_papers, paper_details, and get_bibtex because citation traversal is a unique operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides concrete usage context by explaining accepted paper_id formats and the two direction modes with forward/backward semantics. It does not explicitly state when not to use the tool or name alternatives, but the context is clear enough for an agent to select it appropriately.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_paper_full_textA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description discloses the full pipeline: it resolves the open-access PDF, downloads it, caches locally under ~/.cache/research-mcp/pdf/ by default, extracts text with PyMuPDF, and truncates to max_chars. This goes well beyond a bare verb, though it does not cover error behavior when no open-access version is found.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise paragraphs, frontloaded with the action and accepted identifiers, then process details and usage advice. No redundant filler; every sentence adds operational value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 3-parameter tool with an output schema, the description covers all needed call context: accepted input formats, resolution strategy, download/caching behavior, truncation, and fallback advice. Nothing essential is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It clarifies the primary parameter accepts DOI, arXiv ID, or direct PDF URL, and explains max_chars controls truncation. The cache parameter is only implied via the caching default, so coverage is not complete but sufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear action ('Read the full text of a paper') and a specific resource (a paper identified by DOI, arXiv ID, or PDF URL). It contrasts with siblings such as search_papers and paper_details by focusing on full-text retrieval, so an agent can select it unambiguously.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The second paragraph explicitly instructs the agent to use paper_details or search_papers to find the open_access_pdf first and to pass a direct URL if DOI resolution returns no open-access copy. This gives concrete when-to-use guidance and routing to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.