Server Quality Checklist

Tool Scores

• confirm_doiB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully convey behavior. It mentions fetching metadata and validating existence but does not explain what happens if the DOI is invalid, what metadata is returned, or any side effects (e.g., network requests, rate limits). This is insufficient for a tool with no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise at one sentence, front-loading the action and purpose. It is efficient but loses a point for being too brief, sacrificing helpful details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity (one parameter, no output schema, no annotations), the description is incomplete. It does not cover error handling, return value, prerequisites, or authentication needs. A validation tool should at least explain the expected outcome (e.g., status or metadata).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It mentions 'a DOI' as the parameter, which adds minimal meaning beyond the schema. It does not provide format examples, restrictions, or guidance on values, leaving the agent underinformed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Fetch metadata', the resource 'DOI from CrossRef', and the purpose 'to validate it exists before adding to Zotero'. It effectively distinguishes this tool from siblings like 'add_item_by_doi' which presumably adds the item after validation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool should be used before adding to Zotero, providing clear context. However, it does not explicitly state when not to use it or mention alternatives, though the sibling list gives some guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• zotero_get_itemB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description indicates a read-only operation ('Retrieve'), which is consistent with typical get semantics. However, no annotations are provided, and the description does not elaborate on potential side effects, authentication needs, or rate limits. It is adequate but lacks deeper transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, clear sentence with no extraneous information. It is appropriately sized and front-loaded with the action and object.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple retrieval tool with one parameter and no output schema, the description is minimally complete. It covers what and how, but lacks information about return value structure, error conditions, or relationship to sibling tools like zotero_search_items.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description mentions 'by itemKey' but does not explain what an itemKey is (e.g., format, length, source). With 0% schema description coverage, the description should compensate, but it only adds minimal context beyond the schema's type and requirement.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Retrieve' and the resource 'details of a specified item in the Zotero library by itemKey'. It distinguishes from sibling tools like zotero_search_items (which searches) and add_item_by_doi (which adds).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives such as zotero_search_items for finding items or zotero_read_pdf for PDF content. The description implies usage for getting item details by key but provides no explicit context or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_item_by_doiA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. Discloses key behaviors: fetches metadata from CrossRef, checks duplicates by DOI, creates only if not exists. Good transparency, though lacks details on failure modes or auth requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences, each adding meaningful information: action, process, and condition. No redundant text, front-loaded with purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no output schema, the description provides a complete workflow: fetch metadata, check duplicate, create if new. It could mention return value or error handling, but overall sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Only one parameter 'doi', and schema has 0% description coverage. The description does not add extra meaning about the parameter format or constraints, but it explains how the DOI is used (fetch metadata and check duplicate), which provides context. The parameter is simple and self-explanatory from the tool name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the verb 'Add', resource 'item to Zotero library', and method 'by DOI'. Distinct from sibling tools like zotero_get_item (retrieve) and confirm_doi (confirm).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies use when you have a DOI to add an item, but does not provide explicit guidance on when to use this tool versus alternatives, nor exclusion criteria. No mention of prerequisites or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• zotero_read_pdfA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description carries the full burden. It discloses that it reads a PDF and can read all pages or a single page. However, it does not mention failure modes, authentication, rate limits, or return format, leaving some transparency gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no fluff. The first sentence front-loads the tool's purpose, and the second sentence explains the parameters efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool (3 params, no output schema), the description covers the core functionality and parameter behavior. However, it lacks details on return values and error handling, which would be expected for completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description adds meaning for two of three parameters: attachment_index (1-indexed, default 1) and page_number (1-indexed, optional). itemKey is not explained but is implied by the tool name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Read' and the resource 'PDF attachment from a Zotero item'. It is distinct from sibling tools which handle adding items, confirming DOIs, or searching items.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains how to use the parameters (attachment_index and page_number) but does not explicitly state when to use this tool versus alternatives or provide exclusions. Usage is implied by the context of reading PDFs.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• zotero_search_itemsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses search constraints (author/title, excludes attachments, limit 30) but does not mention whether the tool is read-only, authentication needs, or rate limits. The behavior is implied to be a simple search, which is adequate but not fully transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is concise, front-loaded with the action, and contains no unnecessary words. Every part is relevant and earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema), the description covers the essential aspects: what it searches (by author/title), exclusions (attachments), and result limit (30). It could mention search syntax or result structure, but overall it is fairly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has one parameter 'q' with no description (0% coverage). The description adds significant meaning by specifying that 'q' searches by author and title, compensating for the schema's lack of documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches Zotero items by author and title, and specifies exclusions (attachments) and limit (30 results). This differentiates it from siblings like zotero_get_item (single item retrieval) and add_item_by_doi (adding items).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool (searching by author/title) but does not explicitly mention when not to use it or alternatives. It provides clear context for its typical use case.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.