Server Quality Checklist

Tool Scores

• zotero_item_fulltextB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states the tool 'Get[s] the full text content', implying a read-only operation, but doesn't disclose behavioral traits such as authentication needs, rate limits, error conditions, or what happens if the item key is invalid. For a tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: a single, clear sentence that states the purpose and parameter context without any wasted words. Every part of the sentence earns its place by conveying essential information efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (1 parameter, no output schema, no annotations), the description is minimally adequate. It covers the basic purpose and parameter semantics but lacks details on behavioral aspects, output format, or error handling. Without annotations or an output schema, the description should do more to be complete, but it meets the bare minimum for a simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 1 parameter with 0% description coverage, so the description must compensate. It adds meaning by explaining that 'item_key' refers to 'a parent item or specific attachment', which clarifies the parameter's purpose beyond the schema's generic 'Item Key' title. However, it doesn't provide details on format, examples, or constraints, leaving some ambiguity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get the full text content of a Zotero item' with the specific verb 'Get' and resource 'full text content'. It distinguishes from sibling tools like 'zotero_item_metadata' (which likely returns metadata) and 'zotero_search_items' (which searches for items). However, it doesn't explicitly contrast with siblings, so it's not a perfect 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by specifying 'given the item key of a parent item or specific attachment', which provides some context on when to use it. However, it lacks explicit guidance on when to use this tool versus alternatives like 'zotero_item_metadata' for non-full-text data or 'zotero_search_items' for finding items first. No exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• zotero_item_metadataB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states it's a read operation ('Get'), but doesn't mention whether it requires authentication, rate limits, error conditions (e.g., invalid item keys), or the format of returned metadata. For a tool with zero annotation coverage, this leaves significant gaps in understanding how it behaves beyond the basic purpose.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('Get metadata information') and includes the key constraint ('given the item key'). There is no wasted text, repetition, or unnecessary elaboration, making it highly concise and well-structured for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no nested objects) but lack of annotations and output schema, the description is minimally complete. It covers the purpose and parameter semantics adequately but misses behavioral details like authentication needs or return format. Without an output schema, the description should ideally hint at what metadata is returned, which it doesn't, leaving room for improvement.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaning by explaining that 'item_key' is used to identify 'a specific Zotero item', which clarifies the parameter's role beyond the schema's generic 'Item Key' title. With 0% schema description coverage and only one parameter, this compensates adequately by providing context, though it doesn't detail the key's format or source. Baseline is high due to low parameter count.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the resource 'metadata information about a specific Zotero item', making the purpose immediately understandable. It distinguishes from 'zotero_item_fulltext' (which likely retrieves full text content) and 'zotero_search_items' (which searches multiple items) by focusing on metadata retrieval for a single item. However, it doesn't explicitly mention what metadata fields are included, keeping it from a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by specifying 'given the item key', suggesting this tool is for when you have a specific item identifier. It doesn't provide explicit when-to-use guidance versus alternatives like 'zotero_search_items' (e.g., use this for known items, use search for unknown items) or mention prerequisites like authentication. The context is clear but lacks detailed exclusions or comparisons.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• zotero_search_itemsA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It describes the search functionality and mentions that results can be looked up with other tools, which adds useful context. However, it doesn't disclose important behavioral traits like whether this is a read-only operation, what permissions are needed, pagination behavior beyond the 'limit' parameter, or error conditions. The description adds some value but leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured in two sentences: the first explains the core functionality with key parameters, the second provides important follow-up context about sibling tools. Every word earns its place with no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with 4 parameters, 0% schema coverage, no annotations, and no output schema, the description does a reasonable job explaining the search purpose and parameters. However, it lacks information about return format, error handling, authentication requirements, and doesn't fully document all parameters (missing 'limit'). Given the complexity and lack of structured documentation, this leaves significant gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must compensate. It explains the purpose of 'query', 'qmode' (with specific mode examples), and 'tag' (including boolean search support). It doesn't mention the 'limit' parameter, but covers 3 of 4 parameters with meaningful context beyond their names. This significantly improves understanding compared to the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Search for items'), resource ('in your Zotero library'), and scope ('given a query string, query mode... and optional tag search'). It distinguishes from siblings by mentioning that results can be looked up with 'zotero_item_fulltext' or 'zotero_item_metadata', indicating this is a search tool while siblings provide detailed item data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool (searching the library with query parameters) and implicitly distinguishes from siblings by noting that results can be looked up with those tools. However, it doesn't explicitly state when NOT to use this tool or provide alternative search methods within the same tool family.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.