Server Quality Checklist

Tool Scores

• fulltext_searchB
  3.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and idempotentHint=true, so the safety profile is covered. The description adds context about the 'indexed fulltext' requirement and 'local paragraph context' return, which are not in annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences that front-load the core purpose and citekey format with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 5 parameters with 0% schema coverage, and the description provides minimal parameter semantics and no usage context beyond the citekey format. The output schema exists but input semantics are incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, and the description only mentions that citekey must be a single token with no spaces. There is no explanation for query, before, after, or limit, so the description does not compensate for the missing parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as searching within a specific Zotero item's indexed fulltext and returning paragraph context, with a specific verb+resource+scope. It does not explicitly contrast with sibling tools like 'search' or 'fulltext', so it's not fully differentiated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance is provided on when to use this tool versus alternatives. The citekey requirement is mentioned but there is no 'use this when...' statement, leaving the agent to infer usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• searchB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare read-only and idempotent behavior, lowering the bar. The description adds that search covers metadata and indexed fulltext, implying a read-only search but without additional context such as result limits or query syntax. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the primary purpose and a clear alternative. No wasted words; each sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With four parameters, no parameter semantics, and an output schema present, the description covers only the core keyword search behavior. It lacks explanation of filtering by tag or collection, limit semantics, or interaction with sibling fulltext tools, leaving significant gaps for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description only mentions 'by keywords', which adds meaning to the query parameter. The tag, limit, and collection parameters are entirely unexplained, so the description fails to compensate for the lack of schema-level documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches Zotero metadata and indexed fulltext by keywords. It distinguishes from lookup by directing users to use lookup for known citekeys, but does not differentiate from fulltext or fulltext_search siblings, so it is not fully distinguished.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises using lookup for known citekeys, providing a clear exclusion and alternative. However, it does not mention when to use fulltext or fulltext_search instead, so guidance is partial.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fulltextA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and idempotentHint=true, so the safety profile is covered. The description adds useful context about 'indexed' fulltext and 'numbered paragraphs' but does not disclose error behavior, missing fulltext handling, or effects of limit/offset.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with front-loaded purpose and a helpful example for the key parameter. Every sentence contributes value with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With an output schema present, return format is covered. However, the description lacks guidance on when to choose this tool over fulltext_search and does not explain pagination parameters, making it adequate but not fully complete for a 3-parameter tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It clarifies only the citekey format ('single token with no spaces, e.g. 'atz2022''), but does not explain the semantic meaning of limit and offset (e.g., that limit controls paragraph count), leaving those parameters ambiguous.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Return' and a clear resource 'numbered paragraphs for a Zotero item's indexed fulltext.' It distinguishes from siblings like fulltext_search by specifying an exact output format and per-item scope, even without naming alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you have a citekey and want an item's fulltext paragraphs, but it does not explicitly state when to use this tool over alternatives like fulltext_search, nor does it provide any when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• collectionsA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and idempotentHint, so the description is not required to repeat that. The description adds 'current library' as scoping context, but does not disclose any additional behavioral traits such as whether nested collections are included or how results are ordered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that communicates the action, object, and scope with no filler or redundant information. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters, has an output schema, and has annotations covering its safety and idempotency, the description fully covers the necessary context. The one-sentence description is sufficient for a simple list operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has no parameters, so the empty schema covers all cases. Per scoring guidelines, a tool with zero parameters receives a baseline of 4 since there is nothing to describe.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'List' with a clear resource 'Zotero collections' and scope 'current library', making its action immediately apparent. It distinguishes itself from sibling tools like 'lookup' and 'search' by explicitly being a listing operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no explicit guidance on when to use this tool versus alternatives such as 'lookup' or 'search'. The use case is only implied by the name and action, but there is no stated relationship or exclusionary context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lookupA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and idempotentHint, indicating a safe, repeatable read operation. The description adds a critical behavioral constraint: the citekey must be a single token without spaces, which prevents common input errors. It does not describe error handling for missing items, but the output schema likely covers return values, so the added context goes beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the core purpose, and includes a helpful example. Every sentence adds value: the first defines the tool and input constraint, the second gives a usage priority directive. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple lookup tool with one parameter, read-only annotations, and an output schema, the description covers the essential aspects: what the tool does, the exact input format, and when to use it. The existence of an output schema handles return-value details, leaving no major gaps for this basic operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema only defines 'citekey' as a string with no description (0% coverage). The description fully compensates by explaining that it must be an exact citekey, a single token with no spaces, and provides a concrete example ('atz2022'), making the parameter unambiguous and actionable.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the verb 'look up' with the resource 'Zotero item' and the specific method 'by its exact citekey', clearly stating the tool's function. It distinguishes from siblings like 'search' by emphasizing the need for an exact citekey rather than fuzzy or broad search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The instruction 'Use this first when you know the citekey' provides a clear when-to-use directive, implying that when the citekey is unknown, other tools such as 'search' should be used. However, it does not explicitly name the alternative tool, making the guidance clear but not fully explicit about exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.