Server Quality Checklist

Tool Scores

• find-notesC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool 'finds notes matching a query', which implies a read-only search operation, but doesn't disclose critical details like whether it returns all matches or paginates, what format the results are in, error conditions, or performance characteristics. This is a significant gap for a search tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's appropriately sized and front-loaded, stating the core purpose immediately without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a search operation with one parameter), lack of annotations, no output schema, and low schema coverage, the description is incomplete. It doesn't explain what 'notes' are in Anki context, how results are returned, or any behavioral traits, making it inadequate for an agent to use the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, with one parameter 'query' undocumented in the schema. The description mentions 'matching a query' but doesn't add any meaning beyond the parameter name—it doesn't explain query syntax, examples, or constraints. With low schema coverage, the description fails to compensate, leaving the parameter semantics unclear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('find notes matching a query') and resource ('in Anki'), providing specific verb+resource. However, it doesn't distinguish from sibling tools like 'get-cards-reviewed' or 'get-collection-overview', which might also retrieve note-related information, so it doesn't fully differentiate from alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'add-or-update-notes' or 'get-cards-reviewed'. It lacks context about use cases, prerequisites, or exclusions, leaving the agent to infer usage based on the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add-or-update-notesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions adding or updating notes but fails to explain critical behaviors: whether this requires specific permissions, if updates are destructive or reversible, rate limits, or what happens on partial failures. This leaves significant gaps for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It is appropriately sized and front-loaded, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a mutation tool with 1 parameter (but nested objects in schema), no annotations, no output schema, and 0% schema description coverage, the description is incomplete. It lacks details on behavior, parameters, error handling, and output, making it insufficient for safe and effective use by an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for undocumented parameters. However, it adds no parameter-specific information beyond the tool's name implying 'notes' as input. The schema details parameters like 'notes', 'id', 'fields', etc., but the description doesn't explain their semantics or usage, resulting in a baseline score due to inadequate compensation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Add new notes or update existing ones') and the target resource ('in Anki'), which provides a specific verb+resource combination. However, it doesn't differentiate from sibling tools like 'find-notes' beyond the basic add/update vs. find distinction, missing explicit comparison.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description lacks context about prerequisites, when to choose this over other tools, or any exclusions. It merely states what the tool does without usage instructions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-cards-reviewedB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool retrieves aggregated data ('number of cards reviewed by day'), implying a read-only operation, but doesn't specify behavioral traits like whether it requires authentication, has rate limits, returns historical data only, or handles errors. For a tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence: 'Get the number of cards reviewed by day'. It is front-loaded with the core purpose, has zero waste, and is appropriately sized for a simple tool with no parameters. Every word earns its place by specifying the action, resource, and temporal aspect.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (simple retrieval with 0 parameters), no annotations, and no output schema, the description is minimally adequate. It states what the tool does but lacks context on usage, behavioral traits, or return values. Without an output schema, the description doesn't explain what data is returned (e.g., format, date range), leaving gaps for the agent to infer.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% description coverage, meaning no parameters are documented in the schema. The description doesn't add parameter details, but since there are no parameters, this is acceptable. The baseline for 0 parameters is 4, as the description needn't compensate for missing parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get the number of cards reviewed by day' specifies the verb (get), resource (cards reviewed), and temporal scope (by day). It distinguishes from sibling tools like 'add-or-update-notes' or 'find-notes' by focusing on aggregated review metrics rather than note manipulation or search. However, it doesn't explicitly differentiate from 'get-collection-overview', which might also provide statistical data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, such as whether user authentication or specific data access is required, nor does it compare to sibling tools like 'get-collection-overview' that might offer overlapping functionality. Usage is implied through the purpose statement alone, leaving the agent to infer context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-collection-overviewB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only states it 'gets' information without disclosing behavioral traits like permissions needed, rate limits, response format, or whether it's a heavy operation. It mentions 'comprehensive information' but doesn't clarify depth or structure, leaving gaps in transparency for a tool with potential complexity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the purpose ('Get comprehensive information') and specifies key components without waste. Every word contributes to understanding the tool's function, making it appropriately sized and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 0 parameters and no output schema, the description adequately covers the purpose and scope. However, with no annotations and potential complexity in 'comprehensive information', it lacks details on behavioral aspects like response format or performance, making it minimally viable but incomplete for full agent guidance.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so no parameter documentation is needed. The description adds value by specifying what information is retrieved ('decks, models, and fields'), which goes beyond the empty schema, earning a baseline score above 3 for clarity in output scope.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('comprehensive information about the Anki collection'), specifying the scope includes 'decks, models, and fields'. It distinguishes this as a read operation from sibling tools like 'add-or-update-notes' (write) and 'find-notes' (search), though it doesn't explicitly name alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving collection metadata, which contrasts with sibling tools focused on notes or cards. However, it lacks explicit guidance on when to use this tool versus alternatives (e.g., for overview vs. detailed queries) or any prerequisites, leaving usage context inferred rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.