Server Quality Checklist

Tool Scores

• list_decksB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states the tool lists decks but doesn't describe any behavioral traits such as whether it requires authentication, has rate limits, returns structured data, or handles errors. This leaves significant gaps for a tool that presumably interacts with Anki.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without any wasted words. It's appropriately sized for a simple tool and front-loaded with essential information, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, output schema exists), the description is minimally adequate but lacks completeness. It doesn't address behavioral aspects like authentication needs or return format, which are important even with an output schema. The description meets basic requirements but leaves contextual gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so the schema fully documents the lack of inputs. The description doesn't need to add parameter semantics, and it appropriately doesn't mention any parameters, earning a baseline score of 4 for this context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List') and resource ('all available Anki decks'), providing a specific verb+resource combination. However, it doesn't distinguish this tool from potential sibling alternatives like 'get_deck_stats' or 'search_cards' that might also involve deck retrieval, which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when this tool is appropriate (e.g., for getting a simple list vs. detailed stats) or when to prefer other tools like 'get_deck_stats' or 'search_cards', leaving usage context entirely implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_deck_statsA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool returns a dictionary with specific counts (new_count, learn_count, review_count), which adds value beyond the input schema. However, it lacks details on error handling (e.g., if deck doesn't exist), performance (e.g., speed, rate limits), or side effects (e.g., read-only nature implied but not stated). The description doesn't contradict annotations, but it's incomplete for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and appropriately sized, with a clear purpose statement followed by Args and Returns sections. Every sentence earns its place by providing essential information without redundancy. It's front-loaded with the main function and efficiently details inputs and outputs, making it easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no nested objects) and the presence of an output schema (implied by 'Returns' details), the description is mostly complete. It covers purpose, parameter semantics, and return values adequately. However, with no annotations, it could benefit from more behavioral context (e.g., read-only confirmation, error cases). The output schema reduces the need for detailed return explanations, but gaps in usage guidelines and transparency slightly limit completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful context for the single parameter 'deck_name' by specifying it as 'Name of the deck to get stats for', which clarifies its purpose beyond the schema's basic type and default. With 0% schema description coverage and only one parameter, this compensation is effective, though it could note the default value 'Default' from the schema. The baseline would be 3 for high schema coverage, but here the description enhances understanding significantly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get statistics') and resource ('for a deck'), including the key metric 'due cards count'. It distinguishes itself from siblings like 'list_decks' or 'search_cards' by focusing on statistical data rather than listing or searching. However, it doesn't explicitly differentiate from all siblings, such as how it relates to 'anki_connection_status' in terms of data retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., deck must exist), compare to siblings like 'list_decks' for deck existence checks, or specify contexts (e.g., for review planning vs. general stats). Usage is implied only by the purpose, with no explicit when/when-not statements.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sync_ankiA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the tool triggers a sync operation and mentions the outcome (cards available on mobile), but lacks details on behavioral traits like whether it's idempotent, requires authentication, has rate limits, or potential side effects (e.g., data conflicts). It adds some context but is incomplete for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured with two sentences: the first states the action, and the second explains the outcome. Every sentence adds value without waste, making it efficient and front-loaded for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters, no annotations, and no output schema, the description provides basic purpose and outcome. However, for a sync operation that likely involves mutation and network calls, it lacks completeness in areas like error handling, return values, or dependencies, leaving gaps in contextual understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description appropriately doesn't discuss parameters, and since there are none, it doesn't need to compensate. A baseline of 4 is given as it avoids unnecessary details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Trigger Anki to sync') and resource ('with AnkiWeb'), explaining it synchronizes local collections with the cloud. However, it doesn't explicitly differentiate from sibling tools like 'anki_connection_status' which might relate to sync status, leaving room for improvement in sibling distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by stating the sync makes cards available on mobile devices, suggesting when to use it (e.g., after adding cards). However, it lacks explicit guidance on when to use this vs. alternatives like 'anki_connection_status' or prerequisites (e.g., internet connection), making it only moderately helpful.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_flashcardsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses key behaviors: batch processing, default deck handling, optional tags, and duplicate handling (returns None for duplicates). However, it misses details like error handling, rate limits, or authentication needs, which are important for a write operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a purpose statement, parameter explanations, return value, and example. It's appropriately sized, though the example is detailed but necessary for clarity. Every sentence adds value, but it could be slightly more front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, 0% schema coverage, and an output schema (implied by 'Returns'), the description does well by covering parameters, behavior, and output. However, it lacks context on error cases or integration with siblings like 'sync_anki', leaving minor gaps for a mutation tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate fully. It clearly explains all three parameters: 'cards' as a list of dicts with 'front' and 'back' keys, 'deck_name' with its default, and 'tags' as optional. This adds essential meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Add multiple flashcards'), target resource ('to Anki'), and scope ('in a single operation'). It distinguishes from the sibling 'add_flashcard' by emphasizing batch processing versus single-card addition.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for batch operations with the phrase 'in a single operation,' which differentiates it from 'add_flashcard' for single cards. However, it lacks explicit guidance on when to use this tool versus alternatives like 'create_deck' or prerequisites such as requiring an active Anki connection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• anki_connection_statusA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool checks connection status and returns version info, which implies a read-only, non-destructive operation, but does not detail error handling, timeouts, or authentication needs. It adds basic context but lacks rich behavioral traits like rate limits or response formats beyond the return statement.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose in the first sentence and efficiently adds return details in the second. Both sentences earn their place by providing essential information without redundancy, making it appropriately sized and zero waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (0 parameters, no annotations, but has an output schema), the description is mostly complete. It covers the purpose and return values, though it could benefit from more behavioral context (e.g., error cases). The output schema existence reduces the need for detailed return explanations, but the description still adds useful semantics.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description does not mention parameters, which is appropriate, and adds value by explaining the return type (Dict with connection status and version), compensating for the lack of output schema details in the context signals.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Check if AnkiConnect is responding' and 'get version info'), distinguishing it from siblings like add_flashcard or sync_anki that perform different operations. It explicitly identifies the resource (AnkiConnect) and the action (checking connection status and retrieving version information).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context (verifying AnkiConnect availability before other operations) but does not explicitly state when to use it versus alternatives or provide exclusions. It suggests a diagnostic purpose, which helps differentiate from siblings like list_decks or search_cards, but lacks explicit guidance on prerequisites or timing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_cardsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds useful context by detailing the return format ('List of cards with their fields and info') and providing search syntax examples, which aids in understanding how the tool behaves. However, it lacks information on potential limitations like rate limits, error handling, or authentication needs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, starting with the core purpose and followed by structured sections for args, returns, and examples. Every sentence adds value, such as the search operators list, though it could be slightly more concise by integrating examples more tightly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no annotations, but with an output schema), the description is largely complete. It covers purpose, parameters, returns, and usage examples. The output schema likely handles return value details, so the description's focus on search syntax and examples is sufficient, though it could benefit from mentioning any prerequisites or constraints.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds significant meaning beyond the input schema, which has 0% description coverage. It explains the 'query' parameter with examples (e.g., 'deck:Default', 'tag:python') and common search operators, and clarifies the 'limit' parameter's purpose ('Maximum number of results to return'). This fully compensates for the schema's lack of descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Search for cards in Anki') and resource ('cards'), using Anki's search syntax. It distinguishes this tool from siblings like 'add_flashcard' (creation), 'list_decks' (deck listing), and 'get_deck_stats' (statistics) by focusing on card retrieval through queries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for usage by explaining that it searches 'using Anki's search syntax' and includes common search operators, which helps users understand when to apply this tool. However, it does not explicitly state when not to use it or name alternatives among sibling tools, such as 'list_decks' for deck-level queries.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_deckA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool creates a new deck and returns a deck ID, which covers the basic operation and output. However, it lacks details on permissions, error handling, or side effects (e.g., impact on existing decks).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, followed by clear sections for arguments and returns. Each sentence is necessary and informative, with no wasted words, making it efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (single parameter, no annotations, but with an output schema), the description is mostly complete. It explains the parameter and return value well, but could benefit from more behavioral context (e.g., error cases). The output schema likely covers return details, reducing the need for extensive output explanation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must compensate. It fully explains the single parameter 'deck_name', including its purpose, format, and an example ('Parent::Child' for nested decks). This adds significant meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Create a new deck') and resource ('in Anki'), distinguishing it from sibling tools like 'list_decks' or 'get_deck_stats'. It precisely defines the tool's function without ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a new deck is needed in Anki, but it does not explicitly state when to use this tool versus alternatives like creating nested decks through other means or handling existing decks. It provides context but lacks explicit exclusions or comparisons to siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_flashcardA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does well by disclosing key behaviors: it creates a new card, supports HTML/markdown formatting, handles duplicates (returns None if duplicate), and specifies default values. It doesn't mention permissions, rate limits, or error conditions, but covers essential operational traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with the core purpose, followed by organized sections for Args and Returns. Every sentence adds value without redundancy, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (mutation tool with 4 parameters), no annotations, and an output schema present (covering return values), the description is complete. It explains the tool's purpose, parameters, behavior, and output adequately, with no significant gaps for an AI agent to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate fully. It adds significant meaning beyond the schema by explaining each parameter's purpose (e.g., 'front' as question side, 'back' as answer side), format support (HTML/markdown), defaults, and optionality. This comprehensively documents all 4 parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Add a new flashcard') and resource ('to Anki'), distinguishing it from siblings like 'add_flashcards' (bulk operation) and 'create_deck' (different resource). It uses precise verbs and identifies the exact function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by specifying the tool's purpose, but does not explicitly state when to use it versus alternatives like 'add_flashcards' for bulk operations or 'create_deck' for deck management. No exclusions or clear contextual boundaries are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.