Server Quality Checklist

Tool Scores

• get_due_cardsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states this is a read operation ('Returns'), but doesn't mention any behavioral traits such as permissions required, rate limits, pagination, or what happens if 'num' exceeds available due cards. The description is minimal and lacks crucial context for safe invocation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's function without unnecessary words. It's appropriately sized for a simple tool with one parameter, though it could be slightly more structured by separating purpose from parameter context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It doesn't explain what 'due for review' means, how cards are selected or ordered, what the return format looks like, or any error conditions. For a tool that presumably interacts with a review system, this leaves significant gaps in understanding its behavior and output.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with the parameter 'num' fully documented in the schema. The description adds marginal value by reinforcing that 'num' represents 'Number of due cards to get', but doesn't provide additional semantics like valid ranges, default behavior, or edge cases beyond what the schema already states.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Returns') and resource ('cards due for review'), and includes the parameter scope ('a given number (num)'). However, it doesn't explicitly distinguish this from sibling tools like 'get_new_cards', which suggests a different type of card retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_new_cards' or 'update_cards'. It mentions 'due for review' but doesn't clarify what constitutes 'due' cards or when this tool is appropriate compared to other card-related operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_new_cardsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool returns cards but doesn't describe what 'new and unseen' means operationally, whether there are rate limits, authentication requirements, or how the tool handles edge cases (e.g., if 'num' exceeds available cards). The description is minimal and lacks critical behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It front-loads the core purpose ('Returns...') and includes only essential details. Every word earns its place, making it highly concise and well-structured for quick comprehension.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (1 parameter, no annotations, no output schema), the description is incomplete. It lacks context about what constitutes 'new and unseen', how results are formatted, or any behavioral traits. For a read operation with no structured support, the description should provide more operational guidance to compensate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the parameter 'num' documented as 'Number of new cards to get'. The description adds no additional meaning beyond this, merely restating that it returns 'a given number (num)'. Since the schema already fully describes the parameter, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Returns') and resource ('new and unseen cards'), with specific scope ('a given number'). It distinguishes from siblings like 'get_due_cards' by focusing on new/unseen cards rather than due ones. However, it doesn't explicitly differentiate from 'add_card' or 'update_cards' beyond the verb choice.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_due_cards' or 'add_card'. It implies usage for retrieving new cards but offers no context about prerequisites, exclusions, or typical scenarios. This leaves the agent without clear decision-making criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_cardsB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It describes the action ('mark them answered and update their ease'), implying a mutation operation, but doesn't disclose critical traits like whether this requires specific permissions, if changes are reversible, potential side effects, or any rate limits. For a mutation tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, consisting of a single, efficient sentence that directly states the tool's purpose and usage context. Every word earns its place, with no redundant or unnecessary information, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity as a mutation operation with no annotations, 1 parameter with 0% schema coverage, and no output schema, the description is incomplete. It covers the basic purpose but lacks details on behavioral traits, parameter semantics, and return values, which are essential for safe and effective use. This is inadequate for a tool that modifies data.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 1 parameter ('answers') with 0% description coverage, and the description doesn't add any meaning beyond what the schema provides. It mentions 'cards' and 'ease' but doesn't explain the parameter structure, such as what 'answers' array contains or how 'ease' values map to actions. With low schema coverage, the description fails to compensate, leaving parameters poorly documented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('mark them answered' and 'update their ease') and identifies the resource ('cards'). It distinguishes from siblings like 'add_card', 'get_due_cards', and 'get_new_cards' by focusing on post-quiz updates rather than creation or retrieval. However, it doesn't explicitly name the resource type beyond 'cards', which could be slightly more specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implied usage context by stating 'After the user answers cards you've quizzed them on', which suggests when to use this tool. It doesn't explicitly mention when not to use it or name alternatives, such as whether other tools handle similar updates. The guidance is clear but lacks explicit exclusions or comparisons to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_cardA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the tool creates a new flashcard (implying mutation) and specifies critical formatting rules, but does not cover other behavioral aspects such as permissions, error handling, or what happens on success/failure. It adds some context but is incomplete for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, followed by detailed formatting rules in a structured list. It is appropriately sized, but the formatting rules section is somewhat lengthy; every sentence earns its place by clarifying critical constraints, though it could be more streamlined.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a mutation tool with no annotations and no output schema), the description covers the purpose and formatting rules well but lacks information on behavioral traits like response format, error conditions, or prerequisites. It is adequate but has clear gaps for a creation tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents both parameters (front and back) with descriptions. The description adds value by emphasizing HTML formatting rules that apply to these parameters, but does not provide additional semantic details beyond what the schema offers, aligning with the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Create a new flashcard in Anki') and identifies the resource ('flashcard'), distinguishing it from sibling tools like get_due_cards, get_new_cards, and update_cards which are retrieval or modification operations rather than creation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by specifying 'for the user' and formatting requirements, but does not explicitly state when to use this tool versus alternatives like update_cards or when not to use it. It provides context but lacks clear exclusions or direct sibling comparisons.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.