Server Quality Checklist

Tool Scores

• create_deckD
  1.3/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description claims the tool is read-only and has no side effects, which directly contradicts the act of creating a deck (which likely modifies state). No annotations are provided, so the description must bear the burden, but it fails by being internally contradictory.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness1/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is overly long, repetitive (e.g., 'Behavioral Transparency' appears twice), and poorly structured, with irrelevant sections that bury the functional purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema (not shown), the description fails to clarify the tool's actual behavior for creating a deck, leaving critical details undocumented and the agent likely to misuse the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must add meaning, but the 'Args' section uses generic phrases like 'analyze or process' for all parameters, adding no value and even conflicting with the initial use case.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with 'Create a new flashcard deck' which is clear, but then diverges into generic analysis/classification language that contradicts the tool's name and purpose. This confusion undermines the purpose clarity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'When to use' and 'When NOT to use' sections are generic and irrelevant to deck creation, providing misleading guidance that does not help differentiate from sibling tools like add_card or quiz_session.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_cardD
  1.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. The description claims read-only behavior and no side effects, directly contradicting the tool's mutative nature. While it mentions rate limits and authentication, the core behavioral disclosure is incorrect.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is overly long with redundant sections (e.g., behavior repeated). The first sentence is concise, but the rest is verbose and includes irrelevant boilerplate, making it poorly structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is relatively simple but the description fails to provide complete context. The output schema exists, but the description's boilerplate about analysis is off-topic and doesn't cover the actual flashcard creation behavior. Key information like idempotency of card addition or error handling for duplicate cards is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description must compensate. For each parameter, it only provides the generic phrase 'to analyze or process', adding no meaningful detail about their purpose, valid values, or format.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The first sentence clearly states the tool adds a flashcard to a deck and creates the deck if needed. However, the subsequent description contradicts this by claiming the tool is read-only and generates structured analysis output, creating ambiguity about the actual purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool vs siblings. The 'When to use' section is misleading, suggesting it's for structured analysis/classification, which is irrelevant to flashcard addition.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• quiz_sessionC
  2.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description claims the tool is read-only and stateless, but starting a quiz session and returning cards due for review based on SM-2 scheduling implies state dependence. This internal contradiction undermines transparency. Rate limits and error handling are provided but are generic and not verified for this specific tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is overly long with redundant sections (e.g., repeating 'Behavioral Transparency' in a separate block). The first sentence is concise, but the rest dilutes clarity with generic text that could be omitted.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema, the description fails to explain core behavior like what 'starting a session' means, how scheduling works, or how the tool interacts with the deck. The generic sections do not add value and leave important gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema coverage, the description should compensate but uses generic phrases like 'The deck name to analyze or process'. Only the mode parameter benefits from the description listing values (standard, reverse, mixed). Other parameters lack meaningful context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The first sentence clearly states the tool starts a quiz session from a deck with modes and returns cards due for review. However, the description then pivots to generic language about 'structured analysis or classification', which obfuscates the tool's actual purpose and may mislead an AI agent.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'When to use' and 'When NOT to use' sections are generic and irrelevant to quiz sessions, suggesting use for 'structured analysis' rather than quizzing. No guidance is provided on when to choose this tool over siblings like add_card or get_stats.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_statsA
  3.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Despite lacking annotations, the description thoroughly covers side effects (read-only, stateless, idempotent), authentication (none for basic, API key for pro), rate limits (10/day free, unlimited pro), error handling (structured errors), and data privacy (no storage). This fully compensates for missing annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (Behavior, When to use, Args, etc.) and front-loaded with the key purpose. It is thorough but somewhat verbose, with some repetition (e.g., behavioral transparency repeated in two sections). Efficient overall.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While behavioral aspects are well-covered, the description does not explain how the 'get statistics' and 'record results' functions interact, which may confuse. The mention of 'structured analysis or classification' seems broader than deck statistics. The presence of an output schema partially compensates for missing return value details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must add meaning. It provides useful format info for record_results (JSON array of {card_id, rating}), but deck_name and api_key are described with generic phrases like 'to analyze or process', adding no value. The added info is minimal.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Get deck statistics and optionally record quiz results', which clearly identifies the tool's primary function and secondary capability. It is specific and distinguishes from sibling tools like add_card and create_deck, but the dual purpose introduces slight ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes explicit 'When to use' and 'When NOT to use' sections, providing clear context for appropriate usage. It advises against real-time production decisions without human review, but does not directly compare to sibling tools like quiz_session.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.