Server Quality Checklist

Tool Scores

• academy_lessonC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden but provides minimal behavioral insight. It implies a read operation ('Get'), but doesn't disclose authentication needs, rate limits, error handling, or response format (critical without an output schema). It mentions 'full content' but doesn't detail what that entails (e.g., structured data, HTML), leaving key behaviors opaque.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('Get full content of a specific lesson'). There is no wasted verbiage or redundancy, making it highly concise and well-structured for quick comprehension.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (3 parameters, no annotations, no output schema), the description is inadequate. It lacks details on authentication, error cases, response structure, and parameter semantics, leaving the agent with significant gaps. Without annotations or output schema, the description should provide more context to be complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is low (33%), with only 'slug' having a description. The tool description adds no parameter details beyond what's in the schema—it doesn't explain 'level' (e.g., course level), 'slug' usage, or 'locale' impact (e.g., language of content). Since it doesn't compensate for the coverage gap, it meets the baseline for minimal value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get full content') and resource ('a specific lesson'), making the purpose immediately understandable. It distinguishes from siblings like 'academy_lessons' (likely lists lessons) and 'academy_quiz' (quiz-related), though not explicitly. However, it doesn't specify what 'full content' includes (e.g., text, media, exercises), keeping it from a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a valid lesson ID), contrast with siblings like 'academy_lessons' (for listing) or 'academy_next_lesson' (for navigation), or specify use cases (e.g., for learning content vs. metadata). The description alone offers no usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• academy_lessonsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states it's a list operation, implying read-only and non-destructive behavior, but doesn't cover critical aspects like authentication needs, rate limits, pagination, error handling, or what 'completed status' entails. For a tool with zero annotation coverage, this leaves significant gaps in understanding how it behaves in practice.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action ('List all lessons within a level') and adds key detail ('with completed status'). There is no wasted language, and it directly communicates the tool's function without redundancy, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no output schema, no annotations), the description is insufficient. It lacks details on return values (e.g., list format, status indicators), error cases, or how it integrates with sibling tools. Without annotations or output schema, the description should provide more context to be complete, but it falls short, leaving the agent with incomplete operational knowledge.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 50%, with 'level' documented but 'locale' lacking a description in the schema. The description mentions 'within a level', aligning with the 'level' parameter, but adds no details on 'locale' (e.g., language selection impact). It partially compensates for the coverage gap by implying level usage, but doesn't fully explain parameter interactions or semantics, resulting in a baseline score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('lessons within a level'), specifying the scope as 'all lessons' with 'completed status'. It distinguishes from siblings like 'academy_lesson' (singular) and 'academy_levels' (levels rather than lessons), though not explicitly. However, it doesn't fully differentiate from 'academy_progress_complete' or 'academy_next_lesson', which might overlap in purpose, keeping it from a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention siblings like 'academy_lesson' for single lessons or 'academy_next_lesson' for progression, nor does it specify prerequisites or exclusions. Usage is implied by the action but lacks explicit context, scoring low due to the absence of comparative or contextual advice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• academy_next_lessonB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It describes the tool as a recommendation based on progress, implying it reads user data, but does not address permissions, rate limits, error conditions, or what happens if no incomplete lessons exist. For a tool with zero annotation coverage, this leaves significant gaps in understanding its behavior and constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded, consisting of two clear sentences that directly state the tool's purpose and behavior. There is no wasted language, and every sentence contributes essential information, making it efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (recommending based on progress), no annotations, no output schema, and one parameter with 0% schema description coverage, the description is minimally adequate. It explains what the tool does but lacks details on parameters, return values, and behavioral nuances. It meets the basic requirement but has clear gaps in context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has one parameter ('locale') with 0% description coverage in the schema itself. The tool description does not mention any parameters, so it adds no semantic information beyond what the schema provides. With one parameter and no output schema, the baseline is 3, as the schema handles the parameter documentation, but the description does not compensate for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Recommend the next lesson based on your progress.' It specifies the verb ('recommend') and resource ('next lesson'), and the additional detail about returning 'the first incomplete lesson in the lowest level you have access to' adds specificity. However, it does not explicitly differentiate from sibling tools like 'academy_lesson' or 'academy_lessons', which might also retrieve lesson information.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention sibling tools such as 'academy_lesson' (which might fetch a specific lesson) or 'academy_progress_complete' (which could update progress), leaving the agent to infer usage based on the purpose alone. There are no explicit when-to-use or when-not-to-use instructions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• academy_certificatesB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions that certificates are 'earned' and includes 'public verification URLs,' which implies read-only access to user-specific data. However, it doesn't address key behavioral aspects like authentication requirements, rate limits, error conditions, or whether the list is paginated/sorted. For a tool with zero annotation coverage, this leaves significant gaps in understanding how it behaves.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence: 'List your earned certificates with public verification URLs.' It is front-loaded with the core action and resource, and every word adds value (e.g., 'earned' specifies ownership, 'public verification URLs' adds useful detail). There is no wasted verbiage or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (simple list operation with 0 parameters) and lack of annotations/output schema, the description is minimally adequate. It explains what the tool does but misses behavioral details (e.g., auth, errors) and doesn't differentiate from siblings. For a read-only list tool, it provides the basic purpose but lacks completeness for optimal agent use without additional context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and schema description coverage is 100%, so there are no parameters to document. The description doesn't need to add parameter semantics, but it does provide context about what the tool returns ('earned certificates with public verification URLs'), which is helpful. Baseline for 0 parameters is 4, as the description compensates adequately by explaining the output.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'List your earned certificates with public verification URLs.' It specifies the verb ('List'), resource ('earned certificates'), and includes additional context about verification URLs. However, it doesn't explicitly differentiate from sibling tools like 'academy_stats' or 'academy_progress_complete' that might also provide certificate-related information.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There are multiple sibling tools (e.g., academy_stats, academy_progress_complete) that might overlap with certificate information, but the description doesn't mention any of them or specify contexts where this tool is preferred. It only states what the tool does, not when to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• academy_progress_completeB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses key behavioral traits: it's a mutation tool (implied by 'Mark as completed'), grants XP, updates streak, and schedules spaced-repetition review. However, it lacks details on permissions, error handling, or what happens if the lesson is already completed, leaving gaps in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core action and efficiently lists additional effects in a single, well-structured sentence. Every part adds value without redundancy, making it concise and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (a mutation tool with no annotations and no output schema), the description covers the main action and effects but is incomplete. It doesn't detail return values, error cases, or full parameter meanings, leaving the agent with insufficient context for robust invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It mentions 'levels you have access to', which loosely relates to the 'level' parameter, but doesn't explain 'slug' or 'locale' (e.g., what a slug represents or locale's impact). With 3 parameters and minimal semantic addition beyond the schema, it falls short of adequately clarifying usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Mark a lesson as completed') and resource ('lesson'), and distinguishes it from siblings by specifying it grants XP, updates streak, and schedules spaced-repetition review. However, it doesn't explicitly differentiate from tools like 'academy_quiz_submit' or 'academy_review_grade', which might also mark progress, so it's not a perfect 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides some context with 'Only works for levels you have access to', which implies a prerequisite, but it doesn't explicitly state when to use this tool versus alternatives like 'academy_lesson' or 'academy_next_lesson', nor does it give clear exclusions or comparisons to other progress-related tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• academy_review_gradeB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It describes the effect of grades ('again resets to 1d, good/easy grows the interval via SM-2'), which adds useful context beyond basic functionality. However, it lacks details on permissions, error handling, or response format, which are important for a mutation tool with no output schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise and front-loaded, consisting of two sentences that directly explain the tool's purpose and parameter effects. Every word earns its place, with no redundant or unnecessary information, making it highly efficient for an AI agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 2 parameters, no annotations, and no output schema, the description provides basic purpose and parameter effects, but it's incomplete for a mutation tool. It lacks details on return values, error cases, or broader context like how it integrates with other tools (e.g., 'academy_review'), leaving gaps that could hinder correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 50% (only 'id' has a description), and the description adds meaning for the 'grade' parameter by explaining the effects of each enum value ('again resets to 1d, good/easy grows the interval via SM-2'). This compensates partially for the low schema coverage, but it doesn't clarify the 'id' parameter beyond what's implied, so it meets the baseline for moderate compensation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Grade a review item after recall attempt.' It specifies the action (grade) and resource (review item), and distinguishes it from siblings like 'academy_review' or 'academy_quiz_submit' by focusing on grading after recall. However, it doesn't explicitly differentiate from all siblings (e.g., 'academy_progress_complete'), so it's not a perfect 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context ('after recall attempt'), suggesting this tool is used post-recall, but it doesn't provide explicit guidance on when to use it versus alternatives like 'academy_quiz_submit' or 'academy_review'. No exclusions or clear alternatives are mentioned, leaving some ambiguity for the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• academy_statsB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions the tool retrieves stats but doesn't disclose behavioral traits such as authentication needs, rate limits, or whether it's read-only. The phrase 'Get your Academy stats' implies a read operation, but this isn't explicitly confirmed, leaving gaps in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured in two sentences: the first states the purpose with specific examples, and the second provides usage guidance. Every sentence adds value without redundancy, making it efficient and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (0 parameters, no output schema, no annotations), the description is adequate but has gaps. It covers the purpose and basic usage but lacks details on behavioral aspects like authentication or output format, which are important for a stats retrieval tool without structured support.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description appropriately doesn't discuss parameters, focusing instead on the tool's purpose and usage, which aligns with the baseline for zero parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with the verb 'Get' and specifies the resource 'Academy stats' along with concrete examples (XP, rank, streak, etc.). It distinguishes from siblings by focusing on overall stats rather than specific components like certificates or lessons, though it doesn't explicitly name alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context with 'Call this first to see where you stand,' suggesting it's for initial assessment. However, it doesn't provide explicit guidance on when to use this versus sibling tools (e.g., academy_certificates for certificates only) or any exclusions, leaving some ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• academy_tutorA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'Pro-only' (implying access restrictions) and context awareness with parameters, but lacks details on rate limits, response format, error handling, or whether this is a read/write operation. For a tool with no annotations, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with only two sentences, both front-loaded with essential information: the core action and key usage constraints. Every word earns its place, with no redundant or vague phrasing. This efficiency makes it easy for an agent to quickly grasp the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and 3 parameters with 0% schema coverage, the description is minimally adequate. It covers the basic purpose and some usage context but lacks details on authentication needs, response format, error cases, or parameter examples. For a Pro-only interactive tool, this leaves room for improvement in completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains that 'level+lessonSlug' provides lesson context to the tutor, adding meaning beyond the schema's type constraints. However, it doesn't clarify the 'message' parameter's purpose or format, or provide examples. With 3 parameters and 0% schema coverage, this partial compensation results in a baseline score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Ask the Academy AI-Tutor a question.' It specifies the resource (Academy AI-Tutor) and action (ask a question), distinguishing it from sibling tools like academy_lesson or academy_quiz. However, it doesn't explicitly differentiate from academy_review or academy_quiz_submit, which might also involve asking questions in different contexts.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for usage: 'Pro-only' indicates a prerequisite, and 'The tutor knows the current lesson context if you provide level+lessonSlug' explains when to use specific parameters. It implies this tool is for interactive tutoring rather than content retrieval or assessment. However, it doesn't explicitly state when not to use it or name alternatives among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• academy_quizA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the tool fetches quizzes but lacks details on permissions, rate limits, error handling, or what the fetched data includes (e.g., quiz questions, answers). For a read operation with zero annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose and efficiently lists parameter usage options in a single, well-structured sentence. Every part contributes directly to understanding how to invoke the tool, with no wasted words or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (4 parameters, no output schema, no annotations), the description is adequate but incomplete. It covers parameter usage well but lacks behavioral details like response format, error cases, or dependencies on other tools. Without annotations or an output schema, more context on what 'fetch' entails would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is low at 25%, with only 'slug' and 'locale' having descriptions. The description compensates by explaining the semantic meaning of parameter combinations: 'slug' for direct access, 'level+lessonSlug' for lesson quizzes, and 'level' alone for checkpoints. This adds valuable context beyond the sparse schema, though it doesn't detail 'lessonSlug' or 'level' ranges beyond the schema's numeric constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Fetch a quiz') and specifies the resource types ('lesson quiz or end-of-level checkpoint'), making the purpose evident. However, it doesn't explicitly differentiate this tool from sibling tools like 'academy_quiz_submit' or 'academy_lesson', which might handle related but distinct operations, preventing a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use specific parameter combinations (e.g., 'slug' alone, 'level+lessonSlug' for lesson quizzes, or 'level' alone for checkpoints), offering practical guidance. However, it doesn't mention when not to use this tool or explicitly compare it to alternatives like 'academy_lesson' or 'academy_quiz_submit', which could help avoid misuse.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• academy_quiz_submitA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses key behavioral traits: it returns score, pass status, XP awarded, per-question correctness, and explanations, and mentions certificate issuance under specific conditions. However, it lacks details on error handling, authentication needs, rate limits, or side effects like data persistence, which are important for a submission tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core action and outcomes, using two efficient sentences with zero waste. Every sentence adds value: the first explains the submission and returns, the second adds conditional certificate issuance. It is appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (submission with outcomes and conditional certificate), no annotations, and no output schema, the description does a good job covering the return values and a key behavioral aspect. However, it could improve by mentioning error cases or response structure more explicitly, but it is largely complete for guiding usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters (slug and answers) adequately. The description does not add any additional meaning or context beyond what the schema provides, such as format examples for slug or validation rules for answers. Baseline 3 is appropriate as the schema handles the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Submit answers to a quiz') and resource ('a quiz'), distinguishing it from sibling tools like academy_quiz (likely for viewing) or academy_review_grade (likely for grading reviews). It explicitly mentions the outcome of submission, which helps differentiate its purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by mentioning 'If checkpoint and passed → certificate issued,' which suggests context for when a certificate might be generated, but it does not explicitly state when to use this tool versus alternatives like academy_quiz or academy_review_grade. No clear exclusions or prerequisites are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• academy_reviewA
  3.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states the tool lists items, implying a read-only operation, but does not disclose behavioral traits such as authentication requirements, rate limits, pagination, or what happens if no items are due. For a tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with zero waste. It is front-loaded with the core action and efficiently conveys the purpose without unnecessary details, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (0 parameters, no output schema, no annotations), the description is minimally adequate. It explains what the tool does but lacks details on behavioral aspects like return format or error handling. Without annotations or output schema, the description should ideally provide more context, but it meets basic requirements for a simple listing tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and schema description coverage is 100%, so no parameter documentation is needed. The description does not add parameter semantics, but this is acceptable given the lack of parameters. A baseline of 4 is appropriate as the description does not need to compensate for any parameter gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('List') and resource ('spaced-repetition items due for review today'), distinguishing it from siblings like academy_lessons (general listing) or academy_review_grade (grading reviews). It precisely defines the scope to items due today, which is more specific than a generic listing tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by specifying 'due for review today,' suggesting this tool is for daily review tasks. However, it does not explicitly state when not to use it or name alternatives (e.g., academy_lessons for general items or academy_review_grade for grading), so it lacks full exclusion guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• academy_levelsA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the tool lists levels with access types and shows current access status and progress, which is useful behavioral context. However, it doesn't mention potential limitations like rate limits, authentication requirements, or whether the data is cached/real-time.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that efficiently conveys the tool's purpose and key features. Every word earns its place, with no redundant or vague language.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no parameters, no annotations, and no output schema, the description provides adequate context about what information is returned (levels with access types and current status). However, it doesn't explain the format of the returned data or potential error conditions, leaving some gaps in completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description appropriately focuses on what the tool does rather than parameters. A baseline of 4 is applied since there are no parameters to document.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('List all 6 levels') and resource ('academy levels'), including what information is provided ('with access info (free / paid / earned)'). It distinguishes this tool from siblings by focusing on levels rather than certificates, lessons, quizzes, or progress tracking.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: to see all academy levels and your current access status. However, it doesn't explicitly state when NOT to use it or name specific alternatives among the sibling tools (e.g., when to use academy_lessons vs. academy_levels).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.