Server Quality Checklist

Tool Scores

• quick_configC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It says 'switch' but gives no indication of side effects, persistence behavior, or the powerful destructive reset_progress option that wipes XP, level, streak, and badges. The schema mentions reset_progress, but the tool description itself does not warn about the impact.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short and front-loaded, but it is more under-specified than genuinely concise. Words like 'Instantly' and 'simple' add little information, and a single vague sentence is not enough for a 7-parameter configuration tool with no annotations and no output schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 7 optional parameters, no annotations, and no output schema, the description is incomplete. It fails to mention the destructive reset behavior, the voice and words_per_minute settings, what happens if multiple parameters are combined, or what response the agent can expect after the call.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 43%, so the description should compensate. It maps a few natural-language labels to parameters (skill_level, category, roadmap_name, current_topic), but it omits voice_enabled, words_per_minute, and the critical reset_progress flag. It also adds no detail about valid values for roadmap_name or current_topic beyond naming them.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific action ('switch') and identifies several configurable targets: skill level, roadmap category, roadmap track, and topic. This is much more informative than a tautology, but it does not explicitly distinguish quick_config from siblings like set_active_roadmap or configure_voice, so it stops short of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus the sibling tools. It mentions switching several settings but never says 'use this instead of set_active_roadmap or configure_voice when changing multiple config values at once.' The intended usage is implied, but not spelled out.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_active_roadmapC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden of disclosing side effects. It states what is configured but does not mention that this likely changes persistent/global active state, whether prior values are overwritten, or what the result/response of the call is.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single efficient sentence with the action and resource front-loaded. Every phrase contributes meaning: the resource, the governed fields, and the counter semantics. There is no filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a state-setting tool with five parameters, no annotations, and no output schema, so it needs more context to be safely invoked. Missing side effects, usage timing, and clearer parameter relationships leave important gaps for an agent deciding whether and how to call it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must compensate, and it partially does by grouping step_index and total_steps as 'progress step counters' and interpreting current_topic as 'current topic node.' However, it does not explain individual parameter meaning, constraints, or how the counters relate to the roadmap state.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Configure') with a clear resource ('the active roadmap') and lists the key fields involved. This makes the tool's purpose understandable and distinguishes it from retrieval-oriented siblings like get_next_roadmap_command, though it does not explicitly name a sibling alternative.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no guidance on when to choose this tool over alternatives such as quick_config or get_next_roadmap_command. There are no prerequisites, exclusions, or contextual signals explaining the intended workflow placement.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_next_roadmap_commandB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits clearly. It does disclose the optional side effect ('Optionally advance the step counter') and implies the command is not executed directly ('copy-pasteable'). However, it does not elaborate on other state changes, error conditions (e.g., missing active roadmap), or the nature of the return value beyond 'suggest a command'. Basic transparency is present but not thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no wordiness. The primary purpose is front-loaded in the first sentence, and the brief second sentence covers the optional parameter. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one optional parameter and no output schema, the description covers the main function and side-effect clearly. It does not mention behavior when no active milestone exists, which is a minor gap, but the description is sufficiently complete for typical use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and the schema's parameter description ('Advance step_index by one before suggesting.') fully documents the 'advance' parameter. The tool description adds only a synonym ('step counter') without new meaning, so it neither compensates for nor expands upon the schema. Baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb ('Suggest') and resource ('the next copy-pasteable terminal command for the active roadmap milestone'). It is easy to understand the tool's core purpose, but it does not explicitly distinguish itself from sibling tools like run_teaching_command or verify_quiz_answer, so it lacks overt sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It mentions the context ('for the active roadmap milestone') but does not state exclusions, prerequisites, or how it relates to sibling tools such as run_teaching_command. This leaves the agent to infer usage independently.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• configure_voiceA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the full burden. It says the tool toggles audio and adjusts speech rate, but it does not disclose whether settings persist, whether permissions are needed, or what side effects occur. The test_phrase behavior is only in the schema, not the description.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One efficient sentence with no fluff. The core actions are front-loaded and every word contributes meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple, low-complexity configuration tool with zero required parameters and no output schema, the description is mostly sufficient. But it lacks context about persistence, prerequisites, or the full role of test_phrase, and no output schema means the agent is given no clue about the result of calling the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 33%, but the description compensates partially by mapping 'toggle on/off' to enabled and 'speech rate in WPM' to words_per_minute. test_phrase is covered by the schema's own description. The added meaning is useful but modest.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Toggle'/'adjust') and names the exact resource ('tutor audio' and 'speech rate'). This clearly separates it from siblings like quick_config and run_teaching_command, which are not voice-specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied: use this when the user wants to enable/disable tutor audio or change speech rate. However, there is no explicit 'when not to use' statement or mention of alternatives like quick_config, leaving some ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• export_roadmap_notesA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does say 'Write', which implies a file mutation, but it does not disclose that the default behavior overwrites an existing ROADMAP_PROGRESS.md, that the file is written to the server's cwd, or any other side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single focused sentence with no filler words. It front-loads the primary verb and deliverable, then specifies the content requirements. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with two optional, fully documented parameters and no output schema, so the description covers the core purpose well. However, it omits important behavioral context such as the overwrite-by-default behavior and appropriate invocation timing, leaving mild gaps for an agent deciding when and how to call it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so both parameters ('append' and 'output_path') are already documented in the schema. The description does not add parameter-specific meaning beyond the schema, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Write'), a specific deliverable ('clean ROADMAP_PROGRESS.md summary'), and the exact content to include ('roadmap position, player stats, and every concept and command covered'). This clearly differentiates it from the sibling tools, none of which are export/summary tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'summary of the session' implies it should be used after a teaching session to persist progress, but there is no explicit guidance on when to invoke it versus alternatives or whether it should be run at the end of every session. Usage is inferred rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_teaching_commandA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full behavioral burden. It usefully discloses that errors return a Tutor Hotfix Diagnostic instead of throwing, and that dry-run is possible. However, it does not mention side effects of executing commands, the behavior of the is_dangerous flag, or any safety caveats, which are important for a command-execution tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no filler. The main action is front-loaded and the output components are listed compactly. It is slightly long due to the enumerative output list, but every listed item adds meaningful context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description explains what the tool returns and how errors behave, which is good for a tool with no output schema or annotations. However, it lacks guidance on safety, dry-run usage trade-offs, and how optional parameters like cwd or concept affect behavior, leaving some practical gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all five parameters. The description adds no extra parameter-level detail beyond mentioning dry-run in prose, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb-resource pair ('Execute (or dry-run) a shell command') and enumerates the full teaching card contents, making the tool's purpose unmistakable. It is clearly differentiated from siblings like verify_quiz_answer or get_next_roadmap_command, which handle different tasks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies when to use the tool: whenever a shell command needs to be taught with explanation and practice materials. It does not explicitly name alternatives or state exclusions, so it stops short of a perfect score, but the context is clear enough for an agent to select it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_quiz_answerA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It discloses the side effects: updating streak, XP, and badges, plus speaking feedback. This gives the agent a clear sense of what will change, though it does not mention reversibility or any prerequisites.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one concise, information-dense sentence. It includes the action, target, and side effects without any filler or repetition of the tool name.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with no nested objects and no output schema, the description covers the essential action and effects. It could optionally mention that a quiz must be currently pending, but the phrase 'the most recent active-recall quiz' provides enough contextual framing for correct use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already covers the single parameter with 100% description coverage, explaining that 'answer' is either a letter (A-D) or answer text. The tool description adds no additional parameter meaning beyond what the schema provides, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Evaluate') and names the precise resource ('the learner's answer to the most recent active-recall quiz'). It also lists the tool's effects (streak, XP, badges, feedback), which clearly distinguishes it from the unrelated sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies the tool should be used when a learner provides an answer to the most recent active-recall quiz. It does not explicitly name alternatives or exclusion conditions, but the context is clear and the sibling tools are not overlapping in function.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_user_statsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of indicating behavior. 'Return the current player profile' clearly signals a read-only retrieval with no apparent side effects. It does not detail voice behavior for speak=true, but that is covered by the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, information-dense sentence that front-loads the core purpose and then lists return fields. Every word earns its place with no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter read-only stats tool, the description is complete: it states what is returned, and the schema fully documents the optional speak parameter. The agent has enough information to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and the only parameter 'speak' is fully documented with type, default, and description. The tool description adds no parameter-specific meaning, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource: 'Return the current player profile' and enumerates exactly what is included (XP, level, title, quiz streak, unlocked badges, roadmap progress). This clearly differentiates the tool from siblings like verify_quiz_answer or set_active_roadmap.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: this tool is used when the current player profile or stats are needed. It does not explicitly discuss when not to use it or name alternatives, but the use case is obvious from the description and sibling context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.