Server Quality Checklist

Tool Scores

• statusA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the full transparency burden. It implies a read-only operation ('Where this topic stands') and adds a behavioral guideline about not reading due dates, but it does not explicitly confirm that no state is modified or describe error handling. The presentation instruction is helpful, but side-effect disclosure remains implicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded. It opens with the key purpose, then offers usage guidance, a presentation tip, and the parameter note—all without superfluous words. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple status tool with one parameter, the description covers purpose, usage timing, presentation style, and parameter source. It also lists the output contents (phase, mastery counts, locks) despite lacking an output schema. It omits edge cases like invalid session_id, but overall it is sufficiently complete for the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has zero description coverage for the single parameter 'session_id'. The description compensates by stating 'From start_topic,' which tells the agent exactly where to obtain the session ID. This adds meaning beyond the schema, though it does not describe ID format or validation, so it's not a perfect 5.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Where this topic stands: phase, mastery counts, and what's still locked.' It uses a specific verb ('stands' implies showing status) and specifies the resource (topic) and the exact data points. This differentiates it from sibling tools like start_topic and submit_step, which are action-oriented.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage context: 'Useful for orienting mid-session or after a break.' It also instructs how to relay the output ('Report it... as progress, not as a schedule — don't read out due dates'), which is valuable. However, it does not explicitly contrast with alternative tools or state when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• next_drillA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full behavioral disclosure burden. It discloses important behaviors: the choice is algorithmically determined, and the returned explanation must not be shown to the user unless they answer incorrectly. This is valuable beyond the schema, though it does not explicitly confirm read-only safety or side-effect-free behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well structured. It front-loads the primary purpose, adds a crucial usage directive, notes a key output behavior, and documents the parameter in a compact format. Every sentence earns its place without unnecessary fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has only one parameter but no output schema, so the description should clarify return values. It does mention returning the element's explanation and the kind of question, but does not fully enumerate all response fields or their shapes. Given the tool's simplicity, this is a minor gap, and the overall description is largely complete for invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It adds semantic meaning to the single parameter by stating 'session_id: From start_topic,' which tells the agent where to obtain the value. This goes beyond the schema's bare 'Session Id' label and provides useful origin context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource: 'Get the next element to drill and the kind of question to ask about it.' It clearly identifies what the tool does and naturally distinguishes itself from siblings by focusing on 'next' and 'drill,' which aligns with the drill workflow.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: 'take what you're given rather than picking a topic yourself.' It implicitly tells the agent when to use this tool (whenever a next drill item is needed) and even mentions the prerequisite session_id comes from start_topic. However, it does not explicitly state when not to use it or name alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• submit_stepA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosing behavior. It explains the validation check ('artifact is checked for shape'), the failure mode ('thin or malformed submission comes back with a reason and the phase is unchanged'), and the recommended follow-up ('fix it and submit again'). It does not describe success-side effects beyond receiving the next instruction, but this is sufficient for a submission tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a clear one-sentence purpose, a behavioral note about validation, and a brief Args list. Every sentence contributes value, and the most important information (how to advance) is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple tool (2 params, no output schema, no annotations), the description covers the essential aspects: when to use, what to submit, failure behavior, and how to recover. It could be slightly more explicit about the success path (e.g., phase progression), but 'receive the next instruction' implies the next step. Overall, it is complete enough for an agent to invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides no descriptions (0% coverage), so the description compensates by explaining both parameters: `session_id` is sourced from `start_topic`, and `artifact` is defined as 'whatever the current instruction asked you to produce' with concrete examples (markdown string or list of objects). This adds meaningful guidance beyond the schema's simple type/title fields.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Hand in the current step's work and receive the next instruction.' It uses a specific verb ('Hand in') and resource (current step's work), and explicitly differentiates itself from siblings by noting 'This is the only way to advance,' making its unique role unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use the tool: whenever you have produced the artifact for the current step and want to advance. It states 'This is the only way to advance,' implying that other tools like `next_drill` or `status` are not alternates for this action, though it does not explicitly name them as exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• start_topicA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses meaningful behavior beyond the basic purpose: it returns 'the instruction for the current step and nothing about later ones,' and it warns about mastery tracking: 'mastery is tracked per element, so a second pass would split the record rather than double it.' With no annotations, this provides valuable transparency about side effects and output scope.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured. It starts with a clear purpose, adds usage guidance, then behavioral notes, and ends with a neatly formatted Args section. Every sentence contributes value, and there is no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-parameter tool with no annotations and no output schema, the description covers purpose, usage, resume behavior, and the return constraint. It does not detail the output format, but it does say the return is an instruction, which is adequate for an agent to proceed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides no parameter description, but the description fills the gap with 'topic: What to learn, e.g. "kafka consumer groups" or "options pricing."' This gives both semantic meaning and concrete examples, fully compensating for the 0% schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a clear verb phrase 'Begin learning a topic, or rejoin one already in progress,' which immediately states the tool's function. It also distinguishes from siblings by positioning itself as the entry point: 'Call this first, whenever the user wants to learn or study something.'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives explicit when-to-use guidance: 'Call this first, whenever the user wants to learn or study something.' It also explains the resume case with 'Re-calling this for a topic already underway resumes it rather than starting over.' However, it does not explicitly name alternatives or exclusions, such as when to use the sibling 'resume' tool instead.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ask_userA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that the answer returns as a tool result, that the question is recorded verbatim, and it describes an error path. This is more transparent than most, though it does not cover potential side effects or permissions (likely none needed for a user prompt).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with purpose, then gives usage guidance, fallback, and parameter details. It is slightly verbose but every sentence adds value. The parameter block is efficiently formatted.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having no output schema and no annotations, the description covers purpose, usage boundaries, fallback, and parameter origins. For a simple user-prompt tool, this is complete enough for reliable invocation without further clarification.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but each parameter is given meaningful context in the description: session_id comes from start_topic, element_id from next_drill, and question must be worded exactly as the user sees it. This adds far more meaning than the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb and resource: 'Put your drill question to the user and return their answer.' This distinguishes it from siblings like start_topic or next_drill, which handle different stages of a drill workflow.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Prefer this over asking in chat,' and provides a clear fallback: if the tool errors, ask in chat and pass `question` and `answer` to `grade_drill`. This gives direct when-to-use and when-not-to-use guidance with an alternative named.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• resumeA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It clearly indicates a read-only operation ('List topics') and adds context about the time-sensitive nature ('due for review right now') and the recommendation to prioritize these over new topics. It does not explicitly state 'read-only' or describe return format, but the verb implies no destructive effects, and the guidance adds useful behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each earning its place: the first defines the action, the second gives explicit call conditions, and the third provides a prioritization heuristic. It is front-loaded with the core purpose and contains no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters, no output schema, and no annotations, the description provides all necessary context: what it does, when to use it, and how to prioritize it over alternatives. It is complete for an agent to select and invoke the tool correctly without additional assumptions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the description does not need to explain parameter details. According to the rubric, 0 params receives a baseline of 4. The schema is empty and fully covered vacuously.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The first sentence 'List topics with elements due for review right now' uses a specific verb ('List') and resource ('topics') with a clear scope ('due for review right now'). This distinguishes it from siblings like 'start_topic' (which starts new topics) and 'status' (which likely provides general state).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to call: 'when the user opens a session without naming a topic, or asks what they should study.' It also provides a prioritization rule: 'Reviewing something about to decay beats starting something new, so offer these before suggesting a fresh topic,' which guides the agent away from the alternative 'start_topic' toward this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• grade_drillA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully shoulders the burden. It discloses the scheduling side-effect, the rejection behavior for never-asked elements, and the semantics of notes as the only record of how the user was wrong. This is rich behavioral context beyond the raw schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The code-formatted Args list is compact and scannable, with a front-loaded purpose sentence and a dedicated prerequisite paragraph. Every sentence contributes either procedural guidance or parameter semantics; no filler is present.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 6-parameter tool with no output schema and no annotations, the description provides the necessary context: it explains when to use it, all parameters, and the rejection condition. The only potential ambiguity is the exact scheduling behavior, but 'schedule when the element comes back' is sufficient for an agent to invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description compensates thoroughly by explaining every parameter: session_id's source, element_id's meaning, verdict's allowed values and guidance, answer's phrasing, question's conditional use, and notes' purpose. This adds significant meaning beyond the bare property names and types.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The opening sentence 'Record how the user did, and schedule when the element comes back' uses a specific verb and resource, clearly distinguishing this grading/recording tool from siblings like ask_user and next_drill. It goes beyond tautology by stating both the recording and scheduling responsibilities.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly states the prerequisite that a question must exist before grading, naming ask_user as one valid source and chat with passed question/answer as another. It also gives a clear rejection condition ('Grading an element you never actually asked about is rejected') and advises on verdict strictness, making when-to-use guidance concrete.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.