Server Quality Checklist

Tool Scores

• progressA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It adds useful behavioral context by explaining that the tool surfaces history and mentions that the 'detail' flag includes per-attempt rows. However, it does not explicitly state whether the operation is read-only, has side effects, or requires specific permissions, leaving some ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, using two short paragraphs to convey purpose, usage, and parameter behavior. Every sentence earns its place, and the key information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (one boolean parameter, no annotations) and the presence of an output schema, the description covers all essential aspects: what the tool does, when to use it, and what the parameter controls. It is complete enough for an AI agent to select and invoke the tool correctly, though it could mention edge cases or exclusions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must compensate. It explains that the 'detail' parameter includes per-attempt rows, which gives meaning beyond the bare schema. Still, it does not clarify what the default (false) returns or the exact format, so compensation is partial.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as providing the candidate's history, including weakest areas, repeated mistakes, and recent attempts. Though it lacks an explicit verb like 'retrieve' or 'show', the noun-phrase description is specific and distinguishes it from siblings that handle audio, cases, or scoring.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly explains when to use the tool: to recommend practice topics and to open a session with specific references to past performance. It provides a concrete example, giving clear context, though it does not explicitly mention when not to use it or name alternative tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• room_statusA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses important behavioral traits: what it reports ('whether the page acknowledged it, whether the mic is open and whether a tab is even connected') and the meaning of the key field `page_problem` (names the reason the room won't work, or null when healthy). This goes beyond a simple 'inspect' and gives diagnostic context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded: the first sentence states the core function, the second paragraph gives when to use it, and the third explains the most important field. Every sentence adds value; no fluff or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a simple read-only diagnostic tool with one optional parameter and an output schema. The description explains the use case, the kinds of information returned, and the meaning of the key field. It does not need to detail return values since an output schema exists. Minor gaps like not naming sibling alternatives are acceptable given the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter `events` is fully documented in the schema with 'Recent log lines to include.' Since schema coverage is 100%, the baseline is 3. The description does not add any additional semantic detail about the parameter beyond the schema, so no extra credit.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool's purpose: 'Inspect the interview room: state, page health, and a timestamped log.' This is a specific verb ('Inspect') and resource ('interview room'). It does not explicitly contrast with sibling tools like 'probe' or 'progress', so it lacks explicit sibling differentiation, but the purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives strong usage guidance: 'Use this the moment anything looks wrong — silence, a stall, an answer that never arrived.' It also explains the benefit over asking the candidate: 'diagnose without asking the candidate what they see on screen.' This is clear context, though it does not mention when not to use the tool or name alternative tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scoreA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It discloses that the policy is server-side and explains the PARTIAL verdict behavior, which is useful. However, it does not mention side effects, idempotency, or whether the tool can be called multiple times for the same answer.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded, with the core purpose in the first sentence. The second and third sentences provide crucial guidance about the server-side policy and the probe cue without unnecessary wordiness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter tool with an output schema, the description covers the essential context: what to record, what the verdict means, and how to act on a specific verdict. It does not explain all possible verdicts, but the output schema likely covers return values, making the description sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the `covered` parameter as component ids from the rubric. The description adds the concept of 'semantic matching' and reinforces that the agent supplies that matching, but it does not add substantive new parameter semantics beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Record which rubric components the answer covered, and get the verdict.' This clearly states the tool's function and differentiates it from siblings by referencing the verdict and the probe alternative.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit guidance on when to use `probe` instead of moving on: 'A PARTIAL verdict with a named gap is your cue to probe rather than move on.' It also explains the server-side policy, providing clear context for use, though it does not enumerate all exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• start_drillA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It explains the sampling behavior with 'target_weaknesses' and notes that each question is standalone. However, it omits potential side effects (e.g., whether it resets progress) and includes a puzzling 'only hard math' phrase that could mislead.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded, using two sentences to convey the core purpose and a key behavioral nuance. Every sentence earns its place with no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description gives an overview of the drill mode and sampling behavior, but the 'only hard math' phrase contradicts the available 'types' parameter (which supports non-math types). With 7 parameters and low schema coverage, the description does not fully equip an agent to correctly invoke the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is only 43%, so the description should compensate. It adds valuable detail for 'target_weaknesses' (bias without excluding the rest), but leaves tags, count, speed, and difficulty parameters unexplained, leaving a gap for those.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that 'start_drill' begins a drill mode with loose questions pulled across cases and no full-case context. It differentiates from sibling tools like 'start_case' by specifying the standalone nature of the drill.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a clear context for when to use this tool: when you want a drill of standalone questions without case context. It implies a distinction from full-case tools but does not explicitly name alternatives or exclusions, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• next_questionA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It discloses critical behaviors: reading aloud, returning only metadata ('never the text'), and the meaning of read_aloud=false. This goes beyond the basic action and helps the agent understand the tool's interaction model.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with the primary action, and every sentence adds value: action, return behavior, and usage advice. No filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that an output schema exists and the parameter schema is fully documented, the description covers the essential context: purpose, return type, and calibration guidance. It slightly lacks edge-case behavior (e.g., what if no next question), but this is minor for a tool of this complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% and the schema already provides rich semantics for both parameters (read_aloud explains its effect on grading, question_id explains when to jump). The description itself adds no additional parameter-level meaning, so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Advance to the next question and read it aloud.' This clearly distinguishes it from siblings like repeat_question, and the phrase 'never the text' clarifies its scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear context: 'Use the type to calibrate' gives actionable guidance after the call, and 'Use in candidate-led cases' explains when question_id should be used. However, it doesn't explicitly name when not to use this tool or contrast with alternatives like repeat_question.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• probeA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It does disclose behavioral traits: hints escalate weakest-first, and the workflow involves speaking the probe and recollecting the answer. However, it does not disclose what happens when probes are exhausted (e.g., returns a special value, error, or nothing) nor does it explicitly state the absence of side effects or permissions needed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three concise sentences, front-loaded with the primary purpose. The usage tip and exhaustion guidance are integrated without redundancy or extraneous detail. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with an output schema present, the description sufficiently covers purpose, workflow, and termination condition. The only minor gap is the exact behavior when probes run out, but an output schema likely covers return values, so this is acceptable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. There is nothing to add beyond the empty schema, and the description properly focuses on behavior rather than nonexistent parameter details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Get the next hint for the current question, escalating weakest-first.' This is a specific verb ('get') plus resource ('next hint for current question') with a distinctive behavior ('escalating weakest-first') that distinguishes it from sibling tools like next_question or reveal_model_answer.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit usage guidance: 'Speak the probe with say, then collect_answer again.' It also gives an exclusion: 'When probes run out... give the answer and move on.' However, it does not explicitly name an alternative tool to use when probes are exhausted, such as reveal_model_answer, so it stops short of full alternative comparison.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• room_actA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It discloses that the tool simulates the candidate's actions, is intended for testing not cheating, and warns against real-interview misuse. It does not detail all side effects, but the concrete flow mentioned ('press Start, submit an answer, confirm the loop advances') gives a useful behavioral picture.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded: the first sentence states the core purpose, the second clarifies its testing-only nature, and the third adds a concrete example and an ethical guardrail. Every sentence earns its place with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is complete enough for a testing tool: it explains the purpose, the ethical boundary, and a typical flow. The schema covers parameter details and an output schema exists, so the description need not explain return values. It does not explicitly mention the open_mic and ack_speech actions, but those are already described in the schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% and the schema already explains each action value and the text parameter. The description adds no additional meaning beyond the schema, only restating the 'submit an answer' concept generically.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource combination: 'Drive the room as if you were the candidate.' It clearly distinguishes the tool from siblings by emphasizing the testing-only, candidate-impersonation role, and enumerates concrete actions like pressing Start and submitting an answer.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool ('rehearse or diagnose the whole flow') and when not to ('During a real interview, don't answer on the candidate's behalf'). It lacks a named alternative tool, but the context is clear enough to avoid misuse.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• show_exhibitA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations available, the description carries the full burden of behavioral disclosure. It clearly states the primary behavior (displaying an exhibit) and adds context about its role. While it does not mention side effects or state changes, the read-only nature is implied by 'candidate is meant to see,' providing reasonable transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded: the first sentence delivers the core purpose, and the second adds essential context and usage instruction. Every sentence contributes value without redundancy or unnecessary length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one optional parameter and an existing output schema, the description is reasonably complete. It explains what the tool does, why it is used, and the sequence of actions. It could mention the default for exhibit_id, but that is already covered in the schema, so the description is sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides a complete description for the only parameter (exhibit_id) with 100% coverage. The tool description adds no additional parameter details, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource ('Display a chart or table in the room') and further clarifies its unique role in the interview by noting that exhibits are the one thing the candidate is meant to see. This distinguishes it from sibling tools like 'say' or 'reveal_model_answer'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: 'Speak the intro line, then give them a moment before asking what they make of it.' It also explains the purpose relative to the candidate. However, it does not explicitly name alternatives or state when not to use this tool, so it does not fully meet the 'explicit when/when-not' bar.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• start_caseA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It explicitly discloses that the briefing includes only topics of clarifying information, 'never the facts themselves', which is a crucial behavioral limitation. It also states that it opens the interview room in the browser, providing observable side effects. It lacks detail on potential resets or state changes, but the key disclosure is valuable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences long, with the primary purpose front-loaded. Every sentence adds critical information (what it does, what it returns, and what to tell the candidate). No filler or redundant content is present.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the tool's function, return value structure, and a notable constraint (not revealing facts). It also provides practical guidance for the agent. With an output schema present and 100% parameter schema coverage, the description does not need to explain return formats or parameter syntax. It is sufficiently complete for a starting tool, though it could mention behavior if the room is already open.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description does not add parameter-level meaning beyond what the schema already provides; it only mentions the briefing content, which relates to output, not parameters. Thus, it meets the baseline without enhancing parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Begin a full case interview' and 'Opens the interview room in the browser', providing a specific verb and resource. It distinguishes itself from siblings like start_drill by specifying a full case interview and returning a briefing, making the tool's purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool ('Begin a full case interview') and gives actionable post-call instructions ('Tell the candidate the room is open and confirm they can hear you before starting'). It does not explicitly name alternative tools or exclusion conditions, but the guidance is strong enough to be above the vague category.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• listenA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description takes on the burden of behavioral disclosure. It reveals the key retry mechanism ('if it returns heard: false ... just call it again') and sets expectations about silence ('Never fill the silence'). However, it does not mention whether the call blocks for the full max_wait or other potential side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is succinct and well-structured. It opens with the primary purpose, follows with the usage context, and closes with the retry behavior—all in a few sentences without repetition or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple listening tool with a single optional parameter and an output schema, the description covers the essential operational aspects: what it does, when to use it, and how to interpret the return value. Given the output schema exists, the lack of detailed return documentation is acceptable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides a description for max_wait ('Seconds to wait this call'), so the parameter semantics are fully covered by the schema. The description adds no additional parameter-level detail, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's core function: 'Hear whatever the candidate says next'. It uses a specific verb ('hear') and resource ('candidate'), and differentiates itself from question-asking tools by noting it is 'not tied to any question'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context by labeling this as the 'conversational channel' and giving example utterances. It also gives an implicit alternative by stating it's 'not tied to any question', suggesting other tools handle questions. It further explains the retry behavior when receiving 'heard: false'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• repeat_questionA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It indicates the action ('deliver') and permission context, but does not disclose potential side effects (e.g., whether it is strictly read-only or whether it changes any state). This is a minor gap for a simple repeat action, but still present.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences, front-loaded with the core purpose and immediately useful context. Every word earns its place, with no wasted or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with a clear purpose, the description is complete. It says what the tool does and who is allowed to use it. The existence of an output schema is not described, but the description does not need to explain return values for a simple question-repeat action.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters, so the baseline is 4. The description adds no parameter-level details, but none are needed since there are no parameters to explain.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Deliver the current question again.' This uses a specific verb and resource, and it is distinct from sibling tools like next_question, which advances rather than repeats.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'Candidates are allowed to ask' provides clear context for when this tool is appropriate—when a candidate requests repetition. It does not explicitly name alternatives or exclusions, but the context is sufficiently clear for such a simple tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• catalogA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It states the tool contains only titles and counts, never content, and is safe to show the candidate. This adds useful behavioral context beyond the tool name, though it doesn't detail output schema or potential edge cases.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core function, then safety and usage example. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with no parameters, and the description provides purpose, safety, and usage context. Given the output schema exists to detail return structure, the description is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and schema coverage is 100%, the description adds no parameter-specific details, but none are required. Baseline for zero parameters is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists every case and drill category with counts, using the specific verb 'list' and resource scope. It implicitly differentiates from sibling tools like start_case and start_drill by focusing on cataloging available options.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs to use it to offer choices, with an example dialogue. It also notes safety for showing the candidate. However, it does not directly mention when not to use it or alternative tools, so a 4 rather than 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• finishA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden and does well: it reveals that the tool terminates the session and returns a scorecard, and explains the semantics of `limiting_factor` and `recurring_habits`. It could additionally note the irreversible nature of ending the session, but the verb 'End' already implies this.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description leads with the core purpose in one sentence, then adds focused sentences about the scorecard's interpretation. No filler; each sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no inputs and an output schema exists, the description provides sufficient context about what the scorecard contains and how to interpret it. The explanation of limiting_factor and recurring_habits adds value beyond the schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool takes no parameters, and the input schema has no properties, so there is nothing for the description to add. The baseline for zero parameters is 4, and the description appropriately focuses on output behavior.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: ending the session and returning the scorecard. This distinguishes it from siblings like score (which likely evaluates individual answers) and progress (which tracks status).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The opening phrase 'End the session' communicates the primary use case—wrapping up the interview. It does not explicitly name alternative tools or exclusion conditions, but the context is clear enough for an agent to know when to invoke it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• release_clarificationA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the decision rule ('matching intent, not deciding generosity') and the selective condition, which is adequate for a simple release action. It could mention irreversibility or output effects, but the core behavioral trait is covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the action, no redundant phrasing. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter tool with an output schema, the description provides sufficient context (when to use, hownot to misuse) and the schema/annotations fill the rest. It is complete for an agent to select and invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (the single parameter 'topic_id' is described as 'Clarification id from available_topics'). The description adds only marginal context ('withheld fact'), so it stays at the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a clear, specific verb ('Release') and resource ('specific withheld fact by id'), immediately distinguishing it from siblings like answer_clarification or reveal_model_answer. The phrase 'when keyword matching missed it' further differentiates it as a fallback mechanism.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit 'Use only when' condition, plus an exclusion ('if they didn't ask for it, don't release it'). This gives the agent precise boundaries for when to invoke the tool and what to avoid.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• answer_clarificationA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations supplied, the description carries the full burden and does so thoroughly. It reveals the deliberate fact-release mechanism, the exact no-match response format ('matched: false'), and the hard rule against fabricating case facts. This prepares the agent for edge cases and sets expectations beyond the tool name.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded, with the primary action in the first sentence and behavioral guardrails in the following sentences. Every clause earns its place; the 'good candidates ask, weak ones assume' line justifies the design without bloating the definition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool with an output schema present, the description covers the trigger, the behavior, the no-match response, and the boundaries. There are no significant gaps that would prevent an agent from using the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already fully describes the sole parameter ('The candidate's clarifying question, verbatim'), so the baseline is 3. The description adds context about the question being a trigger, but no additional format or constraint details. It does not need to compensate for schema gaps because there are none.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb ('Release') and a well-defined resource ('one withheld fact') conditioned on the candidate asking. It also states this is the *only* way case data reaches them, which differentiates it from sibling tools that might present information. The purpose is unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The trigger is explicit: 'if the candidate asked for it.' It also provides clear when-not-to guidance: if nothing matches, say you don't have the information and let them proceed, and never invent facts. Although it does not name alternative tools, the claim that this is the only channel effectively rules out alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ask_case_promptA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description fully discloses the tool's unusual behavior: it returns no content, the prompt reaches the candidate inside the room only, and never through the tool result that would land in the chat transcript. This is critical behavioral context that the schema and annotations do not provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core action, and every sentence earns its place. It avoids redundancy and is efficiently structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that the tool has no parameters and no annotations, the description covers all necessary aspects: purpose, return behavior, and expected next step. The output schema exists, so return values are already defined, and the description does not need to repeat them.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters, so the description does not need to add parameter details. Schema coverage is 100% and the baseline for 0 params is 4; the description could optionally mention the absence of parameters but it is not necessary.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Read') and resource ('case prompt') with an explicit purpose ('to open the interview'), making it clear and distinct from sibling tools like start_case or say.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context ('to open the interview') and sets expectations for what follows ('After this, expect clarifying questions before they start structuring'). It does not explicitly name alternative tools or when-not-to-use, but the context is unambiguous enough for an AI agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• collect_answerA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full behavioral burden. It discloses the waiting/retry loop, the distinction between deterministic math grading and rubric-based framework/synthesis grading, and the handoff to `score`. It even explains the committed transcript and the need for human judgment, which is far beyond what the schema alone would convey.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, then flows logically into usage, retry behavior, and grading semantics. Each sentence earns its place: no filler, no repetition, and the structure mirrors the actual invocation flow. It is appropriately concise for the complexity it covers.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema and the tool's interactive nature, the description covers all essential context: when to call it, what the return states mean, how grading differs by answer type, and what action to take afterward. This is complete enough for an agent to invoke and handle it correctly in a live interview flow.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description does not add extra semantic detail about `max_wait` or `acknowledge` beyond what the schema already provides. It indirectly implies `max_wait` controls the wait via the ready:false retry loop, but it does not explicitly elaborate on parameter usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Listen for the candidate's spoken answer, then grade what can be graded.' It clearly distinguishes from sibling tools like `listen` (which presumably only listens) and `score` (which receives the rubric component IDs from `collect_answer`). The purpose is unmistakable and actionable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to call it ('right after asking a question'), how to handle `ready: false` ('call it again'), and what not to do ('Don't fill the silence'). It also names `score` as the follow-up tool for framework/synthesis answers, giving clear when-to-use guidance relative to an alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reveal_model_answerA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the tool's output (the casebook's answer), the timing restriction ('Only after grading'), and a behavioral directive (paraphrase conversationally, not verbatim). This adds meaningful context beyond a simple 'returns answer' statement.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exactly two sentences, with the core purpose front-loaded. Every clause adds value: the first sentence states what the tool provides, the second gives timing and usage guidance. There is no redundant or wasted text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with an output schema present, the description is highly complete. It covers what the tool does, when it is appropriate to use it, and how the returned answer should be presented. The existence of an output schema covers return-value specifics, so the description does not need to explain those.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the schema is empty (100% schema description coverage). The baseline for 0 params is 4, and the description does not need to explain parameter semantics. It appropriately focuses on the tool's purpose and usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: revealing the casebook's own answer for the current question. It uses a specific verb ('reveal') and resource, and distinguishes itself from siblings by specifying the timing ('after grading') and the intended pedagogical use.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit usage direction: 'Only after grading' is a clear when/when-not constraint, and 'Use it to explain what a strong answer sounds like' defines the intended context. It also provides delivery guidance by instructing to paraphrase rather than read verbatim, which is a strong usage guideline.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sayA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden and succeeds: it discloses voice/text mode behavior, the mic-opening side effect of expect_reply, and formatting constraints (no lists, markdown, or long sentences). These are important behavioral traits beyond what the schema conveys.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately brief, front-loaded with the core purpose, and each subsequent sentence adds essential usage or behavioral detail. No filler or redundant restatement of the schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter tool with an output schema, the description covers purpose, usage modes, formatting rules, parameter behavior, and the natural follow-up action. It is fully sufficient for an agent to select and invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds semantic value by explaining the expect_reply default and its behavioral consequence ('the microphone opens as soon as you stop talking'), and reinforces the text formatting constraints. This goes beyond the schema's parameter descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Speak in your own voice — greetings, probes, transitions, feedback.' It uses a specific verb ('speak') and resource ('your own voice'), and distinguishes it from sibling tools like listen, probe, and ask_case_prompt by framing it as the agent's spoken chat line.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides strong contextual guidance: keep it conversational/brief, no lists/markdown, and explains that expect_reply opens the mic. It explicitly names 'listen' as the follow-up step, but does not contrast when to use this versus other speaking tools like probe or ask_case_prompt, so it lacks explicit exclusions or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.