Server Quality Checklist

Tool Scores

• session_observeC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states what is returned (URL and aria-snapshot) but doesn't clarify whether this is a passive read operation or whether it might alter session state. For an observation tool, the read-only nature is implied by the name but never explicitly stated given annotations are absent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero wasted words. It front-loads the action ('Return') and specifies both outputs clearly. The terseness is appropriate for a simple observation tool, though a touch more context about the aria-snapshot format could justify extending it.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations, no output schema, and zero schema description coverage, the description is thin. It names two outputs (URL and aria-snapshot) but doesn't describe the format of the aria-snapshot, whether both always come together, or what happens if the session is inactive/invalid. Given siblings include session_act (which presumably mutates), this read tool would benefit from clarifying its non-mutating role.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter `session_id` is a plain string with no schema description coverage (0%). The tool description doesn't mention this parameter at all, so the agent must infer that session_id refers to a session previously created by session_start. Since all sibling tools likely use the same session_id pattern, some inference is possible, but the description provides zero compensation for the documented schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns the current URL and aria-snapshot of the session's page. It uses a specific verb ('Return') with a clear resource ('current URL and aria-snapshot'). While it doesn't explicitly differentiate from siblings, the 'observe' name and description of returning page state clearly distinguish it from session_start (initialize), session_act (perform action), session_end (terminate), and qa_get_trace.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool vs alternatives. It doesn't state this is the read/inspection tool for a session's current state, nor does it mention any prerequisites (e.g., session must be active, started via session_start). There's no context about typical usage patterns such as observing before deciding an action.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• session_endC
  2.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the behavioral burden. It does disclose three side effects: closing the session, persisting a trace to disk (a side effect with external consequences), and returning the path. However, it doesn't note that this is likely destructive/terminal (session cannot be resumed after), disk persistence implications, or whether closing is irreversible. Moderate disclosure but incomplete for a state-changing operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One efficient sentence that front-loads the primary action and lists secondary effects compactly. No wasted words. Slightly better than a bare minimum given it packs three behavioral aspects into a single concise clause, though additional parameter guidance could have been added without excessive bloat.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a state-changing terminal tool with no output schema and no annotations, the description is thin. It explains the return value conceptually (the path) but not the format, doesn't address reversibility or whether the session becomes unusable, and gives no parameter semantics. Given it's a terminal lifecycle operation among several session tools, more guidance on ordering and implications is warranted.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and there's a single required parameter session_id with no type/format detail beyond 'string' in the schema. The description provides zero parameter information—it doesn't explain what session_id identifies, where to obtain it, or any constraints. Since coverage is 0% (below the 50% threshold), the description must compensate, and it fails to do so.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb and resource: 'Close the session, persist the trace to disk, and return the path.' It describes a specific end-of-session action that distinguishes it functionally from siblings (session_start, session_act) though it doesn't explicitly name them. The three-part behavior (close, persist, return path) is specific and actionable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool vs alternatives. With sibling tools like session_act and session_observe present, there's no 'when not to use' or comparison. The description implies it's used at the end of a session lifecycle, but this is implicit rather than stated. No exclusions or ordering requirements (e.g., must session_act first) are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• session_startB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It reveals the tool 'opens a fresh browser tab' and 'starts recording' — some behavioral disclosure — but doesn't say what happens to an existing session, whether the URL is navigated immediately, what start/recording side effects occur, or whether this is destructive to prior state. For a stateful action with no annotation coverage, this is a notable gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single concise sentence, front-loaded with the verb and purpose. Zero wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a stateful session-initiation tool with no annotations and no output schema, the description is somewhat thin. It doesn't explain what the agent gains from the 'recording' (observations? trace?), what the expected return indicates, or how session state carries to siblings. But it's a relatively simple 2-param tool that establishes an entry point; adequate overall, with room to improve on session lifecycle expectations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the schema already documents both parameters. The description's 'goal-driven' and 'fresh browser tab' wording adds minimal context (e.g., implying URL is the initial page, goal is the objective), but doesn't meaningfully exceed what the schema describes. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description has a clear verb+resource (open a fresh browser tab, start recording) with a specific purpose: goal-driven session. However, it doesn't distinguish itself from session_act/session_observe siblings, though those are clearly different actions (acting and observing within a session, not starting one). The 'goal-driven' qualifier adds useful scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this is the entry point for a session ('start recording'), and the required 'goal' parameter signals that the agent should first define a goal. However, it provides no explicit when-to-use guidance or alternatives, and doesn't mention session_end as the counterpart or any prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• session_actA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations provided, so the description carries the disclosure burden. The description states it performs a browser action, which implies side effects (navigation, DOM changes), but it does not disclose potential security implications of 'eval' (arbitrary JS execution), rate limits, or what happens on invalid actions or missing sessions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, zero waste, and the action enum is embedded directly in the description, making it instantly scannable. Front-loaded with the verb+resource and immediately enumerating the valid actions.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a polymorphic tool with 4 parameters and no annotations or output schema, so it demands more documentation. The description resolves the target polymorphism well but leaves ambiguity about which actions require 'value' versus 'target', error behaviors, and return format. Reasonable for a simple action dispatcher but notable gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is only 50%, but the description compensates by clarifying the polymorphic 'target' parameter meaning across action types (selector for click/fill, URL for goto, key for press, JS expression for eval). This adds meaningful semantics for the most ambiguous polymorphic parameter. However, 'value' and 'session_id' semantics are left to the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb+resource ('Perform a browser action in a session') and enumerates the supported action types (goto/click/fill/press/eval). It clearly communicates the tool's scope as a browser action executor within a session context, distinguishing it from siblings that start/observe/end sessions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by listing the action verbs, and the sibling tools (session_start/observe/end) establish the session lifecycle context. However, it does not explicitly state when to use this tool versus alternatives, nor does it mention prerequisites like requiring an active session before executing actions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• qa_get_traceA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden for behavior disclosure. It discloses that the tool returns the full trace and its constituent parts, which is useful. However, it doesn't state whether the trace persists after being read, whether it can be called multiple times, whether the session must be ended already, or error behavior (e.g., calling before a session exists). It's a read operation, which is fairly clear, but lacks depth beyond what's stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, zero wasted words. It front-loads the primary action, enumerates the return contents, states the purpose, and adds the key precondition. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a simple tool with one parameter and no output schema. The description adequately covers return contents (goal, steps, final URL, aria-snapshot) and the prerequisite (session_end first). It could mention the interaction with session_end's own behavior and replayability, but for a single-param read tool the description is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for the session_id parameter. The description mentions 'session_end first' and the tool name references 'trace,' but the description never explicitly states that session_id identifies which session's trace to retrieve. The parameter's role is only implied by context, not clearly explained in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description states a specific verb+resource ('Return the full recorded trace') with a precise scope enumerated: goal, every step, final URL, final aria-snapshot. It also names the purpose ('so the calling LLM can judge whether the goal was achieved'), which clearly distinguishes it from the sibling session_start/act/observe/end tools that operate during the session rather than reading back its record.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear usage context: it's a read-back/reporting tool used after a session, and it explicitly instructs 'Call session_end first.' While it doesn't enumerate exclusions or alternatives, the precondition (session_end first) and the clear purpose effectively guide when to use it. It could name alternatives explicitly but the sibling set makes the distinction obvious.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.