Server Quality Checklist

Tool Scores

• session_closeA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose all behavioral traits. It explains that keepHistory=true retains state.json and cookies for reuse, which is good. However, it does not mention what happens to ongoing chats or errors, leaving some gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one sentence plus a parenthetical on keepHistory. It is front-loaded with the action and contains no unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with 2 parameters and no output schema, the description covers the core action and one option. But it lacks information on return values, prerequisites (session must be open), and error scenarios, so it is adequate but not complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description compensates somewhat by explaining the keepHistory parameter (default true and what it does). However, it does not describe the sessionId parameter beyond its role implied by the action. Partial coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool closes a Sakana chat session and releases browser context. It is specific about the verb (close) and resource (session), and it distinguishes from sibling tools like session_open (open) and session_list (list).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage (when you want to close a session) but does not explicitly state when to use this tool versus alternatives like chat_interrupt. No exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• chat_interruptA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses the action (clicks stop button), effect (returns partial text, terminates via AbortSignal). Lacks details on safety (e.g., if no in-flight send) but is otherwise transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with key action, no extraneous details. Every sentence provides essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given simplicity (1 param, no output schema, no annotations), description covers main behavior and mechanism. Missing mention of preconditions (must have in-flight send) and error handling, but overall adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has one required parameter 'sessionId' with 0% description coverage (no schema descriptions). Description does not explain its purpose or how it is used, leaving the agent to infer from context. With low coverage, description must compensate but fails to do so.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool interrupts an in-flight chat send, with specific verb 'Interrupt' and resource 'chat send'. It distinguishes from siblings like chat_send (initiate) and session_close (close session).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description implies usage when a chat send is in progress and needs to be stopped. It does not explicitly state when not to use or provide alternatives, but the context is clear enough for an agent to infer appropriate usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• chat_sendA
  4.2/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  It discloses streaming behavior via MCP logging notifications, the final answer in the result, and the ability to abort with the 'signal' parameter. No annotations present, so description bears full burden and satisfies it well.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences front-load the purpose, add behavioral details, and provide usage for abort. No superfluous content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers streaming behavior and abort, but omits explanation of two boolean parameters and their defaults. No output schema, so description lacks return details beyond 'final answer'.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so description must explain parameters. It only mentions 'signal' (abort) and implicitly 'message' but does not explain 'sessionId', 'webSearch', or 'enableThinking'.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Send a message' and the resource 'open Sakana chat session'. It distinguishes from sibling tools like 'chat_interrupt' by focusing on sending messages.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the session must be open and explicitly mentions using signal:'abort' to interrupt. However, it does not compare with sibling tools like 'chat_interrupt' or provide when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• session_openA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It reveals that the session is persistent and headless, returns sessionId and rate limit, and has a one-time TOS acknowledgment. It could elaborate more on rate limiting or behavior on subsequent calls, but it is largely transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is only two sentences, front-loaded with the core purpose, and contains no unnecessary words. Every sentence provides essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of output schema and low schema coverage, the description does a reasonable job covering key aspects: return values, first-call requirement, and mode restriction. It is complete enough for a session creation tool but could expand on the headless parameter and other login options.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description must add meaning. It explains tosAccepted and the login 'anonymous' mode, but fails to describe the 'headless' parameter or other login enum values meaningfully. Partial coverage leaves some parameters underspecified.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Open a persistent Sakata chat session'), the resource ('Sakana chat session backed by headless Chrome'), and distinguishes itself from sibling tools (interrupt, send, close, list) by focusing on session creation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides specific usage constraints such as the requirement for tosAccepted on first call and that anonymous mode is only available in P1. It does not explicitly list alternatives but the sibling tool names make the context clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• session_listA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses that the tool lists both active and idle sessions, providing essential behavior. No contradictions or missing critical details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single, focused sentence with no wasted words. Front-loaded with the key action and resource.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (0 params, no output schema). The description fully covers the purpose and scope without missing details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, so the description adds value by clarifying scope; baseline is 4 as per guidelines.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (list) and resource (all known Sakana chat sessions), and distinguishes from sibling tools like session_open or chat_send.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use (to see all sessions) but does not explicitly state when not to use or provide alternatives. The context of siblings suggests usage, but guidance is minimal.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.