Server Quality Checklist

Tool Scores

• ask_intensive_chatA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Despite missing annotations, the description discloses key behaviors: returns user's answer or indicates non-response, maintains chat history, and supports predefined options. However, it lacks details on error cases or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with labeled sections and front-loaded summary. However, some repetition exists (e.g., features overlap with usage notes). Could be slightly more concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description explains return behavior. For a tool with 3 parameters and simple interaction, it covers essential aspects: session requirement, repeated use, and optional options. Missing potential edge cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has 100% coverage, but the description adds value with examples and clarifies optional nature of 'predefinedOptions'. This exceeds the baseline 3 by providing practical usage context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it asks a new question in an active intensive chat session previously started, with specific verb and resource. It distinguishes from siblings like 'start_intensive_chat' and 'stop_intensive_chat' by focusing on continuation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'whenToUseThisTool' section explicitly lists scenarios for use, and importantNotes highlight the prerequisite session ID and repeated usage within the same response. This provides clear guidance on when and how to use vs alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• stop_intensive_chatA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Given no annotations, the description carries full burden and discloses key behaviors: closes console window, frees system resources, marks session complete. It omits potential side effects like idempotency or error handling, but the core behavioral traits are well covered for a termination action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Highly structured with clear sections (description, importantNotes, whenToUseThisTool, etc.). Every sentence adds value, and the core purpose is front-loaded. No unnecessary verbosity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one required parameter and no output schema, the description is fully complete. It covers what it does, when to use, how to use (with example), and what to expect. No gaps remain for an agent to select and invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for the single parameter 'sessionId', with the schema providing a description. The description repeats the same parameter info without adding new semantic meaning, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states 'Stop and close an active intensive chat session' with a specific verb and resource. It clearly distinguishes from siblings like 'start_intensive_chat' and 'ask_intensive_chat' by noting it must be called after all questions have been asked.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use conditions: after completing 'ask_intensive_chat', when the multi-step process is complete, and as the final action. Also includes a strong directive that it 'must be called' and 'should always be called', leaving no ambiguity about its role in the workflow.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• message_complete_notificationA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries behavioral info. It specifies cross-platform OS notifications and best practices like consistent projectName usage. Lacks details on potential side effects, but for a simple notification tool this is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured into sections (description, notes, when to use, features, best practices, parameters, examples). It is detailed but each section adds necessary value; no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with 2 string parameters and no output schema, the description is fully complete: it explains purpose, usage, parameters, examples, and best practices. No gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 100% coverage with concise descriptions. The description adds value by explaining parameter use (title vs body) and providing examples, exceeding the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool notifies when a response completes and must be used exactly once per message. It distinguishes itself from sibling chat tools by focusing on signaling completion.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit 'whenToUseThisTool' and 'importantNotes' provide comprehensive guidance: use at end of query, after tool sequences, or multi-step processes. The mandatory once-per-message rule is emphasized.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• request_user_inputA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description fully discloses behavior: pop-up display, return of user response or timeout after 60 seconds, context maintenance, graceful handling of empty responses, and formatting with project context. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with sections but is lengthy (many sentences). Some redundancy between importantNotes and bestPractices (e.g., both emphasize frequent use). Could be tightened without losing clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 3-parameter tool with no output schema, the description is exceptionally complete: covers purpose, usage guidance, features, best practices, and examples. Leaves no gaps in understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline 3. The description's parameters section adds context beyond schema: e.g., projectName is 'used in prompt formatting', predefinedOptions are optional. This adds meaningful value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool sends a question to the user via a pop-up command prompt, with explicit purpose of clarifying requirements, confirming plans, or resolving ambiguity. It distinguishes from sibling tools like ask_intensive_chat by specifying a pop-up prompt rather than a chat message.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  A dedicated 'whenToUseThisTool' section provides exhaustive scenarios, and 'bestPractices' explicitly instructs not to use the tool when another tool can answer the question, offering clear alternatives. This provides excellent decision support.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• start_intensive_chatA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully discloses behaviors: opens persistent console window, returns session ID, must be closed, configurable timeout, maintains chat history, and warns against unnecessary questions. This is comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with separate sections but contains some redundancy (e.g., 'Highly recommended' and 'Very useful' are similar). It is thorough but could be slightly more concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple parameter list, no output schema, and missing annotations, the description covers all necessary aspects: purpose, usage, important notes, parameters, examples, and best practices. It feels complete for the tool's role.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Only one parameter (sessionTitle) with 100% schema coverage. The description adds context that the title appears at the top of the console, which goes beyond the schema's description. A score of 4 is appropriate for the added value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it starts an intensive chat session for gathering multiple answers quickly. It uses specific verbs like 'start', 'gather', 'opens', and distinguishes from sibling tools such as ask_intensive_chat and stop_intensive_chat.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes explicit when-to-use scenarios (e.g., collecting series of quick answers, multi-step processes) and when-not-to-use (e.g., prefer other tools if they can answer). It also provides important instructions on using ask_intensive_chat and closing with stop_intensive_chat in the same response.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.