Server Quality Checklist

Tool Scores

• generate_session_summaryC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool generates a markdown summary but doesn't explain what happens to the summary (e.g., saved, displayed, sent), whether it modifies session state, or any side effects like permissions or rate limits. This is inadequate for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief and front-loaded with the main purpose, but includes redundant sections like 'Args:' and 'Returns:' that repeat schema information without adding value. It could be more streamlined by focusing only on clarifying aspects not covered elsewhere.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and low parameter coverage, the description is incomplete. It doesn't explain the return value beyond 'Confirmation message', leaving the agent uncertain about what to expect. For a tool with this complexity, more context on behavior and output is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, and the description adds minimal parameter semantics. It mentions 'summary: The llm generated summary of the conversation' but doesn't clarify format, length, or content expectations beyond what's implied by the name. With one undocumented parameter, this doesn't compensate for the schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Generate a markdown summary of the journaling session.' This specifies the verb (generate), resource (markdown summary), and context (journaling session). However, it doesn't explicitly differentiate from sibling tools like 'record_interaction' or 'start_new_session', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, such as whether a session must be active, or how it relates to siblings like 'start_new_session' for initiating sessions. This leaves the agent without context for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• record_interactionB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool records messages but doesn't explain where or how (e.g., in a database, log file, or session context), whether it requires permissions, or if it has side effects like overwriting data. This leaves critical behavioral traits unspecified for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with the core purpose stated first in a clear sentence. The Args and Returns sections are structured efficiently, though the 'Returns' part could be more specific than 'Confirmation message.' Overall, it avoids unnecessary verbosity while conveying essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a mutation with 2 parameters), lack of annotations, and no output schema, the description is minimally adequate. It covers the basic action and parameters but misses details like behavioral context, usage relative to siblings, and return value specifics. This leaves gaps that could hinder an agent's understanding in a broader workflow.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful semantics beyond the input schema, which has 0% coverage. It explicitly defines 'user_message' as 'The user's message' and 'assistant_message' as 'The assistant's response,' clarifying the purpose of each parameter. Since there are only 2 parameters and the schema lacks descriptions, this compensation is effective, though not exhaustive (e.g., no format constraints).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Record both the user's message and assistant's response.' It specifies the verb ('record') and resources ('user's message' and 'assistant's response'), making the action explicit. However, it doesn't differentiate from sibling tools like 'generate_session_summary' or 'start_new_session', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools or contexts where recording interactions is appropriate, such as after a conversation or as part of session management. Without such guidance, agents may struggle to select this tool correctly in relation to others.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• start_new_sessionA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes the action ('clearing previous conversation log') and the return value ('Welcome message with current save location'), which are useful. However, it lacks details on potential side effects (e.g., whether clearing is irreversible), authentication needs, or rate limits. It doesn't contradict annotations, but it's not fully transparent for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is highly concise and well-structured: two sentences that clearly state the action and the return value, with zero wasted words. It's front-loaded with the primary purpose, making it easy for an agent to parse quickly. Every sentence earns its place by providing essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (simple mutation with no parameters) and the absence of annotations and output schema, the description is somewhat complete but has gaps. It explains what the tool does and what it returns, but for a mutation tool that clears logs, it should ideally mention whether this action is destructive or reversible. Without annotations or output schema, the description could do more to cover behavioral aspects.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and the schema description coverage is 100%, so there's no need for parameter details in the description. The description appropriately focuses on behavior and output without redundant parameter information. A baseline of 4 is applied since no parameters exist, and the description doesn't attempt to add unnecessary details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Start a new journaling session by clearing previous conversation log.' It specifies the verb ('Start') and resource ('journaling session') with the action of clearing previous logs. However, it doesn't explicitly differentiate from sibling tools like 'generate_session_summary' or 'record_interaction' in terms of when to use each, which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by stating it 'clears previous conversation log,' suggesting it should be used when starting fresh rather than during ongoing sessions. However, it doesn't provide explicit guidance on when to use this tool versus alternatives like 'record_interaction' or 'generate_session_summary,' nor does it mention any exclusions or prerequisites. This leaves some ambiguity for the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.