Server Quality Checklist

Tool Scores

• clear_sessionC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions 'free memory', which hints at a destructive operation, but fails to disclose critical behavioral traits such as whether clearing is reversible, what data is lost, permission requirements, or side effects. This is inadequate for a tool that likely modifies state.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's front-loaded with the core action and purpose, making it easy to parse quickly. Every word earns its place without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (destructive operation with 2 parameters) and lack of annotations or output schema, the description is incomplete. It doesn't cover behavioral risks, return values, or error conditions, leaving significant gaps for the agent to operate safely and effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents both parameters. The description adds no additional meaning beyond implying that clearing sessions frees memory, but it doesn't explain parameter interactions (e.g., how 'session_id' and 'all' relate) or usage nuances. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Clear session(s) to free memory' clearly states the action (clear) and resource (session(s)), with a specific purpose (free memory). It distinguishes from siblings like 'get_session' or 'list_sessions' by indicating a destructive operation, though it doesn't explicitly contrast with 'scratchpad' or 'compress'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'scratchpad' or 'compress', which might also manage memory. The description implies usage for freeing memory but lacks explicit context, prerequisites, or exclusions, leaving the agent to infer based on general knowledge.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_sessionC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but provides minimal behavioral context. It mentions format options but doesn't disclose what data each format returns, whether this requires authentication, if there are rate limits, or how the 'active' session fallback works. The description is insufficient for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise single phrase with zero wasted words. The description is front-loaded with the core purpose and includes essential format information in a compact format.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 3 parameters, no annotations, and no output schema, the description is inadequate. It doesn't explain what a 'session' represents in this context, what data is returned in each format, or how the tool behaves when session_id is omitted. The description should provide more context given the lack of structured metadata.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters thoroughly. The description adds minimal value by mentioning format options, but doesn't provide additional semantic context beyond what's in the schema. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and resource 'session', with the additional detail about format options (full/summary/compressed). It distinguishes from siblings like 'clear_session' (destructive) and 'list_sessions' (multiple sessions), but doesn't explicitly contrast with 'compress' or 'scratchpad'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives like 'list_sessions' or 'scratchpad'. The description mentions format options but doesn't explain when each format is appropriate or any prerequisites for use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scratchpadC
  2.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations are empty, so the description carries the full burden. It discloses several behavioral traits: auto-verification for chains >3 steps, auto-compression, confidence-drift detection, consistency checks, and status-driven actions (e.g., 'verification_failed→revise|branch|override'). It also mentions defaults like 'confidence_threshold=0.8' and 'token_budget=3000'. However, it lacks details on error handling, performance limits, or side effects, leaving some gaps in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is overly verbose and poorly structured, with dense sections like 'OPS', 'DEFAULTS', 'STATUS→ACTION', and 'FLOW' that mix operational details, defaults, and usage flow without clear separation. Sentences are fragmented (e.g., 'Auto-verifies when chain >3 steps.'), and it includes unnecessary symbols like '[]=optional'. It is not front-loaded with a clear purpose, making it difficult to parse efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the high complexity (44 parameters, no output schema, no annotations), the description attempts to cover behavior and flow but falls short. It explains operations and status transitions but lacks details on return values, error responses, or integration with sibling tools. Without an output schema, the description should ideally explain what the tool returns, but it does not, leaving gaps in completeness for such a complex tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 98%, so the schema already documents most parameters extensively. The description adds minimal semantic value beyond the schema: it lists operation types (e.g., 'step', 'complete') and hints at parameter usage in the flow (e.g., 'step(question="...",thought="...")'), but does not explain parameter interactions or provide examples. With high schema coverage, the baseline is 3, and the description does not significantly compensate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description begins with 'Structured reasoning w/verification, trap detection, self-challenge', which provides a vague high-level purpose but lacks a specific verb-resource combination. It then dives into operational details without clearly stating what the tool fundamentally does (e.g., manage a reasoning session, perform stepwise analysis). The title is null, and the name 'scratchpad' is generic, making the purpose unclear without reading the entire description.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes a 'FLOW' section with numbered steps (e.g., '1.step(question="...",thought="...")→primes trap detection'), which implies usage in a sequential reasoning process. However, it does not explicitly state when to use this tool versus alternatives like 'clear_session' or 'compress', nor does it provide context on prerequisites or exclusions. The guidance is implied through the flow but not clearly articulated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_sessionsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only states what the tool does, not how it behaves. It lacks details on permissions needed, rate limits, whether it's read-only or mutating, pagination, or error handling. This leaves significant gaps for a tool that likely interacts with session data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action ('List active sessions') and adds clarifying detail ('with counts/branches'). There is no wasted verbiage, making it appropriately sized for a simple tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and the tool's potential complexity (listing sessions with counts/branches), the description is incomplete. It doesn't explain what 'counts/branches' means, the format of returned data, or behavioral aspects like safety or performance. This leaves the agent with insufficient context for reliable use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and schema description coverage is 100% (since there are no parameters to describe). The description doesn't need to add parameter semantics, so it meets the baseline for this case. No additional value is required beyond stating the tool's purpose.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('active sessions'), and specifies the scope ('with counts/branches'), making the purpose understandable. However, it doesn't explicitly differentiate from sibling tools like 'get_session' or 'clear_session', which would require more specific comparison.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_session' (for a single session) or 'clear_session' (for deletion). It implies usage for listing active sessions but offers no context about prerequisites, timing, or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compressB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds some context about the compression method (TF-IDF + NCD scoring) and performance (10× faster), but lacks details on permissions, rate limits, error handling, or output format. For a tool with 9 parameters and no annotations, this is a moderate gap, scoring at the baseline of adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with key information in the first part. It uses technical terms efficiently (e.g., 'CPC-style,' 'TF-IDF + NCD scoring') without unnecessary elaboration. However, it could be slightly more structured (e.g., separating performance claims from functional details), keeping it from a perfect 5.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (9 parameters, no annotations, no output schema), the description is moderately complete. It covers the compression approach and performance but lacks details on output format, error cases, or integration context. For a tool with rich input schema but no other structured data, this is adequate but has clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description doesn't add any specific parameter semantics beyond what's in the schema (e.g., it doesn't explain how 'target_ratio' interacts with 'max_tokens' or detail the algorithms). With high schema coverage, the baseline is 3, and the description doesn't compensate further.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs 'CPC-style sentence-level compression' with specific techniques mentioned (TF-IDF + NCD scoring, coreference/causal chains, filler removal). It distinguishes the tool by mentioning it's '10× faster than token-level LLM compression' and 'Keeps query-relevant sentences,' giving a clear sense of what it does. However, it doesn't explicitly differentiate from sibling tools (which appear unrelated to compression), so it doesn't reach a perfect 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It mentions it's faster than 'token-level LLM compression,' which implies a comparison, but doesn't name specific alternatives or provide explicit when/when-not scenarios. With no usage context provided, this falls to a minimal score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.