Server Quality Checklist

Tool Scores

• rollbackB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It mentions the tool is 'useful if a merge produced incorrect results,' implying it's a corrective action, but fails to disclose critical behavioral traits such as whether the rollback is destructive, irreversible, requires specific permissions, or has side effects on related data. This is a significant gap for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose and followed by a usage hint. It's efficient with minimal waste, though the second sentence could be more integrated for a perfect 5.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a mutation operation with potential data impact), no annotations, and no output schema, the description is incomplete. It lacks details on behavior, outcomes, error conditions, or what 'revert' entails, leaving the agent with insufficient context for safe and effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the parameter 'steps' clearly documented in the schema. The description adds no additional parameter semantics beyond what the schema provides, such as explaining what 'versions' refer to or constraints on 'steps.' Baseline 3 is appropriate as the schema handles the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('revert') and target ('previous checkpoint version'), distinguishing it from siblings like 'checkpoint' (create) and 'resume' (continue). However, it doesn't explicitly specify what resource is being reverted (e.g., data, state, or process), leaving some ambiguity compared to a perfect 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implied usage context ('if a merge produced incorrect results'), suggesting when to use it. However, it lacks explicit guidance on when not to use it or alternatives (e.g., vs. 'resume' or manual correction), and doesn't mention prerequisites like needing an existing checkpoint.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• resumeB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool loads a checkpoint and returns formatted context, but lacks details on behavioral traits like error handling (e.g., what happens if no checkpoint exists), side effects, or performance considerations. The description adds some value by specifying the return action, but it's insufficient for a tool with potential state implications.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is highly concise and front-loaded: two sentences that directly state the tool's action and outcome without waste. Every sentence earns its place by providing essential information, and there is no redundant or verbose content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (state-related with no annotations and no output schema), the description is minimally adequate. It explains what the tool does and its return purpose, but lacks details on the formatted context's structure, error cases, or interaction with siblings like 'checkpoint'. Without annotations or output schema, more behavioral context would improve completeness for safe agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and schema description coverage is 100%, so no parameter documentation is needed. The description does not add parameter information, which is appropriate. Baseline for 0 parameters is 4, as the description need not compensate for any gaps, and it correctly avoids unnecessary details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Load the last checkpoint at session start' specifies the action (load) and resource (last checkpoint), with context about when it operates (session start). It distinguishes from siblings like 'checkpoint' (create) and 'rollback' (revert), though not explicitly. However, it lacks full sibling differentiation, such as contrasting with 'status'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context: 'at session start' suggests when to use this tool, likely for initialization or recovery. However, it does not provide explicit guidance on when not to use it or alternatives, such as whether to use 'rollback' for different recovery scenarios. No misleading information is present, but the guidance is limited to implied context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• checkpointA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes the action ('Save'), storage location ('Redis'), and method ('Merges new context with existing project state using LLM-based summarization'), which covers key behavioral aspects. However, it lacks details on permissions, error conditions, or what happens if the merge fails.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that efficiently conveys the tool's purpose, usage context, and method. Every word earns its place with no redundancy or unnecessary details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (involving Redis storage and LLM summarization), no annotations, and no output schema, the description is adequate but has gaps. It explains what the tool does but doesn't cover return values, error handling, or dependencies, which could be important for an agent to use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents both parameters fully. The description adds no additional meaning about the parameters beyond what the schema provides, such as format examples or usage tips. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Save current context to Redis') and resource ('context'), while distinguishing it from sibling tools by mentioning '/clear' and the summarization method. It goes beyond the tool name 'checkpoint' to explain what is being checkpointed and how.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('before running /clear'), which implies it's a preparatory step for clearing operations. However, it doesn't explicitly mention when not to use it or name alternatives among the sibling tools (resume, rollback, status).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• statusA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It clearly indicates this is a read-only operation ('Show') and specifies what information will be returned, but doesn't disclose behavioral traits like whether it requires authentication, has rate limits, or how frequently the state updates. The description adds value by listing the metadata components but lacks operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('Show current state metadata') followed by specific examples. Every word earns its place by either stating the action or enumerating the metadata components without unnecessary elaboration or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's zero-parameter simplicity and lack of both annotations and output schema, the description provides adequate but minimal context. It tells what information will be shown but doesn't explain format, structure, or potential limitations of the returned metadata. For a status monitoring tool with no structured output definition, more detail about the return values would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters with 100% schema description coverage, so the baseline is 4. The description appropriately doesn't discuss parameters since none exist, focusing instead on what the tool returns. No parameter information is needed or provided beyond what the schema already indicates.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Show') and the comprehensive scope of resources ('current state metadata including version, active files, tasks, token usage, and checkpoint history'). It effectively distinguishes from siblings like checkpoint, resume, and rollback by focusing on read-only status display rather than state manipulation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context through the listed metadata items, suggesting it's for monitoring system state. However, it doesn't explicitly state when to use this tool versus alternatives like checking specific resources individually, nor does it mention any prerequisites or exclusions for usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.