Server Quality Checklist

Tool Scores

• ketan_list_filesA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It only explains the subdirectory parameter relative to workspace root and the default, but does not disclose whether listing is recursive, whether hidden files are included, or the return format. Essential behavioral details are missing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences, front-loaded with the primary purpose, and contains no redundant text. Every clause contributes to understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple and has an output schema, so return-value documentation is unnecessary. However, the description leaves ambiguity about whether file listing is recursive or includes subdirectories, which is a notable gap for a list operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides only the parameter name and default with 0% description coverage. The description compensates by clarifying the subdirectory is relative to the workspace root and defaults to the root, adding meaning that the schema alone lacks.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List' and the resource 'all files in the workspace', with explicit optional subdirectory scoping. This distinguishes it from sibling tools like ketan_read_file, which reads file contents, and ketan_get_status, which shows system status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage context is implied: use this when you need to enumerate files. However, there is no explicit mention of when not to use it or alternatives. For a simple listing tool, this is adequate but not elevated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ketan_list_beliefsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations available, the description carries the full burden of behavioral disclosure. It indicates a read-only action ('List') and scoping to the session, but it does not explicitly state that the tool is non-mutating, mention return format, or describe behavior when no beliefs exist. It adds some context beyond the name but leaves room for more transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary verb and resource, and includes a clarifying second sentence. It has no redundant filler or unnecessary details, making it highly concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has zero parameters and an existing output schema (per context signals), so the description does not need to explain return values. It covers what is listed, what a belief is, and the session scope. A minor omission is the lack of explicit mention of it being read-only, but for a simple listing tool, the description is largely complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. The description does not (and need not) elaborate on parameter semantics since none exist. The input schema already shows an empty property set, and the description focuses on the tool's action rather than parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'List all active beliefs tracked by the Epistemic Belief Engine.' It uses a specific verb ('List') and resource ('beliefs'), and distinguishes itself from sibling tools like ketan_observe_belief by focusing on listing existing beliefs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you need to see current session beliefs, but it does not explicitly state when to use this tool over alternatives or when not to use it. The context about 'facts observed about the workspace during this session' provides some situational guidance, but not formal exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ketan_observe_beliefA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosing side effects. It explains the contradiction-detection mechanism, which adds context, but it does not state whether the operation is reversible, how conflicts are handled (e.g., error vs. warning), or any persistence semantics. This is a partial disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured: a concise opening sentence, a helpful explanatory second sentence with an example, and a clear bullet-style Args list. Every sentence adds value, and the format is easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with four parameters and an output schema, the description covers the core purpose, parameters, and basic behavioral context (contradiction detection). It omits explicit usage guidelines and details on how contradictions are surfaced, but the output schema likely handles return values. Overall, it is nearly complete for this complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description's Args section provides thorough explanations for all four parameters (subject, predicate, value, confidence) with examples, compensating fully for the 0% schema description coverage. This goes beyond the raw schema, which only lists types and defaults.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool records a factual observation into the Epistemic Belief Engine, with a specific verb ('Record') and resource. It distinguishes itself from siblings like ketan_list_beliefs by focusing on the act of recording rather than querying.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for recording observations to detect contradictions but does not explicitly mention when to use this tool vs alternatives or provide exclusions. Sibling tools such as ketan_check_invariant or ketan_list_beliefs are not referenced, so guidance is only implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ketan_check_invariantA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden for behavioral disclosure. It explicitly states the tool does NOT execute anything, which is a key safety trait. However, it does not disclose potential side effects (if any), prerequisites (e.g., workspace initialization), or what occurs on validation failure. The return behavior is also not described here (though an output schema exists).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-structured: two purpose sentences followed by an Args list. Every sentence adds information, and it is front-loaded with the core behavior. No filler or repeated content from the schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity (2 simple params) and presence of an output schema, the description covers the essentials: what the tool does, how to invoke it, and parameters. However, it lacks broader context about how this fits with sibling tools (e.g., use it before ketan_write_file_safe or ketan_run_bash_safe) and does not mention any prerequisites. This is a minor gap, not a showstopper.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does provide meaningful definitions for both parameters: tool_name is the target tool with an example, and tool_args_json is described as a JSON string of the arguments. This goes beyond the bare schema names, but could further specify JSON format constraints or escaping requirements.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verbs ('Run a pre-flight invariant check', 'Validate a planned action') and clearly identifies the resource (planned tool call). It distinguishes from siblings by highlighting the pre-commit validation role without executing the action, which is unique among tools like ketan_get_status or ketan_write_file_safe.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states a clear context for use: validate before committing to an action. However, it does not explicitly mention when not to use it or alternatives, which would elevate it to a 5. The 'without executing anything' and 'before committing' phrasing imply the appropriate timing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ketan_explain_failureA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that it 'traces the exact execution chain' and returns a 'clean message' when no failures exist, but it does not explicitly state that the operation is non-mutating or mention any potential side effects. For a read-only explanation tool, this is a moderate level of transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three concise sentences: purpose, behavior, and edge case handling. Each sentence adds value without redundancy, and the most important information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters) and the existence of an output schema, the description covers the essential aspects: what it does, how it behaves, and the no-failure case. It lacks an explicit statement of non-mutation, which could be inferred but is not stated, so a slight deduction.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4 as per the rubric. The description adds nothing parameter-related, but no parameters exist to document, so no gap is present.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Get a root-cause explanation of the most recent failure in the CTG.' It uses a specific verb (get) and resource (root-cause explanation), and it distinguishes from siblings like ketan_get_status and ketan_get_checkpoints by focusing on failure explanation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool: when there is a recent failure to explain. It does not explicitly name alternatives or exclusions, but the purpose is unambiguous, which satisfies a clear-context score without explicit comparisons.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ketan_get_ctgA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the output type (Mermaid diagram), the contents of the graph (every tool call, checkpoint, failure, rollback with causal edges), and suggests pasting into mermaid.live for visualization. This gives meaningful behavioral context, though it does not explicitly state read-only safety or potential side effects, which are minor for a get tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the primary purpose. The second sentence adds valuable context about the graph's content and provides actionable advice (paste into mermaid.live). Every word earns its place with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a no-argument tool with an output schema, the description is nearly complete: it explains what the tool does, what the output represents, and how to visualize it. The only gap is lack of explicit guidance on when to use it compared to sibling tools, but that falls under usage guidelines. The tool's simplicity and clear purpose make the description sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. The description adds no parameter semantics since there are none to describe, and no additional parameter information is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description states 'Get the Causal Trace Graph (CTG) as a Mermaid diagram' – clearly identifying the verb (get), resource (CTG), and output format. It also explains the content of the CTG (live DAG with tool calls, checkpoints, failures, rollbacks), which distinguishes it from sibling diagnostic tools like get_status or get_checkpoints.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description provides context that the CTG is a live DAG showing causal edges, implying it should be used for understanding causal relationships. However, it does not explicitly state when to use this tool versus alternatives like explain_failure or session_summary, nor does it mention any exclusions or prerequisites. This is only implied usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ketan_get_statusA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It discloses the exact data points returned, which is useful, but it does not explicitly state that the operation is read-only or non-destructive. However, 'Get' and 'Returns' strongly imply a safe query, making this adequate but not fully transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two succinct sentences, begins with the action ('Get'), and lists the output components without any fluff. Every word contributes to understanding the tool's purpose and behavior, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple getter with no parameters and an output schema, the description is complete enough. It lists the key fields returned and implies overall state reporting. It does not discuss prerequisites or failure modes, but none are obvious, so the description adequately supports the agent's decision to invoke the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema covers 100% of the parameter space (which is empty). The description adds no parameter-specific information, and according to the rubric, a baseline of 4 is appropriate for 0 parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Get' and identifies the resource as the 'current Ketan-OS ground-truth state report.' It enumerates the returned contents (step count, checkpoint count, CTG node/edge counts, failure count, workspace directory), clearly distinguishing it from sibling tools like ketan_get_checkpoints or ketan_get_ctg that would provide more detailed but narrower information.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies this tool should be used when an agent needs an overview of the current Ketan-OS state. The context is unambiguous, but it does not explicitly mention alternatives or exclusions, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ketan_init_workspaceA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must disclose behavior on its own. It mentions creating a KetanHarness that manages transactional snapshotting, invariant enforcement, and causal tracing, which gives some insight. However, it does not disclose potential side effects on the workspace (e.g., writing files, requiring permissions) or whether the operation is idempotent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and well-structured: a clear first sentence, a mandatory usage note, a sentence about what is created, and an args section. Every sentence earns its place with no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema (likely documenting return values), the description adequately covers purpose, usage, and parameter semantics. It could be more complete by mentioning prerequisites (e.g., the workspace must exist) or whether initialization is idempotent, but overall it is sufficient for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides only the parameter name and type with no description (0% coverage). The description fully compensates by defining workspace_path as the absolute path to the project/workspace directory to protect, adding clear meaning and constraints beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool initializes Ketan-OS for a workspace directory and creates a KetanHarness, making its purpose specific and distinct from sibling operational tools. It explicitly identifies the resource (workspace) and the action (initialize).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says this must be called before any other Ketan-OS tools if switching workspaces, providing strong when-to-use guidance. It does not discuss when not to use it (e.g., if already initialized), but the prerequisite nature is made clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ketan_read_fileA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the transparency burden. It discloses a non-obvious side effect ('Also records its existence as a belief'), which is valuable for an AI agent. The read operation itself is clearly implied, and no other hidden behaviors are apparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences, front-loaded with the primary action. Every word adds value, with no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter read tool, the description covers the main behavior, the side effect of belief recording, and parameter semantics. The presence of an output schema handles return value details, so the description is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema only defines 'filepath' without any description. The tool description compensates by specifying that the path is relative to the workspace root and provides a concrete example ('src/main.py'), which is critical for correct invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Read' with a clear resource ('a file from the workspace'). It also uniquely mentions the side effect of recording existence as a belief, distinguishing it from sibling tools like ketan_list_files and ketan_write_file_safe.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for reading files but does not explicitly state when to use it over alternatives such as ketan_list_files. However, the clear purpose and sibling context make the intended use obvious.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ketan_session_summaryA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. 'Get' implies a read-only aggregation, and 'markdown summary' plus the list of included data categories adds useful behavioral context. However, it does not explicitly state whether any state is modified, whether the summary is computed fresh, or any potential limitations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short, front-loaded sentences that immediately state what the tool returns and then list its main content categories. There is no filler or repetition of schema information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter read-only summary tool with an output schema available, the description is complete. It names the output format, the scope of the summary, and the key tracked categories, leaving no significant gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema already fully covers parameter semantics (100% coverage). The description correctly adds no parameter details because none are needed, matching the baseline of 4 for a zero-parameter tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the verb 'Get' with the resource 'complete markdown summary of the Ketan-OS session' and explicitly enumerates the contained items (steps taken, checkpoints created, failures recorded, beliefs tracked). This clearly distinguishes it from sibling tools that focus on subsets like checkpoints or beliefs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'complete ... session' plus the listing of all included components makes it clear this is the tool for a holistic overview, not a specific sub-slice. It does not explicitly name alternative sibling tools or exclusions, so it stops short of a 5, but the usage context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ketan_rollbackA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It explicitly discloses the destructive nature of the operation: 'Reverts ALL filesystem changes (file writes, deletions, renames)' and also notes the performance trait 'in sub-second time.' It does not mention reversibility of the rollback itself or any error conditions, but the core behavioral trait—destructive rollback—is clearly stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and efficient: two sentences covering purpose and scope, plus an Args line for the parameter. It is front-loaded with the primary action and every sentence adds value. No fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that an output schema exists, the description need not explain return values. For a single-parameter destructive tool, it covers the action, the parameter source, and the scope of changes. It could add a warning about irreversibility or the need for a snapshot, but the current description is largely sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema only specifies checkpoint_id as a string type. The description adds meaning by stating that it is 'The checkpoint ID to roll back to' and identifies its source (ketan_get_checkpoints or ketan_snapshot). Since schema coverage is 0%, this compensation is essential and well-executed, though it could provide more detail on format or validation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Time-travel rollback the workspace to a previous checkpoint.' It specifies a concrete action (rollback), a resource (workspace), and the scope ('ALL filesystem changes: file writes, deletions, renames'). This distinguishes it from sibling tools like get_checkpoints or snapshot, as it is the only rollback tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool (to revert filesystem changes after a checkpoint) and even explains where to obtain the required checkpoint_id (from ketan_get_checkpoints or ketan_snapshot). However, it does not explicitly mention alternatives or when not to use it, which keeps it from a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ketan_get_checkpointsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden, and it delivers by specifying what checkpoints contain (workspace filesystem state, conversation prompt stack) and behavior (newest first, atomic rollback). It doesn't mention edge cases like empty lists, but this is a minor omission for a list operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: the first states the core function and ordering; the second explains what a checkpoint is and why it's useful. No wasted words, front-loaded with the action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple zero-parameter list operation and the presence of an output schema, the description fully covers purpose, content, ordering, and use case. No gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. The description adds meaning by explaining the content and purpose of the checkpoints, going beyond the empty schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List'), the resource ('all available checkpoints'), and ordering ('newest first'). It distinguishes this tool from siblings like ketan_snapshot (creates) and ketan_rollback (uses) by focusing on enumeration.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context by explaining that checkpoints enable atomic rollback, implying this tool is used to find rollback points. It doesn't explicitly name alternatives or exclusion criteria, but the zero-parameter interface makes usage straightforward.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ketan_write_file_safeA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully carries the behavioral transparency burden. It discloses the pre-flight AST syntax check, dangerous pattern detection, atomic snapshot, and automatic rollback on failure, which is rich, non-obvious behavior and goes far beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a short numbered list for the safety workflow and a separate Args section. It is front-loaded with the core purpose and every sentence serves a distinct purpose, with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the tool's safety workflow, failure rollback, and required parameters sufficiently. An output schema exists, so returning values need not be detailed. Given the tool's moderate complexity, this is complete and actionable for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description compensates well by explaining both parameters: filepath is 'Path relative to workspace root' with an example, and content is 'Full file content to write.' These definitions add meaningful semantics beyond the bare schema property names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool writes a file to the workspace, with a specific safety/rollback protocol. It distinguishes itself from siblings by emphasizing 'safe, reversible mutations' and references direct file writes as the alternative, making its purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly advises 'Always prefer this over direct file writes for safe, reversible mutations,' providing clear usage context. However, it does not explicitly mention when not to use this tool or name alternative sibling tools (e.g., ketan_run_bash_safe) for other write scenarios, so it lacks full exclusion guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ketan_snapshotA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It clearly states the tool is atomic, creates a checkpoint, and returns a checkpoint ID, effectively setting expectations for behavior and output.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is four sentences, front-loaded with the core action, and every sentence earns its place by adding either purpose, usage guidance, or return value. There is no fluff or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with an output schema, the description provides all essential context: what it does, when to use it, and what it returns. It also references ketan_rollback, linking to sibling tools for a complete workflow.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema coverage is 100% (empty), so the baseline is 4. The description doesn't add parameter-level detail, but that's unnecessary here; it does mention the return value, which is helpful.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Take') and resource ('atomic filesystem snapshot of the entire workspace'), making the tool's function immediately clear. It also distinguishes itself from sibling tools like ketan_rollback and ketan_write_file_safe by focusing on snapshot creation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool: 'Always call this before a risky sequence of operations.' It also frames the tool as a safeguard, referencing rollback as a complementary action, which provides clear contextual guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ketan_run_bash_safeA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and does excellent work: it discloses the pre-flight guard, atomic snapshot, workspace directory execution, and automatic rollback on non-zero exit or exception. This gives agents a full understanding of side effects and safety mechanisms.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a numbered list of safety steps, an explicit preference note, and a concise args section. No unnecessary words; every sentence adds useful information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's medium complexity and presence of an output schema, the description covers all critical behavioral aspects: protection mechanisms, rollback behavior, execution context, and parameter meanings. It is complete and self-contained.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description fully compensates by explaining both parameters: 'command' is a shell command executed in the workspace directory, and 'timeout_seconds' is the max execution time with a default of 30. This adds meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool executes a shell command with protection, using a specific verb ('Execute') and resource ('shell command with FULL Ketan-OS protection'). It distinguishes itself from sibling tools by emphasizing the automatic snapshot/rollback safety features.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly instructs to 'Always prefer this over raw bash for state-changing commands,' naming the alternative and indicating when this tool is appropriate. This is clear actionable guidance for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.