Server Quality Checklist

Tool Scores

• pleach_statusB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It does not state whether the tool is read-only, whether it has side effects, or any prerequisites. The description only lists the data categories without clarifying the operation's safety profile.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no filler or redundant wording. Every word contributes meaning, making it highly concise and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description lists the key data categories returned (session/fork counts, live sessions, edges), which gives the agent a good idea of what to expect. However, it does not specify the output format or response structure, and since there is no output schema, the description leaves some ambiguity. Still, for a simple status tool with one optional parameter, this is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides full documentation for the only parameter 'cwd' (with a clear description and optionality). The description adds no information about parameters, so the baseline score of 3 is appropriate given complete schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description identifies the resource (pleach tracking data) and lists specific content (session/fork counts, live sessions, observed vs inferred edges), giving a clear idea of what the tool reports. However, it lacks an explicit action verb like 'get' or 'show', relying on the noun phrase 'What pleach is tracking'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like pleach_log or pleach_summary. It only states what the tool reports, without any context on selection criteria or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pleach_nameB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It does disclose that labels show in the graph, but it does not mention whether labeling is idempotent, overwrites existing labels, or requires the session to exist. For a mutation tool, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that immediately states the action and effect. There is no filler or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple, but the description omits behavioral context such as overwrite behavior, session existence requirements, or relationship to other tools. The lack of annotations and output schema means the description should provide more context, but it remains minimal.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema describes 'cwd' and 'session' but leaves 'label' undocumented. The description adds that labels show in the graph, giving purpose to the label parameter, but it does not clarify format or constraints. With 67% schema coverage, the description should compensate more for the missing label description, but this is partial credit.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Label' and resource 'session', and states the effect 'labels show in the graph.' This distinguishes it from sibling tools like pleach_log and pleach_show, which have different actions and resources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use (when you want to label a session for graph display) but provides no explicit when-to-use or when-not-to-use guidance, nor does it mention alternatives among sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pleach_logA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description is the sole source of behavioral info. It describes the output format ('ASCII git-style graph plus one record per session...') and ordering ('newest first'), which is helpful. However, it doesn't explicitly state that the tool is read-only or whether it has any side effects, though the 'log' name and output description imply a safe listing operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core output, and every sentence adds value. It avoids repetition of schema details and is appropriately sized for the tool's scope.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description appropriately summarizes the return value: an ASCII graph plus records with fields listed. It covers the operation's breadth (listing sessions) and ordering. Given the tool's moderate complexity, this is nearly complete; a minor gap is that it doesn't elaborate on what the graph looks like or what 'live' means in the records.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the parameters (cwd, limit, all_projects) are already fully documented. The description only adds 'newest first' which reinforces the limit ordering but doesn't explain parameter usage beyond what the schema provides. Thus the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool produces an ASCII git-style session branch graph and per-session records, with the instruction 'Start here to see what branches exist.' This gives a specific verb and resource, and positions it as the overview tool among siblings, though it doesn't explicitly differentiate from pleach_branch or pleach_status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'Start here to see what branches exist' provides clear context for when to use this tool: as an entry point for exploring sessions. It implies a first-stop purpose but doesn't mention when not to use it or name alternative tools explicitly, which separates it from a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pleach_branchA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral disclosure burden. It explicitly states that the operation copies the transcript under a new id, indicating non-destructive behavior, and discloses the return value (session id and resume command). This is meaningful context beyond a simple 'fork' statement, though it omits potential failure modes or prerequisites.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that leads with the core action, includes a precise parenthetical clarifying the copy behavior, and states the output. Every word contributes, with no wasted content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one required parameter and no output schema, the description provides the essential context: what it does, how it behaves (copies transcript), and what it returns. Minor gaps include potential branch naming collisions or behavior when the session does not exist, but these are not critical for basic use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers all three parameters with descriptions (cwd, name, session), so the schema is doing the heavy lifting. The description adds no parameter-specific meaning beyond what the schema already provides, placing it at the baseline 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb+resource: 'Fork a session into a new tracked branch' and further clarifies the mechanism ('copies the transcript under a new id'). This clearly distinguishes it from sibling tools like pleach_log, pleach_show, and pleach_grep, which are read-only operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies when to use the tool: when you want to branch a session into a new tracked copy. However, it provides no explicit exclusions or alternative tool references (e.g., when to prefer pleach_merge or pleach_name), so it stops short of full guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pleach_exportA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses that tool dumps are elided, transcripts are truncated at max_chars, and the output is markdown. This is meaningful behavioral context, though it doesn't specify how truncation is signaled or how output is delivered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two front-loaded sentences deliver purpose and key limitations with no waste. Every word earns its place, including the alternative tool recommendation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema or annotations, the description gives sufficient context: output format, elision, truncation, and when to use a sibling. Minor gaps include absence of explicit return delivery method, but the core behavior is well covered.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description reinforces the truncation behavior tied to max_chars but adds no substantive meaning beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool renders a session's conversation as readable markdown with tool dumps elided. It distinguishes from the sibling pleach_summary by contrasting truncation with whole-branch digest.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly recommends pleach_summary for whole-branch digests, providing a clear alternative for a different use case. It also notes the local/free nature, but lacks broader when-to-use or when-not-to-use guidance beyond that.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pleach_grepA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that search is over conversation text, not tool dumps, and that results are returned newest-first with snippets. It does not mention auth, rate limits, or side effects, but for a search tool this is reasonable context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two dense sentences deliver the core purpose, scope, an example, and output behavior with zero filler. The description is front-loaded with the main verb and resource.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 5 parameters and no output schema, the description explains what it returns (sessions with snippets) and ordering (newest-first). It lacks edge cases like empty results or handling of all_projects/case_sensitive, but the overall context is sufficient for basic use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 60%, with descriptions for cwd, term, and limit. The description does not add further parameter semantics; all_projects and case_sensitive remain unexplained. Since coverage is moderate, the description should compensate, but it does not provide additional detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Full-text search across session transcripts' and clarifies scope by saying 'conversation text, not tool dumps'. It even gives an example question it answers, which clearly differentiates it from sibling tools like pleach_log or pleach_show.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides an explicit example use case ('which branch was the one where we worked out X?') implying when to use it, and adds an exclusion ('not tool dumps'). However, it does not name alternative tools or state when not to use it beyond that exclusion.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pleach_merge_planA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full disclosure: it states 'Dispatches nothing and writes nothing', notes that summaries are cached, and mentions small cost. This is strong behavioral transparency for a read-only dry-run operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core purpose, and each clause adds meaningful detail (dry-run, outputs, safety guarantees, usage hint). No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema or annotations, the description covers what it does, what it doesn't do, what it returns, and how to use it in sequence. It doesn't detail error cases or exact return structure, but those are not essential for a dry-run planning tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with all parameters described. The tool description does not add extra semantic detail beyond the schema's minimal 'Session id or prefix' for branch_a/branch_b, but it's already clear from context. Baseline of 3 applies when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action ('Dry-run'), identifies the resource ('a merge of two branches'), enumerates the outputs (branch point, merge-base method, confidence, exact prompt), and implicitly distinguishes itself from the sibling tool 'pleach_merge'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly instructs 'Review this before calling pleach_merge', creating a clear sequential workflow. While it doesn't formally list exclusions or alternative scenarios, the 'before' guidance implies its place in the tool pipeline.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pleach_showA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosing behavior. It explicitly lists the type of information returned (lineage, transcript path, entry count, merges, resume command), which implies a read-only inspection tool. It does not mention error behavior or permissions but is substantially transparent for a show tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that is front-loaded with the main purpose and lists specific details without unnecessary words. Every element earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With two simple parameters and no output schema, the description adequately explains what the tool returns and the scope of one session. It could mention potential error conditions or permission requirements, but given the tool's simplicity, it is largely complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the parameter semantics are fully documented in the schema. The description adds no additional parameter information, but it does not need to; baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool shows full detail for one session, enumerating specific content (lineage, transcript path, entry count, merges, resume command). This distinguishes it from sibling tools like summary or status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'Full detail for one session' clearly indicates when to use this tool, but it does not explicitly mention alternatives or when-not-to-use conditions. Context is clear, though no exclusions are stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pleach_summaryA
  4.2/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully carries the behavioral transparency burden. It discloses caching per branch head, the cost implications of a cache miss (model calls), and the capability to handle any transcript size, which are valuable behavioral traits beyond the basic purpose.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the purpose and follows with behavioral/cost details. Every word contributes value, with no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema and no annotations, the description covers purpose, caching behavior, cost, and scale, which is quite complete. It could mention what the output looks like or any prerequisites, but the purpose is clear enough for an agent to infer the return value.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all parameters are explained in the schema. The description adds context about caching and cost that relates to the no_cache parameter, but it does not elaborate on individual parameter semantics beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb ('Summarize') and defines the resource as a branch's attempt outcomes. It distinguishes itself from siblings like pleach_log (raw log) and pleach_status (current state) by focusing on attempted, worked, failed, and decided aspects.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for obtaining a concise summary of branch activity, and notes it works at any transcript size, but it does not explicitly state when to use this tool over alternatives or provide exclusions. No sibling tools are named as alternatives, so guidance is only implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pleach_mergeA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses the side-effectful nature ('SIDE-EFFECTFUL'), background execution, merge-base materialization, and the default read-only behavior. This gives the agent essential safety and behavioral context beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two dense, information-rich sentences with no redundancy. Front-loaded warning and clear action guidance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While it covers side effects, background dispatch, read-only default, and preconditions, it omits expectations for the agent's output or completion status. For a complex, side-effectful tool with no output schema, a bit more detail on the result or failure modes would round it out.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers only cwd and allow_writes; branch_a and branch_b lack descriptions. The description mentions 'two branches' but does not clarify their roles or ordering, leaving ambiguity. It adds no meaningful semantics beyond the schema for the required parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool's function: materialize a merge base and dispatch a background agent to reconcile two branches. It distinguishes from sibling pleach_merge_plan by explicitly positioning this as the execution step after planning, with a different scope and side-effect profile.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs to use pleach_merge_plan first and confirm with the user before calling, providing clear sequencing and an alternative. It also specifies the read-only default unless allow_writes is set, clarifying when to enable writes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.