Server Quality Checklist

Tool Scores

• reelier_pushA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses several behavioral traits: requires an apiKey, reports skipped-no-key when absent, always resolves cloud URL, and on first push also uploads the skill file. It does not cover all side effects (e.g., cursor behavior, state changes) but is fairly transparent for a push operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, fitting key points into a single paragraph with a clear 'USE WHEN' signal. It avoids redundancy but could be slightly more streamlined by merging the apiKey note into a single sentence. Overall, it is well-structured and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 5 parameters and no output schema, the description covers the main purpose and usage context but lacks details on parameter-specific behavior (e.g., how 'all' resets cursor, dryRun impact) and output format. It is adequate but not fully complete for a complex tool with multiple options.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description does not add meaningful information about parameters beyond the schema; it mentions the apiKey requirement but not how parameters like 'all', 'dryRun', or 'withSkill' affect behavior. Thus, it adds minimal value over the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Push a skill's local run records... to your receipt ledger') and the result ('shareable permalink + verified-replay badge'), with a specific verb and resource. It distinguishes from sibling tools by focusing on run record pushing, while siblings like scan, replay, from_session, and diff serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes a 'USE WHEN' clause ('a run's receipt should be durable or shareable') that provides clear context for when to invoke the tool. It also notes the apiKey requirement, but does not explicitly compare to sibling tools or state when not to use it, slightly reducing the score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reelier_replayA
  4.1/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It thoroughly explains Level 0 replay, zero LLM calls, reproduction of tool calls only, read-only default, and handling of idempotent-write steps. It also discloses that it never fabricates a pass and reports drift reasons.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is generally concise and front-loaded with the main action. It uses multiple sentences but avoids excessive verbosity. Could be slightly tighter, but still efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema, but description clearly states what the return value includes: per-step outcomes, timing, totals, and failure reasons. It covers key aspects like templating, read-only safety, and idempotent-write handling. Minor omission of error scenarios.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so schema descriptions exist for all parameters. The description adds context for vars (templating), allowWrites (read-only default), and wrap (MCP server connections), but these are already clear from schema. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs a skill file at Level 0 deterministic replay, returning the run record. It specifies the verb 'run' and resource 'skill file'. It distinguishes from siblings by noting it does not schedule itself and is read-only by default, but does not explicitly compare to reelier_scan or reelier_push.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides guidance on when to use: for deterministic replay with zero LLM calls, and mentions pairing with cron/CI for recurring runs. It explains how to pass templated vars and the read-only nature with allowWrites option. However, does not explicitly differentiate from sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reelier_scanA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses that native file/shell actions are reported as skipped and never fabricated, which is a key behavioral trait. Additional details like side effects or auth needs are absent but acceptable for a scan tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences plus a 'USE WHEN' clause, all front-loaded and free of fluff. Every sentence contributes essential information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has one parameter, no output schema, and is a straightforward scanner, the description fully covers purpose, behavior, and usage guidance. It explains return values (transcript paths) and even suggests the next step (feed to reelier_from_session).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with one parameter 'dir'. The description adds value by noting the default value (~/.claude/projects) and clarifying the context of scanning agent session transcripts, going beyond the schema's basic description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool discovers replayable workflows in session history and returns transcript paths. It specifies the default directory and explicitly names a sibling tool (reelier_from_session) for follow-up, distinguishing its purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes an explicit 'USE WHEN' statement: deciding which past work to compile into a replayable skill. It also clarifies that only MCP/HTTP tool-call sequences are replayable, guiding proper use, though it does not explicitly state when not to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reelier_diffA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully discloses behavior: it reads from .reelier/runs/<skill>.jsonl, compares per-step outcomes, reports SAME or DRIFTED with reasons, and does not mutate state. It also explains limitations (replay cannot reveal model changes). This is comprehensive for a read-only comparison tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-organized paragraph. The first sentence captures the core purpose, followed by details on comparison criteria, usage example, and edge case. Every sentence serves a purpose without redundancy, making it highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description explains the return values (SAME or DRIFTED with reasons). It covers file paths, default behavior, legitimate change handling, and when the tool cannot be used (fewer than two runs). For a tool with 4 parameters and no annotations, this is remarkably complete and leaves no critical gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers all 4 parameters with descriptions (100% coverage), so baseline is 3. The description adds value by explaining defaults (baselineIndex and candidateIndex default to last two runs) and the file path pattern. It also gives contextual usage examples, raising the score above baseline but not to 5 since schema already does most work.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with a clear verb ('Compare two runs') and specific resource ('same skill'), immediately distinguishes from siblings by calling itself a 'drift-detector'. It outlines output ('SAME or DRIFTED') and mentions comparison dimensions (per-step outcomes, structure, heal-level), making purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use guidance (e.g., 'To check a MODEL upgrade'), explains defaults ('defaults to the last two runs'), and warns when not to use ('Honest when there aren't two runs yet'). It also clarifies what constitutes drift vs. legitimate changes, leaving no ambiguity about proper invocation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reelier_from_sessionA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses return values (skill path, stats, open questions, or indication of nothing replayable), the behavior with relative time windows (binding to date variable), and that openQuestions flags this. It also notes the 'force' parameter for overwriting.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Front-loaded with purpose and usage. Each sentence adds value, though slightly verbose. Could be tightened slightly, but overall well-structured and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema, but description explains all return values. Covers required parameter source, common pitfalls (time window binding), and provides complete context for a tool with 4 parameters and no annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, baseline 3. Description adds significant value: explains transcriptPath should come from reelier_scan, default output path, force overwrite, name as filename stem, and the openQuestions tip about relative time windows. This goes well beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool compiles a SKILL.md from a session transcript, using a specific verb ('Compile') and resource ('SKILL.md'). It distinguishes from sibling tools like reelier_scan and reelier_replay by specifying it's for work that already happened, not for live recording.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use ('work ALREADY happened') and when not to use ('NOT for work that hasn't happened yet'). Advises to get transcriptPath from reelier_scan first, providing clear context and exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.