Server Quality Checklist

Tool Scores

• check_repoB
  3.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description explicitly notes that the tool is side-effecting and always persists a ReportFile, which is critical behavioral information, and no annotations are provided to cover this. It also mentions that the return value may be a summary when large, adding transparency. This is helpful but does not go into deeper details like exact file paths or side effects on the environment.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded with the main action and return type. The mention of side effects and persistence is valuable. However, one sentence might be dense, potentially combining multiple ideas, but it remains clear and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 4 parameters with zero schema descriptions, no output schema, and no annotations, the description is incomplete because it does not explain parameter usage or the full behavior. However, the mention of side effects and large output handling adds some completeness haven't fully addresses the parameter gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema coverage is 0%, so no parameters are described in the schema. The description mentions 'working tree or staged diff' but does not explicitly connect these to parameters like 'since' or 'staged'. The enum for threshold is not explained, and the meaning of 'task' and 'since' remains unclear, so the description fails to compensate for the lack of schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: run safety checks against working tree or staged diff and return a report. It also distinguishes from siblings like classify_risk or list_risky_files by mentioning the ReportFile output and the scope of checks. However, it omits any detailed scope of what the safety checks actually include.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The tool is described as running safety checks, which implies it should be used to assess code changes for risks, but there is no explicit guidance on when to use it versus alternatives like classify_risk or list_risky_files. The description does not mention exclusions or alternative tools, so while the context is somewhat clear, it lacks explicit direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_fix_promptA
  3.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description explicitly discloses side effects: 'Side-effecting (class B per D99.V); writes the sibling fix-prompt.txt via atomic temp+rename.' It also transparently explains output truncation behavior: 'the MCP wire response's prompt_text is capped at 262144 bytes (D99.U) -- when truncated, the response carries truncated:true + bytes_omitted.' This goes beyond typical annotations and fully informs about side effects and output limitations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but well-structured, starting with the main purpose, then the return values, then side effects, and finally truncation details. Each clause adds necessary information, and while it is somewhat long, it avoids redundancy and is appropriately sized for the complexity of the behavior it describes.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers behavioral aspects thoroughly (side effects, truncation) and mentions return values, but it lacks explanation of the input parameters and when to use the tool. Given the simple schema (two string fields) and no annotations, the description is moderately complete but leaves gaps in parameter semantics and usage context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description does not relate the input parameters (`report` and `session`) to their meaning or usage. It mentions 'from a VibeRevert check report' but never explicitly states that the `report` parameter is the report ID or that `session` refers to a session. This leaves the schema fields completely unexplained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Render a fix-prompt from a VibeRevert check report.' It uses a specific verb ('Render') and a specific resource ('fix-prompt'), and distinguishes itself from sibling tools by focusing on fix-prompt generation from check reports.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide any guidance on when to use this tool versus alternatives. It lacks context or prerequisites, such as 'use when a check report is available' or comparisons to other tools like create_checkpoint. The only usage hint is implicit in the tool's purpose, but no explicit direction is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• explain_diffA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It explicitly states the tool is read-only and does not run new checks, which covers safety and side effects. It also references a class A security classification (per D99.V), adding context. However, it does not describe potential errors, return format details, or dependencies beyond reading a report.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences that efficiently convey the core function and key behavioral caveats. It is front-loaded with the primary purpose and contains no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and 0% parameter coverage in the description, the tool is incomplete for an agent to use correctly. The description covers the primary action and read-only nature but lacks parameter semantics and output expectations. For a simple read-only tool, it might suffice if parameters were self-explanatory, but they are not.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description provides no explanation of the parameters (report, session, threshold). Schema description coverage is 0%, and the schema itself has no descriptions. The tool requires three parameters with no guidance on their meaning or format, making correct invocation difficult.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Render') and resource ('CommonMark explanation of a check report'), clearly indicating the tool's function. It also distinguishes itself from siblings by stating it does not run new checks, differentiating it from check_repo and similar tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by stating it reads an existing ReportFile and is read-only, which suggests it should be used after checks have been run. However, it does not explicitly name alternative tools or provide a detailed when-to-use vs. when-not-to-use, but the 'does not run new checks' caveat gives clear guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_risky_filesA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosure. It explicitly states the tool is read-only (class A per D99.V) and capped at 500 entries (per D99.U), which are important behavioral constraints. It also discloses the sorting behavior, adding value beyond what annotations would typically provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is only two sentences, with the primary function front-loaded in the first sentence and constraints in the second. Every word adds value, no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema or annotations, the description does a good job of specifying the return format (list of fields) and sorting, but it leaves the 'session' parameter unexplained and provides no usage context relative to sibling tools. The references to D99.V and D99.U are opaque without external knowledge, creating additional ambiguity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. While 'report' is implicitly clarified as the 'check report', the 'session' parameter is never explained, leaving its purpose and relationship to the report ambiguous. This is a significant gap for an agent trying to invoke the tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: projecting a check report into a per-file risky-files list with explicit output fields (path, max_severity, finding_count) and sorting order. This specific verb+resource+output format differentiates it from sibling tools like check_repo or classify_risk.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The context implies the tool is used after a check report exists ('Project a check report'), but it does not explicitly state when to use this tool over alternatives, nor does it provide any when-not-to-use guidance. No alternative tools are named.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• classify_riskA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Explicitly states 'Read-only' and 'no threshold input', providing key behavioral context. Since no annotations exist, the description carries the burden and addresses safety and side-effect concerns adequately, though other traits (e.g., performance) are omitted.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two succinct sentences with no redundancy. The structure flows logically from purpose to behavioral notes, making it easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description adequately describes the return (per-level histogram) and mentions the lack of threshold filtering. It covers core functionality, though parameter details are missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% coverage for parameters 'report' and 'session'. The description hints that 'report' is a check report but does not explain 'session' or clarify expected formats, leaving significant ambiguity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb+resource: 'Project a check report's findings into per-severity counts' precisely describes the operation and output. Distinct from siblings by its focus on severity histograms.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implicitly indicates when to use (when severity counts are needed) and mentions 'no threshold input' to clarify scope, but does not explicitly contrast with alternative tools like 'list_risky_files'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_policyA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and handles it well: it explicitly declares 'Read-only (class A per D99.V); does not run checks or mutate state' and discloses the defaults-application behavior ('Applies M C defaults (D57) before returning'). This adds safety and resolution context beyond any structured annotation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three tight sentences with zero waste: first sentence states the verb+resource+contents, second explains the resolution behavior, third covers the safety profile. Each sentence earns its place; the description is front-loaded with purpose and contains no redundant filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-param, read-only tool with no output schema, the description is nearly complete: it lists the return content, the defaults-resolution behavior, and the safety profile. The only minor gap is that it doesn't describe the exact output structure/format of the returned policy slice, but for this simple tool the component list suffices.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the rubric baseline is 4. The description compensates by describing the return-value content (the policy components), which gives the agent a clear picture of what it will receive even without an output schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb+resource ('Return the project's resolved policy slice from .viberevert.yml') and enumerates the exact contents (risk thresholds, check toggles, frameworks, rollback exclude patterns). It clearly distinguishes from siblings — none of create_checkpoint, check_repo, explain_diff, classify_risk, or list_risky_files describe retrieving a resolved policy.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context — it's a policy-lookup tool that's safe because it doesn't run checks or mutate state — but it never explicitly states when to choose this vs alternatives, and no sibling tool is named for comparison. The guidance is implicit rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• start_sessionA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full weight. It does an excellent job of disclosing side effects: it states it is 'side-effecting (class B per D99.V)', writes session state, and uses the 'D22 start lock' during execution. It also specifies what resources are affected (checkpoint, git status, lock) and provides a reference to a classification system (D99.V) for risk assessment.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, three sentences, and front-loads the primary action. It efficiently packs essential details: what it does, what it returns, and what side effects to expect. Every sentence adds value, from the core function to the side-effect warning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (multiple internal steps: checkpoint, git status, lock) and lack of output schema, the description covers the key aspects. It explains the side effects, returns, and even classifies the side-effect level. It could potentially specify what happens if the lock is already held, but that's a minor omission given the details provided.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has one parameter 'task' with only a pattern requiring non-space characters. The description doesn't explicitly define 'task', but it mentions that the session creation is the main action. Since there's only one parameter and it's not described in the schema beyond the pattern, the description could have hinted at what 'task' means. However, given the schema coverage is 0%, the description's silence on 'task' is a minor gap, but the low complexity (1 param) mitigates the impact. The description adds value by explaining the side effects but not the parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: starting a VibeRevert session. It specifies the concrete actions (create inner checkpoint, capture git status, acquire lock) and the return values (session id, checkpoint id, start timestamp). While the sibling tools are listed, the description itself includes enough specificity to distinguish it from potential siblings like create_checkpoint.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool (start of a session) and notes it acquires a lock, which hints at concurrency management. It doesn't explicitly say when not to use it or name alternative tools, but the context of session initialization and the lock acquisition gives clear usage context. The mention of 'start lock' suggests a prerequisite or constraint.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_checkpointA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully carries the burden of disclosing side effects. It explicitly states the operation is side-effecting (class B), writes under '.viberevert/checkpoints/', and explains the name-uniqueness lock behavior. It also discloses the return values (id and timestamp). This is a high level of transparency for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose, then quickly covers returns, side effects, and conditional behavior. Every word earns its place; there is no redundancy or filler. The structure is logical: purpose → output → side effect → conditional nuance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with a single optional parameter and no output schema, the description is complete. It covers the primary action, the return format (id and timestamp), the location of side effects (checkpoint directory), and the lock behavior under a specific condition. No critical information is missing, such as error cases or prerequisites, which are not needed given the simplicity of the operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no description for the 'name' parameter (0% coverage), but the description compensates by explaining that supplying 'name' triggers the D22 name-uniqueness lock, implying that 'name' is the checkpoint name and affects uniqueness. This adds meaningful semantics beyond the schema's pattern and maxLength constraints, though it could be more explicit about the checkpoint naming purpose.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Create'), the resource ('a standalone VibeRevert checkpoint'), and the specific action ('capturing the current working tree state'). It also mentions the returned values (checkpoint id and creation timestamp), making the purpose unambiguous. The term 'standalone' differentiates it from potential sibling operations like start_session or check_repo, though none are directly similar.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on usage: it's a side-effecting operation (class B) that writes a checkpoint directory, and it details the conditional lock behavior when 'name' is supplied. While it doesn't explicitly state when to use it versus alternatives, the sibling tools are distinct and no direct alternative exists. The lock condition effectively guides parameter usage (when to supply 'name' and when not).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.