Server Quality Checklist

Tool Scores

• propose_fixesA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It does not explicitly state that the operation is read-only (though 'returns' implies it), nor does it explain prerequisites like requiring a prior scan or any side effects, leaving the agent with ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, front-loaded with the verb, and every word earns its place. There is no redundancy or wasted language.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is adequate for a simple, parameterless tool, but it leaves gaps: it doesn't define what constitutes a 'current scan', doesn't describe the structure of returned fix candidates, and doesn't connect to the apply_fix sibling workflow. Given the absence of an output schema and annotations, this is a notable gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has zero parameters, so coverage is trivially 100%, but the baseline is 4. The description adds meaningful context by mentioning 'current scan', indicating that the tool relies on the scan context rather than explicit arguments, which is valuable beyond the empty schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool returns exact low-risk fix candidates from the current scan, using the specific verb 'returns' and a distinct resource. It differentiates from siblings like apply_fix by focusing on proposal/generation rather than application.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus its siblings. The mention of 'current scan' implies a prerequisite but does not explicitly say to use it after doctor_scan and before apply_fix, nor does it mention any alternatives or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• doctor_scanA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing behavioral traits. It only mentions scanning and returning ranked findings, but does not state whether the operation is read-only, what permissions are needed, or how the findings are structured. This leaves significant unknowns for an agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads the action, scope, and result. Every word adds value, with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool without an output schema, the description provides a good high-level overview but lacks details about the return value (e.g., what constitutes 'ranked findings' or severity levels). It also omits any prerequisites or environment context. This leaves the description moderately complete but with gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the description does not need to elaborate on parameter usage. The baseline of 4 applies for no-parameter tools, and the description correctly provides no parameter-related information that would be redundant or missing.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action ('scans') and lists the specific targets (txAdmin status/logs, server.cfg, resources, manifests, dependencies) along with the outcome ('returns ranked findings'). This distinguishes it from siblings like resource_inspect, which focuses on individual resources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a comprehensive diagnostic scan but does not explicitly state when to use this tool versus alternatives. There are no usage exclusions or mentions of when to prefer resource_inspect or propose_fixes. The context is clear from the title and scope, but not spelled out.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• resource_inspectA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries a heavier burden. The verb 'inspects' implies read-only, but the description does not explicitly state that nothing is modified, nor does it mention return formats, error behavior, or permissions. It adds some context by listing what is inspected (manifest, dependencies, file evidence) but stops short of full transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, tightly worded sentence that gets straight to the point. Every word contributes meaning, with no filler or redundancy. It is appropriately sized for a simple tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (1 parameter, no output schema, no annotations), but the description could be more complete. It lists what it inspects but does not explain the nature of the returned evidence or that the operation is read-only. Without an output schema, the agent relies on the description to set expectations for results, which is somewhat vague.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with a clear description of the 'name' parameter ('Resource folder name, for example oxmysql'). The tool description adds little beyond the schema, only reiterating 'one resource folder' which is already implied by the singular parameter. Baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'inspects one resource folder' and specifies what it covers: manifest, dependencies, and related file evidence. This specific verb+resource+scope distinguishes it from siblings like doctor_scan (broader scan), propose_fixes, and apply_fix.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for detailed inspection of a specific resource, but it does not explicitly state when to use this tool versus alternatives. It lacks explicit 'when' or 'when-not' guidance, relying on the tool name and sibling context to convey its niche.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• apply_fixA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It reveals that this is a write operation with safety checks (writes enabled, hash match, root path), implying it will not apply under other conditions. However, it does not explicitly state failure modes, reversibility, or what happens if checks fail.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the core action and key conditions. It is concise with no unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given two parameters, no output schema, and no annotations, the description provides a high-level overview but omits critical operational details like what fixId refers to, the expectedHash parameter's role, and potential return values. It is adequate as a starting point but not fully complete for an agent to invoke the tool confidently.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero description coverage, and the description does not explain either parameter. It mentions 'file hash still matches' which loosely relates to expectedHash, but fixId is never explained. This leaves the agent guessing about parameter purpose and format.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool applies a generated low-risk fix, using a specific verb ('applies') and resource ('generated fix'). It distinguishes itself from siblings like propose_fixes by indicating this is the execution step, not the proposal step.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly provides conditions for use: 'only when writes are enabled, the file hash still matches, and the target path is under a configured root.' This gives clear guidance on when the tool can be used, though it does not explicitly name alternatives or when not to use it beyond these conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.