Server Quality Checklist

Tool Scores

• deploy_processC
  2.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It explicitly states that deploying twice is harmless, which is a useful idempotency trait. However, it does not disclose side effects, required permissions, or what 'deploying' entails beyond the name, leaving significant behavioral context unaddressed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no fluff. The primary action is stated first, followed by a single relevant caveat about idempotency. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Although an output schema exists, the description lacks key operational context for a deployment action: prerequisites, effects on existing processes, environment assumptions, and how the bpmn_path parameter influences the deployment. The idempotency note is the only behavioral extra.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has one parameter (bpmn_path) with 0% description coverage, and the description does not mention it at all. The optional nature and default null are not explained, leaving an agent without any understanding of why or how to use the parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Deploy' and names a resource ('the release process') and destination ('the engine'), making the core action clear. It does not explicitly distinguish itself from siblings like start_release, but the artifact named ('release process') differentiates it enough.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no mention of when to use this tool versus alternatives such as start_release or list_processes. The only usage hint is the idempotency note, which implies it can be run repeatedly, but no explicit when/when-not guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• approve_gateB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must carry the full burden of behavioral disclosure, and it falls short. It does not reveal what side effects approving or rejecting might have (e.g., triggering a deployment, sending notifications, or whether the action is reversible). The only extra behavioral hint is that this is the 'human decision,' which is thin. For a mutation tool affecting a release, significantly more transparency is expected.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately short and front-loaded with the action. The second sentence adds a modicum of context without unnecessary verbosity. However, the phrase 'in the process' is slightly vague and could be more informative without adding bulk, keeping it just shy of perfect conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Although the tool is relatively simple (3 flat parameters, no nested objects, and an output schema exists), the description omits any mention of prerequisites, side effects, or the consequence of choosing 'approve' versus 'reject.' For a tool that likely gates the rest of a pipeline, an agent would need more context to use it correctly, such as whether the release proceeds immediately or whether a comment is required in certain cases. It is adequate for very simple usages but not fully comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description should compensate by clarifying parameter usage, but it mentions none of the three parameters (task_id, approve, comment). While the parameter names are self-explanatory, the description adds no meaning beyond the schema, and this is a clear miss given the low coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Approve or reject') and the resource ('a waiting release'), which unambiguously identifies the tool's purpose. It adds a contextual clue ('the human decision in the process') that positions it as the manual approval step, distinguishing it from sibling tools like deploy_process or start_release, though it does not name them explicitly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a release is waiting for approval, but it does not provide explicit guidance on when to choose this tool over its siblings, nor does it mention any prerequisites or conditions. The phrase 'in the process' hints at a pipeline context, but there's no explicit when-not-to-use or alternative routing, so this is only implicitly guided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_runB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions what it reports (state, activity, gate report, variables) but does not disclose whether it is read-only, requires specific permissions, or what happens if the run does not exist. It also does not mention any side effects or performance implications. For a read-type tool, this is a partial disclosure but lacks depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is concise and lists the key content of the report. However, it is slightly terse and could benefit from a bit more detail on the scope, but for a simple tool it is efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there are no annotations, the description must carry the full burden. It identifies the tool's purpose and output components but omits critical context such as whether this is a read-only operation, error conditions for missing/invalid process_instance_id, or when this tool should be used relative to siblings like 'list_processes' or 'engine_status'. An agent may not know if this requires a running process or if it returns historical data.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has zero description coverage; the parameter 'process_instance_id' has no description. The tool description implies this is the run identifier, but does not explain format, source, or how to obtain it. With only one parameter and no schema documentation, the description should at least provide guidance on what constitutes a valid process_instance_id; it does not.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reports the state, current activity, gate report, and variables of a run. This distinctly separates it from siblings like list_processes (listing) or start_release (starting), and it specifies a precise resource ('a run') with a clear action (report). The verb 'report' is specific and the content is enumerated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool is used to fetch details about a run, but it does not explicitly state when to use it versus alternatives. It doesn't mention that it's for a specific run identified by process_instance_id, nor does it exclude scenarios like checking engine status or listing gates. The usage is implied but not formally guided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_gatesB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With zero annotations provided, the description carries the full disclosure burden. It does disclose the single most important behavioral trait — that this tool has no effect on the engine state (read-only in effect), which is significant given the mutation-heavy sibling set. It does not go further (e.g., scope of repo_path, failure behavior), but the key side-effect signal is present.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, zero filler. The first identifies the resource and the non-executing nature; the second reinforces the no-side-effects constraint, which is load-bearing given the sibling mutations. Every word earns its place and the key constraint is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  An output schema exists, so return values need no description, and the no-engine-touch disclosure covers risk context. The main completeness gap is the undefined repo_path parameter, which an agent needs to resolve before calling. For a list-style tool this is otherwise fairly complete, but the undocumented parameter keeps it from being fully self-sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% — the only parameter, repo_path, has just a title 'Repo Path' with no schema description. The tool description adds nothing about the parameter either: it never explains what repo_path should be (absolute path, existing repo on disk, branch/commit reference). For a single-parameter tool with zero schema coverage, the description should compensate but does not.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb 'List' and specific resource 'the gates a release of this repository would run.' The phrase 'would run' signals this is prospective inspection, not triggering a release. The closing 'Does not touch the engine' works with sibling names (engine_status, deploy_process, start_release) to differentiate it from engine-affecting tools, though no sibling is named explicitly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  'Does not touch the engine' gives an implicit when-not: use this when you only want to inspect gates without side effects, rather than for the mutation siblings (start_release, deploy_process, approve_gate). But no explicit alternative is named and no clear 'use X instead when...' guidance is given. The context is implied rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_processesA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry behavioral disclosure. It is extremely minimal, implying a read-only operation but not explicitly stating so, nor mentioning any side effects, ordering, or snapshot semantics. For a simple list tool this might be acceptable, but it leaves ambiguity about whether results are live or cached.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence with zero wasted words. It conveys the core purpose in a clear, efficient manner appropriate for a trivial parameterless tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the zero parameters and an output schema, the description is minimally sufficient for a list operation. However, it lacks any hint at output ordering, pagination, or whether results reflect persisted or in-memory state, which could confuse an agent expecting a specific format or volume.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters and the input schema is empty, which raises the baseline. The description adds no parameter semantics because none are needed; it correctly implies the tool requires no arguments.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('list') and resource ('process definitions') and adds the scope ('the engine currently knows about'). This clearly distinguishes it from siblings like list_gates and engine_status, which operate on different resources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to prefer this tool over alternatives, nor any exclusions or prerequisites. The phrase 'currently knows about' implies a snapshot of engine state, but without explicit comparisons to list_gates or deploy_process, an agent may not know when this is the right choice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• engine_statusA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the behavioral disclosure burden. It communicates a read-only check and version report, but does not explicitly state that no state is modified, nor does it mention error behavior or connectivity caveats.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, information-dense sentence covers the action, resource, and expected output without any filler words. It is well front-loaded and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there are no parameters and an output schema exists, the description sufficiently covers what the agent needs to invoke this tool correctly. It would benefit from a brief note about whether any environment context is required, but for a simple status check this is minimally sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema leaves nothing unexplained. The description's reference to checking reachability and version provides enough semantic context for a parameterless invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies a specific verb ('check') and resource ('local CIB seven engine'), and states the additional output of reporting its version. This distinguishes it from sibling tools like deploy_process or approve_gate, which target different actions and resources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The purpose is clear enough to imply when this tool is appropriate: when an agent needs to verify engine reachability or discover its version. However, it does not explicitly state when not to use it or mention any alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• start_releaseA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full disclosure burden. It discloses that the tool runs quality gates, waits for human approval, and that dry_run default prevents tagging/publishing. It could go further on approval mechanics and permissions, but the core behaviors are clear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short, purposeful sentences each add distinct value: what the tool does, its workflow, and the dry_run safety caveat. There is no filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a small-parameter tool with an output schema and no annotations, the description covers the key operational details: quality gates, human approval, and dry-run safety. It falls slightly short on explaining how approval is provided or how to know when the run is waiting.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema describes params with 0% textual description coverage, so the description must compensate. It does clarify dry_run strongly, but repo_path and version receive no additional format, constraint, or semantic context beyond their names and titles.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The first sentence states a specific verb and resource: start a release run for a repository. The second sentence adds what that run does (executes quality gates, waits for human approval), distinguishing it from siblings like deploy_process and approve_gate.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description conveys a clear usage context: start a release run first, quality gates run, then the run waits for approval. However, it does not explicitly name when to prefer this tool over deploy_process or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.