Server Quality Checklist

Tool Scores

• get_pipelineA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does explain that the tool returns names, states, exit codes, and lists failed step ids in 'failed_step_ids', plus suggests fetching step logs with get_step_logs. However, it does not mention safety (read-only), error behavior, or any other side effects, leaving some transparency gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that states the main purpose immediately, followed by a useful pointer to get_step_logs. Every word earns its place, with no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity and lack of output schema, the description adequately covers the main return content (workflows, steps, states, exit codes) and the failed_step_ids field, plus a cross-reference to another tool. It does not detail the exact shape of the response or error cases, but for a read-only get operation this is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema already fully documents each parameter (instance, repo, branch, number). The description adds no additional parameter-specific meaning beyond what the schema provides, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'Gets one pipeline with its workflows and steps (names, states, exit codes)', which is a specific verb+resource combination. It also distinguishes itself from sibling tools like list_pipelines by focusing on a single pipeline and its internal details, and points to get_step_logs for failed step output.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you need detailed pipeline information including workflows and steps, and references get_step_logs for following up on failed steps. However, it does not explicitly state when to use this tool instead of list_pipelines or provide exclusions, so usage guidance is only implied rather than explicitly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_step_logsA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden and does well: 'Fetches' indicates a non-destructive read, and 'Returns the last tail lines' discloses the output limiting behavior. It does not cover error handling or auth, but for a read-only log tool this is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences, front-loaded with the main purpose and a useful behavioral detail. Every word earns its place, with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read tool with fully documented parameters, the description covers the core action and the tail output limitation. Without an output schema, it could specify the log format, but 'log output' implies plain text, and the edge cases are minor for this use case.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds minimal new meaning beyond the schema—it repeats the get_pipeline dependency for step_id and mentions tail, but the schema already describes each parameter in detail, including defaults and max values.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool fetches log output for one pipeline step, using a specific verb and resource. It also distinguishes itself from siblings by referencing get_pipeline for step IDs, making clear it's for step-level logs rather than pipeline-level information.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by noting step IDs come from get_pipeline, but it does not explicitly state when to use this tool versus alternatives like get_pipeline. It gives a prerequisite but no exclusions or direct comparison.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_instancesA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description adds useful behavioral context: the check parameter verifies reachability and token validity. It doesn't mention side effects, but for a read-only list operation, no side effects are expected.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary purpose and followed by a concise usage tip. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with one optional parameter; the description explains the core behavior and the check option well. Return format is implied by 'lists' but not explicitly described, which is a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already fully describes the only parameter (check) with a clear explanation, so the description adds minimal new semantic detail beyond the schema. Baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Lists the configured Woodpecker CI instances' with a specific verb and resource, distinguishing it from sibling tools that operate on repos, pipelines, and logs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not explicitly state when to use this tool versus alternatives, but the purpose as a listing operation is clear from the name and sibling context, so usage is implied. It does provide usage guidance for the check parameter.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_pipelinesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description must disclose behavior on its own. It adds 'newest first' ordering and the set of filter dimensions. But it omits pagination behavior (though schema covers page/per_page), what 'recent' means, and doesn't explicitly state read-only/non-mutating nature.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no filler. First states purpose, second adds filter guidance and a concrete example. Ideal size.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a list tool with 7 params but no output schema or annotations, the description covers the core operation, ordering, and filters. It falls slightly short of fully complete because 'recent' is vague and there's no mention of return format, but the schema and sibling names cover much of the gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% – every parameter has a description. The description only mentions branch/event/status filters, which are already documented in schema, without adding new semantic detail. Baseline 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Lists recent pipelines for a repository, newest first' – a specific verb, resource, and ordering. The filter guidance and example clarify it targets the list-level operation, distinguishing it from get_pipeline (single pipeline) and list_repos (repos).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Gives context for when to use: filtering by branch/event/status to find 'CI runs for a pull request branch.' However, it doesn't explicitly compare to sibling tools like get_pipeline for single-pipeline lookups, so no exclusions or alternatives are named.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_reposA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It does disclose authorization scope ('token has access to') and the output purpose ('repo ids usable with the other tools'), but it does not mention the default active-only behavior (schema covers that), pagination, error handling, or rate limits. This is a moderate disclosure, comparable to a baseline where the tool's behavior is partially transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exactly two sentences, front-loaded with the essential verb-object, and the second sentence adds value by stating the return value's utility. No wasted words, and it avoids repeating schema information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with two parameters and no output schema, the description provides the core purpose, access scope, and return value relevance. It could have been more complete by stating default behavior (active-only) or potential pagination, but the schema fills the parameter gap, making the description adequately complete for typical use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds no specific parameter-level detail beyond the schema; it does not explain the 'all' boolean or the 'instance' string beyond what the schema already states. The mention of 'token access' and 'repo ids' provides context but does not enrich parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Lists repositories the token has access to on a Woodpecker instance', which clearly states the verb (Lists), resource (repositories), and scope (token access, instance). It distinguishes itself from sibling tools like list_pipelines and get_pipeline by focusing on repositories and mentioning that it returns repo IDs usable with other tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool: when you need repo IDs to pass to other tools ('Returns repo ids usable with the other tools'). It does not explicitly name alternatives or state when not to use it, but the context of being the repository-list tool among pipeline/log tools provides sufficient guidance. An explicit reference to 'use this before get_pipeline or get_step_logs' would have earned a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.