Server Quality Checklist

Tool Scores

• kbcA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It only mentions automatic branch resolution and setting KBC_BRANCH_ID, but does not disclose potential destructive actions, side effects, or requirements. The tool could run any command, including write operations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with purpose, no wasted words. Efficient and clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With full schema coverage and output schema present, the description covers the basic purpose and branch context. However, it lacks behavioral information about what commands are safe, potential side effects, or error conditions, which is important for a general CLI executor.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Both parameters are described in the schema (100% coverage). The description adds value by explaining branch context but does not enhance parameter semantics beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool executes a Keboola CLI command with automatic branch context. It uses a specific verb ('execute') and resource ('CLI command'), and distinguishes from siblings which are about mappings, branches, and docs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this is the general CLI executor, but provides no explicit guidance on when to use it versus alternatives. It does not mention when not to use or suggest other tools for specific tasks.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• link_branchA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses that it creates a branch if one doesn't exist and defaults to the git branch name. However, it does not mention idempotency, error handling, or side effects, which would improve transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loads the main action, and contains no redundant information. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a relatively simple tool with a low parameter count and existing output schema, the description is adequately complete. It explains the linking behavior and default naming, though it could mention what the output contains.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage with descriptions for both parameters. The description reiterates the default behavior already present in the schema without adding new semantic meaning beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool links the current git branch to a Keboola development branch, specifying the action and resource. It distinguishes itself from sibling tools like unlink_branch by indicating the linking direction and creation behavior.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides context about default branch naming but does not explicitly state when to use this tool versus alternatives or when not to use it. Usage guidance is implied but not directly addressed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_mappingsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries burden. It states the operation is a list (presumably read-only) and mentions return structure, but lacks disclosure of side effects, authentication, or system impact.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise at two sentences, front-loading the purpose and including return structure. No unnecessary information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters and presence of output schema, the description effectively explains return structure ('Dictionary with mappings dict and current_git_branch'), making it reasonably complete for a simple list tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, so schema coverage is 100% and description adds no param info. Baseline of 4 is appropriate for zero-parameter tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'Lists all git-to-Keboola branch mappings' with a specific verb and resource. It implicitly distinguishes from sibling 'get_mapping' which likely retrieves a single mapping.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this vs alternatives, but the purpose is straightforward. The sibling 'get_mapping' suggests this is for listing all mappings, implying usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_cli_docsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description carries full burden. It states the tool searches documentation but does not disclose behavioral traits like read-only nature, rate limits, or output format. It adds minimal transparency beyond the obvious search operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences: first states purpose, second gives usage guidance and examples. Every sentence is informative with no fluff. Structure is front-loaded and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has an output schema (not shown), the description need not explain return values. It adequately covers the search functionality and query examples. Could be slightly enhanced by mentioning the output type (e.g., 'returns documentation snippets'), but overall sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with a single 'query' parameter. The description adds value by providing specific examples of useful queries (e.g., 'how to push changes', 'branch environment variables'), which enriches the schema's basic description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool searches Keboola CLI documentation for commands, flags, environment variables, and workflows. It provides concrete examples (sync, push, pull, etc.), making the purpose specific and differentiated from sibling tools which deal with mappings and branches.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description says 'Use this to find information about kbc commands...' and gives examples, providing clear context for when to use the tool. However, it does not explicitly mention when not to use it or list alternatives, though sibling tools are unrelated to documentation search.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_mappingA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Despite no annotations, the description discloses safe, read-only behavior and guaranteed success, which provides good transparency for a zero-parameter tool. It does not detail edge cases but is sufficient given the tool's simplicity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: two sentences plus a bullet-like return value listing. Every sentence adds value, and the structure is front-loaded with the core action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters and an output schema present, the description covers the tool's purpose, safety, and return fields (git_branch, keboola_branch_id, linked, is_production). No gaps are apparent for the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has no parameters, so the description adds no parameter details beyond the schema. The baseline for zero parameters is 4, and the description adequately focuses on the tool's action and return value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves the Keboola branch mapping for the current git branch. It uses specific verbs and resource naming, and the purpose is distinct from sibling tools like list_mappings or link_branch.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly notes it is a safe, read-only operation that always succeeds, guiding appropriate use. However, it does not contrast with alternatives like list_mappings or provide when-not-to-use scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• unlink_branchA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses behavior: it does not delete the development branch, only removes local mapping, and cannot unlink default branches. Also specifies return structure (dict with git_branch, unlinked_keboola_branch_id, message).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is concise with front-loaded main action. Uses bullet points for additional context, each sentence adds unique value. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters and presence of output schema, description adequately covers purpose, behavior, constraints, and return format. No gaps remain for agent to infer.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has zero parameters with 100% schema coverage. No parameter details needed. Baseline 4 is appropriate as description adds no redundant parameter info.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool removes the branch mapping for the current git branch, using specific verb and resource. It distinguishes from deleting the actual branch and includes a constraint about default branches, effectively differentiating from sibling tools like 'link_branch' and 'list_mappings'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use (to remove mapping) and when not (for deleting branches or for default branches). Provides alternatives: use Keboola UI or API for deletion. This fully guides the agent on appropriate usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.