Server Quality Checklist

Tool Scores

• get_machine_creation_fieldsB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden but lacks details on side effects, idempotency, authentication, or return format. It only states the output is the machine_creation section, which is minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences: the first states the purpose concisely, and the second provides usage guidance. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite low complexity (1 parameter, no output schema), the description does not explain the parameter or output structure, which an agent would need to use the tool correctly. It is adequate but leaves gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage for the 'machine' parameter, and the description does not explain what the parameter means or any constraints. It adds no meaning beyond the schema field name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns only the 'machine_creation' section of a board's spec, using a specific verb ('Return') and resource. It distinguishes itself from siblings like 'get_board_spec' which returns the full spec.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises to use this from the 'qcom-yocto-new-machine skill' instead of manually parsing Confluence pages, providing clear context for when to use this tool and an alternative to avoid.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_board_specA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries the burden. It states the tool returns data or raises an error, but does not disclose other behaviors like whether it is read-only, required permissions, or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, front-loaded with the core purpose, no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has one simple parameter and no output schema, the description covers the basic functionality and an error condition. However, it could add context about the return format and that it is a read operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema_description_coverage is 0%, but the description adds meaning to the single parameter 'machine' by linking it to a branch name in the repo. However, it doesn't elaborate on format or constraints beyond that.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns the full board spec, specifically mentioning the two sections (machine_creation and partition_conf) it includes. This distinguishes it from siblings like get_machine_creation_fields and get_partition_conf_fields that return only partial data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when the full spec is needed and warns of an error condition (no branch named after the machine), but does not explicitly guide when to use this vs. the sibling tools that return subsets.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_board_specA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It transparently describes the validation checks performed and the return format (valid, schema_errors, warnings) with behavioral consequences (schema_errors block automation, warnings do not). Missing are potential side effects or authorization needs, but it's a read-only validation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, comprising a single paragraph that front-loads the main action and lists specific checks. Could be slightly more structured (e.g., bullet points) but is efficient and avoids fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description explains the validation logic and return format adequately for a simple tool, but it omits the input parameter description and does not mention prerequisites or assumptions about the board spec's existence. Given the context (1 param, no output schema), it is minimally complete but has clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description does not explain the single required parameter 'machine' (type string). Schema coverage is 0%, so the description should compensate, but it does not mention the parameter at all, leaving the agent to infer its meaning from context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: schema-validate a board spec and run cross-checks. It lists specific checks (delta partitions must have notes, no collisions, etc.) distinguishing it from sibling tools that retrieve or list board specs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for validation but does not explicitly state when to use this tool versus alternatives like get_board_spec or refresh_specs. No guidance on when not to use it or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_partition_conf_fieldsA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Description indicates a read-only retrieval of a specific configuration section, but lacks details on permissions or error states; annotations missing so description carries burden, but is adequate for a simple query.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences front-load the purpose and usage, with no extraneous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Sufficient for a simple retrieval tool with one parameter; however, lacks output schema details and error behavior, but the core intent is well-covered.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Single parameter 'machine' has 0% schema description coverage, and the description does not mention or explain the parameter, failing to add any meaning beyond the name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Describes specific verb 'Return' and resource 'partition_conf section of a board's spec', clearly distinguishing from alternative methods like parsing Confluence pages.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use ('from the qcom-partition-conf-new-board skill') and what to avoid ('instead of parsing a Confluence page...'), providing clear context among sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_boardsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, but the description fully describes the read-only behavior of listing board names. No side effects or additional behaviors are needed for this simple operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence with no wasted words. Every part is necessary and clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters and an output schema present, the description is complete. It specifies the return type (board names) and source (branches in board-spec repo). Nothing essential is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so baseline 4 applies. The description adds context that the list comes from the board-spec repo branches, which is useful beyond the input schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it lists all board names from branches in the board-spec repo, using a specific verb and resource. This distinguishes it from sibling tools which retrieve specs, fields, or validate.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use vs alternatives, but it's implied since siblings have different purposes. The description does not provide when-not context or alternative tool names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• refresh_specsA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral transparency. It states the action and a key benefit (no restart), but does not disclose potential side effects, failure modes, or authorization requirements, which are important for a tool that modifies server state.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences. It front-loads the core purpose and provides a usage condition, with no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters and an output schema exists, the description is complete. It tells what the tool does and when to use it, which is sufficient for an AI agent to decide and invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has no parameters, and schema description coverage is 100%. The description need not add parameter details; baseline of 4 is appropriate as it adds no additional value beyond the schema, but none is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action: 'Fetch the latest branches/commits from the board-spec remote.' It uses a specific verb and resource, distinguishing it from sibling tools like get_board_spec or validate_board_spec.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use guidance: 'Call this if a spec was just pushed/updated and you need the MCP to see it without restarting the server.' This indicates a specific scenario, though it does not directly contrast with alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.