Server Quality Checklist

Tool Scores

• get_certificationB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It clearly indicates this is a return/read operation and discloses the kind of data returned, which is adequate. However, it does not mention behavior when the repository/certification record is missing, whether live data is fetched from GitHub, or any response-shape details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence with no filler. The core action is front-loaded, and the parenthetical packs useful output context without bloating the text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple get-by-owner/repo tool, the description covers the key context: what resource is returned and the main output dimensions. No output schema exists, so the parenthetical helps compensate, though a fuller description of the response structure or behavior when not found would make it complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the parameters owner and repo are already fully documented by the schema. The description adds context about the record type but does not explain the parameters beyond what the schema provides, so baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Return') and a specific resource ('dsh-plugin-certification record'), scoped to one repository, with a useful parenthetical of the returned content (grade, snapshot date, five-dimension evidence). It does not explicitly contrast with sibling tools, but 'for one repository' clearly signals singular retrieval versus list_certified.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance about when to choose this tool over list_certified or certification_spec. The phrase 'for one repository' implies a singular lookup, but no explicit when-to-use, when-not-to-use, or alternative routing is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_certifiedA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that the tool lists all plugins, not a filtered subset, and includes the output fields. However, it does not mention ordering, pagination, or any access requirements, leaving some behavioral aspects unstated for a list operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, pointed sentence that front-loads the action and resource, then lists the output fields. Every word contributes meaning; no filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with no parameters and no output schema, the description covers the key information: what is listed, from where, and which attributes are returned. It could be more complete by noting ordering or pagination, but the core calling context is sufficiently clear.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. The description adds no parameter details, but none are needed because the schema is empty and there is nothing to explain.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('List') and resource ('every plugin in the public dsh-plugin-certification registry'), and names the returned attributes (repo, grade, snapshot date). This clearly distinguishes it from sibling tools like get_certification or certification_spec, which suggest retrieving a single item or a spec rather than the full list.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given about when to use this tool versus its siblings. It does not mention alternatives, exclusions, or conditions. While the simple zero-parameter interface reduces ambiguity, the presence of sibling tools makes the lack of explicit routing a notable gap.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• certification_specA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full behavioral disclosure burden. 'Explain' signals a purely informational, non-mutating call, and the sentence specifies exactly which content will be delivered (five dimensions, grade scale, veto rule). It does not explicitly state that nothing is modified or describe the response shape, but for a zero-parameter explainer tool this is largely sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single 17-word sentence front-loads the key verb and resource, then packs the content scope into a compact list. There is no filler, tautology, or repetition — every word contributes information an agent needs.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity — zero parameters, no annotations, no output schema — the description names the full content scope (five dimensions, grade scale, veto rule), which is what an agent needs to decide whether to invoke it. The only omission is the return format, which is a minor gap for a purely explanatory tool that will predictably return a textual explanation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool takes zero parameters, and the rubric sets a baseline of 4 for that case. The empty input schema already exhaustively conveys the parameter surface, so there is nothing further the description needs to add about parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb, 'Explain', bound to a concrete resource, 'the dsh-plugin-certification spec v1', and enumerates the exact scope: 'the five dimensions, the grade scale and the veto rule'. This naturally distinguishes it from the siblings get_certification and list_certified, which connote retrieving certification data rather than explaining the spec itself.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no explicit when-to-use or when-not-to-use guidance, and no alternative tool is named. The appropriate context (an agent wanting to understand the spec's evaluation model before or instead of querying actual certifications) is only implied by the verb 'explain', leaving the agent to infer routing from the tool name and sibling names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.