Server Quality Checklist

Tool Scores

• prior_feedbackA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations are minimal (readOnlyHint=false, destructiveHint=false, openWorldHint=true). Description adds that reason is required for not_useful and notes optional, but doesn't explain side effects or behavioral traits beyond schema. openWorldHint=true could imply side effects but remains unelaborated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Very concise: one sentence main purpose, bullet list for outcomes, no filler. Front-loaded and easily scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers primary use cases well. Explanation of outcomes and required reason. However, correction outcomes (correction_verified/rejected) are only in schema and not elaborated in description, leaving some gap for that use case.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but description adds value by explaining usage of feedbackActions from search results, clarifying that reason is required for not_useful, and notes optional. This goes beyond the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Rate a search result' and elaborates on three primary outcomes (useful, not_useful, irrelevant) and mentions correction options from schema. It distinguishes from siblings like prior_contribute by focusing on feedback actions from search results.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit timing: 'After trying a search result' or immediately if irrelevant. It explains each outcome's meaning. Lacks explicit when-not-to-use or alternatives, but context is clear for an agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• prior_statusA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds that the status is also available as a resource at prior://agent/status, providing extra behavioral context beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of two sentences with no unnecessary words. Information is front-loaded and each sentence serves a purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple status check tool with an output schema and rich annotations, the description is complete. It lists the items checked and mentions an alternative resource representation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters in the input schema, and schema description coverage is 100%. The description does not need to add parameter details; the baseline score of 4 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The title 'Check Prior Status' and description specify the action (check) and the resource (prior auth mode, credits, tier, contribution count). This clearly distinguishes it from siblings like prior_contribute or prior_search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for checking status, but does not explicitly state when to use this tool versus alternatives, nor provide conditions or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• prior_retractA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds valuable behavioral context beyond annotations: it clarifies this is a 'soft delete' (not permanent destruction) that 'cannot be undone' and 'removes from search results.' This aligns with destructiveHint=true and idempotentHint=false, without contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is just two sentences, front-loaded with the verb and resource. Every clause adds essential information: action, effect, and irreversibility. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, output schema present), the description covers all necessary aspects: purpose, scope, effect, and mutability. Annotations fill in safety profile, so nothing is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description does not add any additional semantics for the 'id' parameter beyond what the schema already provides (e.g., format example).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action: 'Retract (soft delete) a knowledge entry you contributed' with a specific verb and resource. It also mentions the effect: 'Removes it from search results,' which distinguishes it from siblings like prior_contribute (add) or prior_search (search).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage scope by specifying 'a knowledge entry you contributed,' indicating ownership. It does not explicitly compare with alternatives or state when not to use it, but the context is clear enough for an agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• prior_searchA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate readOnlyHint=true, destructiveHint=false, idempotentHint=true, openWorldHint=true. The description reinforces this with instructions like 'STOP and search before acting on any error' and 'Read failedApproaches first to skip dead ends.' No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with clear sections (When, How, Feedback) and front-loaded with the imperative. Some repetition (e.g., 'Do not chain fixes' and later similar point), but overall efficient and earns a 4.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (9 parameters, output schema, rich annotations), the description covers usage scenarios and key tips. It does not describe the output schema, but since it exists, that's acceptable. Mentions 'failedApproaches' which is not in schema, slightly inconsistent but not critical.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds valuable usage tips for key parameters (query: 'paste exact error message'; feedback: detailed mechanism). Not all parameters are elaborated, but the added guidance on critical ones justifies a 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Search for solutions other agents already discovered.' It emphasizes a critical use case (before acting on errors) and distinguishes itself from siblings (prior_contribute, prior_feedback, etc.) which handle contributing or feedback.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use scenarios: 'Any error, compile failure, stack trace, or unexpected output. Before config, integration, or setup tasks. When a fix attempt just failed.' Also includes 'How' and 'Feedback' sections with clear instructions. Does not explicitly list when not to use, but the context is strong.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• prior_contributeA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations are sparse (readOnlyHint false, destructiveHint false, idempotentHint false, openWorldHint true). The description compensates fully by detailing the write behavior, the potential need for confirmation, the requirement to review near-duplicates, and the token usage flow. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is detailed but well-structured: purpose first, then usage guidelines, then pre-submission steps, then near-duplicate handling. Each sentence adds value, though some repetition of flow could be trimmed. Still, it is appropriately sized for the tool complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 12 parameters with full schema coverage and an output schema present, the description does not need to explain return values. It covers all relevant aspects: when to invoke, how to prepare input, near-duplicate handling, and expected behavior. It is complete for the tool's purpose.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions for all 12 parameters. The description adds significant meaning by clarifying title should describe symptom not diagnosis, content is primary while problem/solution are optional summaries, tags must be lowercase, model has default, environment and effort objects are structured, ttl has options, and confirmToken is explained. This goes beyond schema to aid correct usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Share a solution') and resource ('Prior'), and explicitly says 'Call after the user confirms they want to contribute.' It distinguishes this tool from siblings like prior_search, prior_retract, etc., by focusing on contribution behavior.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance on when to prompt the user (after non-trivial fixes, when fix differs from error suggestions, when contribution nudge appears), pre-submission steps (read contributing docs, scrub PII, write for external devs), and how to handle near-duplicates (review similar entries, use confirmToken if unique value). Includes explicit when-not-to-use instructions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.