Server Quality Checklist

Tool Scores

• health_checkB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing behavior. It mentions that it returns health, version, and capabilities, but does not state whether it is read-only, safe, or has any side effects. This is a significant gap for a tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, using only a few words to convey the core purpose. It lacks verbosity, but it is structured as a noun phrase rather than a complete sentence, which slightly reduces clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool (zero parameters, no output schema, no annotations), the description is minimally sufficient. It names the key outputs but leaves 'capabilities' undefined and does not explain any error behavior or return format, which could leave an agent uncertain about what to expect.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is nothing to explain. The schema with empty properties is fully covered, and the baseline for 0 params is 4, which is appropriate here.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names the resource (server) and the specific aspects covered (health, version, capabilities), which is clear enough to distinguish from the sibling decision tools. However, it lacks a verb and reads as a noun phrase rather than a full statement of action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives. The description does not mention any prerequisites, exclusions, or context in which this tool is appropriate, leaving the agent to infer usage from the name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compare_twoA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral burden. It discloses the exact outputs (winner, score margin, criteria wins, breakdown) and input flexibility. It does not mention error handling or tie-breaking behavior, but for a pure computation tool, the disclosure is solid and non-misleading.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary purpose, and every clause adds useful information (outputs, input forms). No filler or redundant details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 6 parameters, nested objects, and multiple input formats, and no output schema. The description covers the essential usage (how to pass options, what is returned) and the schema documents criteria/scores. It is complete enough for an agent to invoke correctly, though it omits guidance on method selection or behavior for invalid input, which the schema partially addresses via defaults and enums.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 50%, with criteria and options already well-documented. The description adds value by explaining the two ways to pass options (option_a/option_b vs 2-element options array), which is not obvious from the schema alone. However, it does little to elaborate on scores or criteria structure beyond what the schema provides, so it only partially compensates for coverage gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Head-to-head comparison of exactly two options', which is a specific verb ('comparison') and resource ('two options'), and details the return values (winner, score margin, criteria wins, per-criterion breakdown). This clearly distinguishes it from sibling tools like score_options or sensitivity_analysis.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'exactly two options' sets a clear constraint on usage, and the description clarifies the two acceptable input formats (option_a/option_b or 2-element options array). However, it does not explicitly mention when to use alternative tools like score_options for more than two alternatives, so it stops short of full exclusion guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_decisionA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool is 100% deterministic and states the default method (weighted_sum) plus alternatives, which are key behavioral traits. However, it does not discuss error handling, input validation, or side effects, though for a pure calculation tool these are less critical.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with the primary purpose and outputs. Every sentence contributes: the first details functionality, the second signals priority, and the third notes determinism and default method. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema, the description adequately covers return values (winner, ranking, breakdowns, explanation) and notes determinism. It does not describe input formatting beyond what the schema provides, but given a rich schema, this is acceptable. Slight gaps remain regarding edge cases or exact response structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 75%, with method lacking a description in the schema. The description adds 'method defaults to weighted_sum (also weighted_product, topsis)', but the schema already includes the enum and default for method, so the added value is minimal. Other parameters are already described well in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it ranks named options against weighted criteria and specifies the outputs: winner, full ranking, per-criterion breakdowns, methodology, and explanation. Phrases like 'Main tool' and enumerating return values distinguish it from sibling tools such as score_options and sensitivity_analysis.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: this is the main tool for ranking options with weighted criteria. It implies use this as the primary decision tool, but it does not explicitly mention alternatives or when not to use it. The 'Main tool' label gives some prioritization, though no explicit exclusions or comparisons to siblings are made.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• score_optionsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the transparency burden. It discloses the output: 'full normalized scored matrix + ranking', and explicitly states what is not returned ('without the narrative winner explanation'). This adds meaningful behavioral context beyond input schema, though it doesn't mention normalization details or side effects (which are likely none for a scoring tool).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the action verb 'Score', and every word adds value. It states the purpose, a condition, and the output, with zero redundancy or unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (nested objects, 4 params, no output schema), the description covers the essential purpose and return format (normalized matrix + ranking). It doesn't explain method-specific behavior or the ranking structure, but that is likely out of scope for a brief description. It is sufficiently complete for an AI to select and invoke correctly in most cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 75%, so the schema already documents most parameters (options, criteria with direction and weight, scores matrix). The description adds a small but useful nuance: 'when the score matrix is supplied separately', clarifying the scores parameter's role. However, it doesn't elaborate on the method parameter or how criteria directions affect scoring, which the schema partially covers. This earns a solid baseline 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb 'Score' and resource 'options against criteria'. It distinguishes itself from siblings by specifying the separate score matrix input and the exclusion of narrative explanation, making its scope unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a clear condition for use: 'when the score matrix is supplied separately', which implies when to apply this tool. It also implicitly contrasts with a version that includes narrative winner explanation, but it doesn't explicitly name alternatives or exclusions. This is solid guidance but not exhaustive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sensitivity_analysisA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explains the sweep mechanism, the recomputation of rankings, and the specific outputs (robustness score, critical criteria, flip points). It does not mention side effects or prerequisites explicitly, but the read-only analysis nature is implied.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is only two sentences, front-loaded with the core purpose, and includes specific details about defaults and outputs without any redundancy. Every sentence contributes substantive information, making it highly concise and effective.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description provides a clear overview of the tool's function, process, and outputs, which is sufficient for an agent to understand its role. It lacks explicit mention of prerequisites (e.g., needing a prior decision with weights and scores) and does not specify how to interpret the robustness score, but the description covers the essential behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description explicitly explains 'variation' and 'steps' with defaults, adding meaning beyond the schema. However, it does not mention the 'method' parameter or clarify how options/criteria/scores are used beyond what the schema already describes. With 50% schema coverage, the description only partially compensates for undocumented parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with a specific verb-resource pairing: 'Test how robust the winner is to criteria-weight changes.' This clearly distinguishes it from sibling tools like score_options or compare_two, which focus on scoring or pairwise comparison. The method details further reinforce its unique purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies its use case: testing robustness after a winner has been determined. It provides clear context for when to invoke this tool but does not explicitly name alternatives or exclusions. Since siblings like compare_two and score_options exist, a direct alternative would improve it, but the purpose is self-explanatory.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_methodsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden of explaining behavior. It discloses the content returned (normalization details, score ranges, and usage guidance), which sets expectations for a list operation. It does not mention potential side effects, but for a read-only listing tool this is not critical.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that front-loads the verb and resource, then succinctly enumerates the key detail categories. Every phrase contributes meaningful information with no waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 params, no output schema), the description fully covers the relevant context by naming the specific methods and the types of information returned (normalization details, score ranges, usage guidance), making the output predictable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the schema is empty, so the description need not explain any parameters. The baseline of 4 applies because there is no parameter information to add.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'List' with the resource 'scoring methods' and enumerates the methods ('weighted_sum, weighted_product, topsis'), making the tool's function clear. It also distinguishes from siblings like 'score_options', which likely use these methods rather than list them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states what the tool returns (normalization details, score ranges, and when to use each method), implying it is used to select/understand methods. However, it does not explicitly contrast this tool with siblings like 'score_options' or state when not to use it, so it lacks a clear exclusion.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.