Server Quality Checklist

Tool Scores

• compare_stringsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must fully disclose behavior. It describes fuzzy matching and strategies but omits details like case sensitivity, how threshold is applied, normalization effects, or output shape. The output schema exists but is not referenced.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short and uses bullet points for strategies and args, which is clear. However, it redundantly lists arg names that are already in the schema, wasting space that could be used for additional guidance. A more concise and informative approach would be preferred.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters, no annotations, and an output schema, the description should cover return values and typical usage. It does not mention the output format (a score? a boolean?) or provide examples, leaving significant gaps for the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must add meaning. It lists parameter names but provides no semantic details: e.g., what each 'profile' does, valid threshold range, strategy definitions. This barely adds value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Compare two strings using fuzzy matching' and lists specific strategies, making the tool's purpose explicit. It distinguishes itself from siblings like 'normalize_text' and 'find_best_matches' by focusing on pairwise comparison with multiple algorithms.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool vs. alternatives (e.g., 'find_best_matches' or 'explain_match'). While strategies are listed, there is no explanation of which strategy suits what scenario, leaving the agent to guess.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_best_matchesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure, but it only lists parameters without explaining side effects, permissions, or output behavior beyond ranking. Critical traits like read-only or mutation status are absent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and structured as a docstring with a clear one-line summary. It lists parameters efficiently without extraneous text, though the parameter descriptions are too terse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (6 parameters, enums) and the presence of an output schema, the description omits crucial context such as how ranking works, how threshold and strategy interact, and when to use specific profiles. Sibling tools are not referenced for complementary use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, yet the description adds minimal meaning beyond parameter names (e.g., 'profile: Normalization profile' is vague). It does not explain enum options (e.g., what 'product' profile does) or the effect of strategy choices, failing to compensate for missing schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Find and rank candidate strings most similar to a query,' which is a specific verb+resource. It differentiates the tool from siblings like compare_strings (comparison) and normalize_text (normalization) by focusing on similarity ranking.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like compare_strings or explain_match. There is no mention of prerequisites or exclusions, leaving the agent without context for appropriate use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• explain_matchB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It only states basic functionality without disclosing side effects, computational cost, or that the tool is read-only. The description adds little beyond what is expected from the name.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, with the main purpose stated first. The parameter list is slightly redundant given the schema, but it does not add unnecessary length. Overall, it is well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters (2 required, 2 enums) and no annotations, the description covers the basic operation but omits details about return value format (though output schema exists) and when to use specific profiles or strategies. It is minimally adequate but not thorough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. However, it merely restates parameter names and types (e.g., 'First text value') without adding meaningful behavioral constraints or semantics, such as explaining the effect of different profiles or strategies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: compare two strings and explain why they match or differ. It distinguishes itself from siblings like compare_strings (which just compares) and find_best_matches (which finds best match rather than explaining a specific pair).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when an explanation of string matching is needed, but lacks explicit guidance on when to use this tool versus alternatives like compare_strings or find_duplicate_groups. No exclusions or context are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_duplicate_groupsB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It describes the grouping action but lacks details on edge cases (e.g., empty list, duplicate values), performance considerations, or how scoring works internally. The brief parameter explanations do not cover behavioral nuances.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured with a clear lead sentence, bullet-point use cases, and a labeled parameter list. It avoids unnecessary words and front-loads the core purpose, making it scannable for an agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (4 parameters, 2 enums, output schema present), the description covers purpose and basic parameter semantics. However, it omits expected output format (though output schema exists), behavioral constraints, and more detailed usage context relative to siblings, leaving room for improvement.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It provides short explanations for each parameter (e.g., threshold: 'Minimum score for joining a group'), adding meaning beyond the schema's names and types. However, explanations are minimal and do not elaborate on how values affect behavior, leaving gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Group strings that probably represent the same entity,' which is a specific verb and resource. It lists concrete use cases like company-name deduplication and customer-name cleanup, distinguishing it from sibling tools like compare_strings or find_best_matches that focus on comparison or single match retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit 'Useful for' scenarios, giving context on when to use the tool. However, it does not discuss when not to use it or contrast with alternatives like compare_strings for pairwise comparison or explain_match for explanation, limiting guidance for an agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• normalize_textA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It briefly describes the effect of each profile (e.g., 'removes legal company suffixes') but does not disclose side effects, performance characteristics, or limitations. Minimal behavioral context but adequate for a simple normalization function.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Very concise: two sentences plus bullet points for profiles and an Args section. No unnecessary information. Front-loaded with the purpose, making it quick to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers purpose, usage context, parameters, and profiles. Since an output schema exists (not shown but noted), it does not need to explain return values. Lacks examples but is complete enough given the tool's simplicity and sibling context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It clearly defines both parameters: 'value' as text to normalize and 'profile' with enumerated options and their purposes. This adds significant meaning beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it normalizes text before fuzzy matching, and lists four specific profiles (general, company, product, address) that distinguish its functionality. The name and description together make the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'before fuzzy matching', providing clear context for when to use this tool. While it does not explicitly exclude alternatives, the sibling tools (compare_strings, find_best_matches, etc.) are clearly about matching/comparing, so usage intent is well communicated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.