Server Quality Checklist

Tool Scores

• concept_queryA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It describes format options (minimal vs verbose output), performance aspects ('query optimization and performance metrics'), and processing behavior ('comprehensive result processing and enhancement'). However, it doesn't cover important behavioral traits like rate limits, authentication requirements, error conditions, or pagination behavior for a query tool with 9 parameters.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (Features, Format Options, Filter Parameters, Use Cases), but it's verbose with some redundant phrasing like 'sophisticated multi-parameter filtering' and 'powerful filtering capabilities.' The 'Features' section contains marketing language ('Query optimization and performance metrics') that doesn't add practical guidance for tool selection.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (9 parameters, 0% schema coverage) and presence of an output schema, the description is reasonably complete. It thoroughly documents parameters and their usage, describes output format options, and provides usage scenarios. The main gap is lack of behavioral details like rate limits or error handling, but the output schema reduces the need to describe return values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage and 9 parameters, the description provides excellent parameter semantics beyond the bare schema. It explains each filter parameter (start, end, rel, node, other, sources) with examples and clarifies their usage. It also documents the 'verbose' parameter's behavior and default values, and mentions 'language' and 'limit_results' parameters in the context section, adding significant value beyond the input schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as 'Advanced querying of ConceptNet with sophisticated multi-parameter filtering' and 'exploring ConceptNet's knowledge graph.' It specifies the action (querying/filtering) and resource (ConceptNet knowledge graph), but doesn't explicitly differentiate from sibling tools like concept_lookup or concept_relatedness beyond mentioning 'backward compatibility.'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes a 'Use this when you need' section listing specific scenarios like 'Precise relationship filtering and discovery' and 'Complex queries with multiple constraints.' This provides clear context for when to use this tool, though it doesn't explicitly mention when NOT to use it or name alternatives among the sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• concept_relatednessA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does well by explaining the tool's behavior: it's a calculation/analysis tool (not destructive), uses ConceptNet embeddings, provides score ranges (0.0-1.0), offers format options (minimal vs verbose), and includes analysis components. It doesn't mention rate limits, authentication needs, or error conditions, but provides substantial behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections but somewhat verbose. The 'Features' and 'Analysis Components' sections contain some redundancy (e.g., 'detailed relationship analysis' vs 'likely connection explanations'). Some sentences could be more tightly written while maintaining clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (semantic analysis with 5 parameters), no annotations, and an output schema (which handles return values), the description provides good context about what the tool does, when to use it, and key behavioral aspects. It could benefit from more explicit parameter explanations and error handling information to be fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage for 5 parameters, the description compensates well by explaining the verbose parameter's two format options and implying language parameters support cross-language comparison. However, it doesn't explicitly explain concept1/concept2 parameters beyond their role in the calculation, leaving some semantic gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool calculates semantic relatedness scores between two concepts using ConceptNet's embeddings, distinguishing it from sibling tools (concept_lookup, concept_query, related_concepts) which likely perform different operations like looking up concepts or finding related concepts rather than measuring pairwise similarity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear 'Use this when you need to' section listing specific scenarios (quantify similarity, compare across languages, measure semantic distance, validate relationships), giving good context for when to use this tool. However, it doesn't explicitly state when NOT to use it or mention alternatives among the sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• concept_lookupA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key traits: it's a read-only lookup tool (implied by 'queries'), returns all results by default (not limited), supports language filtering and cross-language exploration, includes performance optimization with automatic pagination, and offers format control (minimal vs. verbose). However, it lacks details on rate limits, error handling, or authentication needs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (purpose, features, format options, usage guidelines) and front-loaded key information. Most sentences earn their place by adding value, though some phrasing (e.g., 'Performance optimized with automatic pagination') could be more concise. Overall, it's appropriately sized for a tool with 5 parameters and no annotations.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (5 parameters, no annotations, but with an output schema), the description is largely complete. It covers purpose, usage, behavioral traits, and parameter semantics adequately. The output schema exists, so the description needn't explain return values. However, it could improve by explicitly linking parameters to features (e.g., naming 'limit_results') and addressing potential constraints like rate limits.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds significant meaning beyond the schema: it explains the 'verbose' parameter with two format options (minimal vs. comprehensive), mentions language filtering and cross-language exploration (hinting at 'language' and 'target_language'), and implies 'limit_results' controls whether to return all results. However, it doesn't explicitly define all five parameters (e.g., 'term' is only implied).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Look up information about a specific concept in ConceptNet' and 'queries ConceptNet's knowledge graph to find all relationships and properties associated with a given concept.' It distinguishes from siblings by specifying it returns 'ALL results (not limited to 20)' and mentions 'Backward compatibility maintained with existing tools,' implying differentiation from concept_query.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly provides usage guidance with a 'Use this when you need to:' section listing four specific scenarios (e.g., 'Understand what ConceptNet knows about a concept,' 'Explore all relationships for a term'). It implicitly distinguishes from siblings by mentioning 'complete information' and 'ALL results,' suggesting alternatives might be limited or partial.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• related_conceptsA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behavioral traits: it explains the similarity scoring range (0.0 to 1.0), result ranking, language filtering defaults, format options (verbose vs. minimal), and includes details like statistical analysis and categorization. However, it doesn't mention potential limitations such as rate limits, error conditions, or performance characteristics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (Features, Format Options, Similarity Analysis, Use cases) and front-loaded with the core purpose. However, it includes some redundant phrasing (e.g., 'comprehensive analysis' and 'detailed similarity analysis') and could be slightly more concise by merging overlapping points about analysis. Every sentence adds value, but minor trimming is possible.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (5 parameters, no annotations, but with an output schema), the description is highly complete. It covers purpose, usage, behavioral traits, parameter semantics, and output format details. The presence of an output schema means the description doesn't need to explain return values, and it adequately addresses all other aspects, making it sufficient for effective tool selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Given the schema description coverage is 0%, the description must compensate, which it does excellently. It explains the semantics of key parameters: 'term' is implied as the input concept, 'language' and 'filter_language' are covered under 'Default English language filtering (can be disabled or changed)', 'limit' is implied by 'Results are ranked', and 'verbose' is detailed in the 'Format Options' section with clear explanations of true/false behaviors. This adds substantial meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Find concepts semantically related to a given concept using ConceptNet's embeddings.' It specifies the verb ('find'), resource ('concepts'), method ('using ConceptNet's embeddings'), and distinguishes it from sibling tools like concept_lookup, concept_query, and concept_relatedness by focusing on semantic similarity discovery rather than lookup, querying, or relatedness scoring.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly provides usage guidelines in the 'Use this when you need to:' section, listing specific scenarios such as 'Discover semantically similar concepts,' 'Expand concept exploration and brainstorming,' 'Find related terms and ideas,' and 'Understand semantic neighborhoods.' This gives clear context for when to use this tool versus alternatives, though it doesn't explicitly name sibling tools as alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.