Server Quality Checklist

Tool Scores

• analyze_emotion_distributionB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the full burden. It discloses the return format (JSON with counts/percentages), which adds value, but omits safety characteristics (read-only vs. destructive), performance implications for large datasets, or idempotency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two distinct sections (purpose and returns) with no wasted words. Information is front-loaded and appropriately sized for the tool's simplicity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters and the existence of an output schema (per context signals), the description adequately covers the tool's function by specifying the return structure. Minor gap regarding dataset prerequisites prevents a 5.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Zero parameters with 100% schema coverage establishes a baseline of 4. The description appropriately requires no parameter clarification since the tool operates on the implicit current dataset context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear action (Get) and resource (distribution of emotions), but fails to distinguish from sibling tool 'count_by_emotion'. Both tools appear to provide counts; the description doesn't clarify when percentages/distribution analysis is preferable to simple counting.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance provided on when to use this tool versus 'count_by_emotion' or other siblings. No prerequisites are mentioned (e.g., whether a dataset must be loaded first), leaving usage context entirely implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_textA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses case-insensitivity and that results are returned as a JSON string. However, it omits error handling behavior, what constitutes a 'sample', pagination details, or performance characteristics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Uses an efficient docstring format (Args/Returns) that front-loads the purpose statement and structures metadata clearly. Every line provides necessary information without redundancy; appropriate length for a 2-parameter tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple search tool with 2 parameters, the description adequately covers inputs and outputs. While additional domain context about 'samples' would help, the return format documentation and parameter details provide sufficient information for invocation given the low complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema description coverage, the Args section compensates effectively by documenting both parameters: noting the case-insensitive nature of 'query' and the default value of 'limit'. It adds semantic meaning beyond the raw schema types.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb (search), resource (samples), and scope (containing specific text). While it doesn't explicitly differentiate from siblings like 'get_sample', the tool name and description make the text-search purpose distinct from the emotion analysis siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives like 'get_sample' (which likely retrieves by ID). The description provides no when-to-use or when-not-to-use conditions, prerequisites, or workflow guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• count_by_emotionA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses return format ('JSON string with count and percentage'), which adds value beyond the schema. However, it omits safety traits (read-only status), rate limits, or side effects that would help an agent understand operational risks.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Uses structured Args/Returns format that efficiently organizes information. Front-loaded with the core purpose. Slightly redundant in documenting return values when an output schema exists, but necessary given the parameter documentation burden.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter counting tool, the description is nearly complete. It documents the parameter domain and return structure. Minor gap: doesn't clarify what 'samples' refers to (text samples inferred from siblings) or whether the count is relative to total dataset (implied by 'percentage').

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Critical compensation for 0% schema description coverage: the description provides the complete enum constraint ('One of 'sadness', 'joy', 'love', 'anger', 'fear', 'surprise'') that the schema lacks. Without this, the agent would have no valid values to provide.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States specific action (Count) and resource (samples) with clear scope (for a specific emotion). Implies distinction from sibling 'analyze_emotion_distribution' by focusing on single-emotion queries, though it doesn't explicitly clarify when to choose one over the other.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides no guidance on when to use this tool versus siblings like 'analyze_emotion_distribution' or 'search_text'. No prerequisites, exclusions, or workflow context is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_sampleA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and successfully discloses constraints ('max: 20') and return format ('JSON string with samples including text and emotion label'). It omits traits like idempotency or caching, but covers the essential behavioral constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Uses a clean docstring format with Args and Returns sections. Every sentence earns its place; no redundancy or fluff. The structure front-loads the core purpose before detailing parameters and returns.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a low-complexity tool with one optional parameter and an existing output schema (reducing the need for extensive return documentation), the description is nearly complete. Minor gap: lacks explicit differentiation from siblings despite having clear siblings available.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, but the description fully compensates by documenting the 'n' parameter with semantics ('Number of samples'), default value, and maximum constraint ('max: 20'), exceeding baseline expectations.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Get'), resource ('samples from the emotion dataset'), and scope ('random'), clearly distinguishing it from sibling tools like search_text (query-based) and count_by_emotion (aggregation).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by specifying 'random samples,' which suggests when to use it (for random inspection), but lacks explicit guidance on when to choose this over search_text or other siblings, and lists no prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.