Server Quality Checklist

Tool Scores

• classify_textC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, placing the full burden on the description. The description mentions 'static embeddings' but does not disclose potential behaviors such as edge cases (e.g., text length limits), performance characteristics, or whether the tool is idempotent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads the purpose. It is appropriately sized with no wasted words, earning a point above the average.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that an output schema exists and parameter descriptions are covered, the description is adequate but sparse. It does not mention the output format or any additional context, but the presence of the output schema compensates partially.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents both parameters. The description does not add any additional meaning or constraints beyond what the schema provides, resulting in a baseline score of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool classifies text into predefined categories, specifying the verb ('classify') and the resource ('text into predefined categories'). However, it does not distinguish itself from the sibling tool 'batch_classify' which performs a similar function but in batch.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'batch_classify' or when not to use it. The description lacks context on prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• remove_categoriesB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description must disclose behavioral traits. It mentions 'remove' implying destruction but does not clarify reversibility, impact on classified items, or required permissions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence that efficiently conveys the tool's purpose with no unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With an output schema present, the description does not need to explain return values. However, for a destructive tool, additional context about consequences and usage scenarios would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (the only parameter is documented in schema). The description adds minimal value beyond noting 'one or multiple', aligning with the array type. Baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the action (remove) and object (categories from the classification system). It distinguishes from sibling tools that add or list categories, but does not explicitly call out the differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives, nor any prerequisites, side effects, or scenarios to avoid. The description simply states the action without context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_custom_categoryB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, and the description lacks details on side effects, required permissions, or whether the operation is idempotent. It only states the action without disclosing behavioral traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that immediately conveys the tool's purpose with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While the tool has an output schema (expected to describe return values), the description omits details about error handling, prerequisites, or when to prefer this over batch alternatives. It is adequate for simple usage but lacks completeness given sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions for both parameters (category_name, description). The description adds no extra meaning beyond what the schema provides, so baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (add) and resource (custom category) with a specific purpose (for classification). It differentiates from sibling tools like batch_add_custom_categories (batch) and list_categories or remove_categories.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives such as batch_add_custom_categories. No context on prerequisites, exclusions, or typical scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• batch_add_custom_categoriesB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It only states 'single operation' without disclosing atomicity, uniqueness constraints, or error handling. Lacks behavioral detail for a batch mutation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One sentence with 10 words, concise and front-loaded. Could potentially add more structure (e.g., mention output) without losing conciseness, but current form is efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given low complexity (1 param, output schema exists), description is adequate but leaves unanswered questions about error handling, category validation, and whether operation is atomic. Could be more complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so description adds little beyond schema. The parameter 'categories_data' already has a detailed description and example. No additional semantic value provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verb 'Add' and resource 'multiple custom categories' with context 'for classification in a single operation.' It clearly distinguishes from the sibling 'add_custom_category' which is singular.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool vs alternatives like 'add_custom_category' or 'batch_classify.' The description implies batching but does not state exclusions or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• batch_classifyA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, and the description lacks behavioral details such as error handling, rate limits, synchronous/asynchronous behavior, or any constraints on input size. The description only states the core function without any additional context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is front-loaded and contains no redundant information. It is as concise as possible while conveying the essential purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the batch operation complexity and existence of an output schema, the description is adequate but missing details like maximum batch size, whether processing is synchronous, or how errors are handled for individual texts. The output schema may cover return values, but constraints are omitted.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, and while the description adds no extra meaning beyond the schema, the schema itself is clear. Baseline 3 is appropriate as the description does not enhance parameter understanding beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Classify multiple texts at once' clearly states the verb (classify) and resource (multiple texts), distinguishing it from the sibling tool 'classify_text' which is for single text classification.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for batch classification, but does not explicitly state when to use it versus alternatives like 'classify_text' for single texts or 'batch_add_custom_categories'. No when-not or prerequisite guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_categoriesA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description partially covers behavior: it is a read operation returning a JSON string. However, it does not clarify if the list includes user-added custom categories or only system defaults, leaving ambiguity about its scope.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with two short sentences. Every word adds value: 'List all available categories for classification' captures the action and purpose, and the return type is explicitly stated. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, output schema exists), the description is mostly sufficient. However, it lacks clarity on whether custom categories are included, which is relevant given sibling 'add_custom_category'. This minor gap prevents a higher score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, and the schema coverage is 100% by default. The description adds no parameter information because none is needed, earning the baseline score of 4 per rubric.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List' and the resource 'all available categories', making the tool's purpose immediately obvious. It distinguishes itself from sibling tools like 'add_custom_category' and 'classify_text' which perform different operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives, such as whether to call it before classification or how it relates to custom categories. The description only states what it does, not the appropriate context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.