Server Quality Checklist

Tool Scores

• analyze_structured_dataC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions the tool analyzes for PII and returns findings, but lacks critical behavioral details: it doesn't specify what 'analyze' entails (e.g., detection only, no modification), required permissions, rate limits, error handling, or performance characteristics. The description is minimal and misses key operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: the first sentence states the purpose, followed by a structured list of args and returns. Every sentence adds value, with no redundancy. It could be slightly more concise by integrating the args/returns into prose, but it's efficient overall.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 4 parameters with 0% schema coverage and no annotations, the description is moderately complete: it covers the purpose and parameters at a high level. However, with an output schema present, it needn't explain return values in detail, but it still lacks behavioral context (e.g., safety, limits) and usage guidelines, making it adequate but with clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It lists all four parameters with brief explanations (e.g., 'JSON string representing structured data'), adding basic meaning beyond the schema's titles. However, it doesn't elaborate on formats (e.g., JSON structure), entity type options, or threshold implications, leaving gaps in understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Analyze structured data (JSON/dict) for PII.' It specifies the verb ('analyze'), resource ('structured data'), and target ('PII'). However, it doesn't explicitly differentiate from siblings like 'analyze_text' or 'batch_analyze' beyond the 'structured data' qualifier, which is implied but not contrasted.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention siblings like 'analyze_text' (for text vs. structured data) or 'batch_analyze' (for batch processing), nor does it specify prerequisites, exclusions, or optimal use cases. Usage is implied by the tool name but not explicitly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• anonymize_structured_dataC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions that the tool anonymizes PII, but doesn't explain what happens during anonymization (e.g., whether data is modified in-place, if original data is preserved, error handling, or performance considerations). For a mutation tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and well-structured: it starts with a clear purpose statement, followed by an 'Args' section listing parameters with defaults, and ends with a 'Returns' section. Each sentence earns its place, but it could be more front-loaded by emphasizing the tool's role relative to siblings upfront.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (5 parameters, mutation operation, no annotations) and the presence of an output schema (which covers return values), the description is moderately complete. It explains parameters and returns, but lacks behavioral context (e.g., how anonymization works) and usage guidelines, making it adequate but with clear gaps for effective agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It lists all 5 parameters with brief explanations (e.g., 'JSON string representing structured data' for 'data'), which adds meaning beyond the bare schema. However, it doesn't provide details on allowed values (e.g., valid 'language' codes, 'operator' options, or 'entities' types), leaving gaps in parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Anonymize PII in structured data (JSON/dict).' It specifies the verb ('anonymize'), resource ('structured data'), and format ('JSON/dict'), which is specific and actionable. However, it doesn't explicitly differentiate from sibling tools like 'anonymize_text' or 'batch_anonymize', which handle similar anonymization tasks but for different data types or in batch mode.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'anonymize_text' (for text data) or 'batch_anonymize' (for batch processing), nor does it specify prerequisites, exclusions, or typical use cases. This leaves the agent without context for tool selection among similar anonymization options.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• batch_anonymizeB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool returns a 'JSON string with anonymized results for each text,' which gives basic output info, but lacks critical details: it doesn't specify what 'anonymize' entails (e.g., redaction, masking, pseudonymization), whether it's a read-only or mutating operation, potential rate limits, error handling, or privacy implications. For a tool with no annotations and 5 parameters, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and appropriately sized: it starts with a clear purpose statement, followed by a bullet-point list of parameters and returns. Each sentence earns its place by conveying essential information without redundancy. It could be slightly more front-loaded by integrating parameter defaults into the initial statement, but overall it's efficient and readable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (5 parameters, no annotations, but has an output schema), the description is moderately complete. It covers parameters and output format, but lacks behavioral context (e.g., how anonymization works, side effects) and usage guidelines. The output schema existence reduces the need to detail return values, but without annotations, the description should do more to explain the tool's operational traits and constraints.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description lists all 5 parameters with brief explanations, adding meaning beyond the input schema, which has 0% description coverage. For example, it clarifies that 'entities' defaults to 'all' and 'score_threshold' is a 'minimum confidence score,' which the schema only titles generically. However, it doesn't elaborate on allowed values (e.g., what 'operator' options exist) or provide examples, preventing a perfect score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Anonymize multiple texts in batch.' It specifies the verb ('anonymize'), resource ('multiple texts'), and scope ('in batch'), which distinguishes it from sibling tools like 'anonymize_text' (likely single-text) and 'anonymize_structured_data' (different resource type). However, it doesn't explicitly contrast with 'batch_analyze' or other batch operations, keeping it from a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'anonymize_text' for single texts, 'anonymize_structured_data' for non-text data, or 'batch_analyze' for analysis versus anonymization. Without such context, an agent must infer usage from tool names alone, which is insufficient for clear decision-making.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• batch_analyzeB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the tool analyzes for PII detection and returns JSON results, but lacks details on permissions, rate limits, error handling, or what 'analyze' entails (e.g., detection only, no modification). For a batch processing tool with zero annotation coverage, this is insufficient to inform safe and effective use.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: the first sentence states the purpose clearly, followed by structured sections for 'Args' and 'Returns' that are easy to parse. Every sentence adds value without redundancy, making it efficient and well-organized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (batch analysis with 4 parameters), no annotations, and an output schema present (implied by 'Returns' note), the description is reasonably complete. It covers purpose, parameters, and return format, but lacks behavioral context like error handling or performance considerations, which would be beneficial for a batch tool. The output schema reduces the need to detail return values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds significant meaning beyond the input schema, which has 0% description coverage. It explains each parameter's purpose: 'texts' as 'List of texts to analyze', 'language' as 'Language code', 'entities' as 'List of entity types to detect', and 'score_threshold' as 'Minimum confidence score', including defaults. This compensates well for the schema's lack of descriptions, though it doesn't detail entity types or language code formats.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Analyze multiple texts in batch for PII detection.' It specifies the verb ('analyze'), resource ('multiple texts'), and domain ('PII detection'), which is specific and informative. However, it doesn't explicitly differentiate from sibling tools like 'analyze_text' or 'batch_anonymize', which would require mentioning batch vs single or analysis vs anonymization.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'analyze_text' (for single texts) or 'batch_anonymize' (for batch anonymization), nor does it specify prerequisites, exclusions, or optimal use cases. Usage is implied by the name and purpose but not explicitly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_anonymization_operatorsB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool returns a JSON string with operator names, descriptions, and example parameters, which is useful behavioral context. However, it lacks details on potential limitations (e.g., rate limits, authentication needs, or whether the list is static or dynamic), leaving gaps in transparency for a read operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured with two sentences: one stating the purpose and another detailing the return format. Each sentence earns its place by providing essential information without waste. It could be slightly improved by front-loading the return details more explicitly, but it's efficient overall.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no annotations, but with an output schema), the description is reasonably complete. It explains what the tool does and the format of the return value, which compensates for the lack of annotations. However, it could benefit from more context on usage scenarios or limitations to fully address the tool's role among siblings.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and the schema description coverage is 100%, so there's no need for parameter details in the description. The description correctly omits parameter information, focusing instead on the return value. A baseline of 4 is appropriate as it avoids redundancy and adds value by explaining the output.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get list of available anonymization operators and their descriptions.' This specifies the verb ('Get') and resource ('anonymization operators'), making it easy to understand what the tool does. However, it doesn't explicitly differentiate from sibling tools like 'get_supported_entities' or 'validate_detection', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With sibling tools like 'get_supported_entities' and 'validate_detection' that might overlap in functionality (e.g., retrieving information about anonymization components), there's no indication of when this specific tool is appropriate or what distinguishes it from others in the server.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• anonymize_textA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the tool 'anonymizes' text but doesn't clarify whether this is a read-only operation, what permissions are needed, if it's destructive to the input, rate limits, or error handling. The return format is mentioned but lacks detail on structure or edge cases.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured with a brief purpose statement followed by a bullet-point style breakdown of args and returns. Every sentence adds value without redundancy, and it's front-loaded with the core functionality.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (6 parameters, no annotations, but has an output schema), the description is reasonably complete. It covers all parameters and mentions the return format. However, it lacks behavioral context and usage guidelines, which are important for a tool that modifies sensitive data (PII). The output schema existence reduces the need to detail return values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It provides clear explanations for all 6 parameters, including defaults and examples (e.g., operator options like 'replace', 'redact', and operator_params example). This adds significant meaning beyond the bare schema, though it could elaborate more on entity types or score_threshold implications.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Anonymize PII in text') and resource ('text'), distinguishing it from sibling tools like 'anonymize_structured_data' which handles structured data instead of text. The verb 'anonymize' is precise and differentiates from analysis tools in the same server.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'anonymize_structured_data' or 'batch_anonymize'. The description mentions what the tool does but offers no context about appropriate use cases, prerequisites, or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_supported_entitiesA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool returns a JSON string with a list of supported entity types and descriptions, which adds useful context about the output format. However, it doesn't mention behavioral traits like error handling, rate limits, or authentication needs, leaving gaps for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: it starts with the core purpose, followed by structured sections for Args and Returns. Every sentence earns its place by providing necessary information without redundancy, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no annotations, but has an output schema), the description is fairly complete. It explains the purpose, parameter semantics, and return format. Since an output schema exists, it doesn't need to detail return values further. However, it could improve by addressing potential errors or usage scenarios relative to siblings.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaning beyond the input schema by specifying that the 'language' parameter is a language code with a default of 'en', which clarifies its purpose. Since there is only one parameter and schema description coverage is 0%, the description compensates well by providing essential semantics, though it could detail format constraints (e.g., ISO codes).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get list of all supported PII entity types for a language.' It specifies the verb ('Get'), resource ('supported PII entity types'), and scope ('for a language'), which is specific and actionable. However, it doesn't explicitly distinguish this from sibling tools like 'analyze_text' or 'get_anonymization_operators', which might also involve entity types, so it misses full sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by mentioning the language parameter and its default, suggesting it's for retrieving entity types based on language. However, it doesn't provide explicit guidance on when to use this tool versus alternatives (e.g., 'analyze_text' for detection or 'get_anonymization_operators' for operators). The context is clear but lacks exclusions or named alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_detectionA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the tool's testing-oriented behavior and the return format ('JSON string with validation results including precision, recall, and F1 score'), which is valuable. However, it doesn't mention potential side effects, error conditions, performance characteristics, or authentication needs, leaving gaps for a tool with 3 parameters.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded: the first sentence states the purpose, followed by clear 'Args:' and 'Returns:' sections. Every sentence earns its place by explaining parameters and output without redundancy. It's appropriately sized for a 3-parameter tool with testing focus.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 3 parameters with 0% schema coverage and an output schema present, the description does a good job: it explains all parameters and the return value. However, as a validation/testing tool with no annotations, it could benefit from more behavioral context (e.g., error handling, what happens if validation fails). The output schema reduces the need to detail return values, but some gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaningful semantics: 'text' is 'The text to analyze', 'expected_entities' is a 'List of expected entities with 'entity_type', 'start', 'end'', and 'language' has a default 'en'. This clarifies purpose and structure beyond the bare schema, though it doesn't detail formats (e.g., what 'entity_type' values are).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Validate PII detection against expected results (useful for testing).' It specifies the verb ('validate') and resource ('PII detection'), and the parenthetical clarifies it's for testing. However, it doesn't explicitly differentiate from siblings like 'analyze_text' or 'batch_analyze' beyond the validation/testing aspect.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context with '(useful for testing)', suggesting it's for validation/testing scenarios rather than production analysis. However, it doesn't provide explicit guidance on when to use this vs. alternatives like 'analyze_text' for detection without validation, or mention prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_custom_recognizerA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the tool adds a recognizer (implying a write operation) and specifies the return format, but lacks details on permissions, side effects, error handling, or rate limits. It adds basic behavioral context but misses critical operational traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with the core purpose, followed by a clear breakdown of arguments and returns. Every sentence adds essential information without redundancy, making it efficient and easy to parse for an agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (5 parameters, write operation) and no annotations, the description does well by explaining parameters and return values. However, it lacks guidance on usage versus siblings and some behavioral details like error cases. The presence of an output schema reduces the need to detail returns, but overall completeness is strong with minor gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate fully. It provides detailed semantics for all 5 parameters, including examples for 'patterns', optionality for 'context', defaults for 'supported_language', and clarifies data types and constraints (e.g., score range 0.0-1.0), adding significant value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Add a custom PII recognizer') and resource ('with regex patterns'), distinguishing it from sibling tools like analyze_text or get_supported_entities. It precisely communicates the tool's function without being tautological.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like get_supported_entities or validate_detection. It lacks context about prerequisites, typical use cases, or exclusions, leaving the agent to infer usage from the purpose alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_textA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses key behavioral traits: it performs PII detection (not anonymization), returns JSON with specific fields, and includes optional detailed decision process. However, it doesn't cover rate limits, authentication needs, or error handling, leaving gaps for a mutation-like analysis tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (Args, Returns), front-loaded purpose statement, and every sentence adds value. No redundant information—each parameter explanation is necessary given the 0% schema coverage.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters with 0% schema coverage and no annotations, the description does an excellent job explaining inputs and output format. However, as an analysis tool with potential side effects (e.g., data processing), it could benefit from more behavioral context like performance characteristics or error cases. The output schema existence reduces but doesn't eliminate this need.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate fully. It provides detailed semantics for all 5 parameters: explains 'text' purpose, 'language' default and format, 'entities' examples and default, 'score_threshold' range and default, and 'return_decision_process' effect. This adds substantial value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verb ('analyze') and resource ('text to detect PII entities'), distinguishing it from siblings like 'anonymize_text' or 'analyze_structured_data'. It explicitly mentions what the tool does beyond just the name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage through parameter explanations (e.g., 'entities: List of entity types to detect'), but lacks explicit guidance on when to use this tool versus alternatives like 'batch_analyze' or 'validate_detection'. No when-not-to-use scenarios or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.