Server Quality Checklist

Tool Scores

• watsonx_chatD
  1.9/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only states a high-level action and fails to mention important behavioral traits like whether the response is streaming, if authentication is required, rate limits, or how errors are handled. The tool appears to be a black box.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, so it is concise in length, but it is under-specified for a tool with four parameters. Unlike the high-calibration example where two sentences carried complete information, here the brevity removes essential context, making it more of an under-specification than effective conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is inadequate for the tool's complexity. There is no output schema, no annotations, and the description does not mention return format, message structure requirements, or any operational details. An agent cannot reliably invoke this tool based on the provided information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds no parameter information beyond the schema. Since schema coverage is only 50% (temperature and max_new_tokens lack descriptions), the description should compensate but does not. It doesn't clarify the role of messages, model selection, or generation parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Have a conversation with watsonx.ai chat models' clearly identifies the tool as a chat-based interaction with watsonx.ai models, using a specific verb and resource. It distinguishes from siblings like watsonx_generate (likely single-turn generation) and watsonx_embeddings, though it doesn't explicitly mention multi-turn conversation or response generation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool vs alternatives such as watsonx_generate or watsonx_list_models. The description does not state any exclusions, prerequisites, or typical use cases, leaving the agent without direction on tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• watsonx_generateC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It only states the action and model examples; it does not disclose output format, non-determinism, latency, authentication requirements, rate limits, or any other behavioral traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no unnecessary words, delivering the essential information efficiently. It is compact and front-loaded, earning a high score for conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no annotations and no output schema, the description is the only source of behavioral and contextual information. It provides only a high-level overview and lacks details on return values, prerequisites, error handling, or how it differs from watsonx_chat. This is insufficient for a text-generation tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with all six parameters having descriptive names and default values. The tool description adds no additional parameter context, but the schema already provides sufficient semantic coverage, yielding a baseline 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Generate') and resource ('text'), and names the platform (IBM watsonx.ai) with example model families. It clearly states the core function but does not explicitly differentiate from the sibling watsonx_chat, which likely also generates text.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool over alternatives like watsonx_chat or watsonx_embeddings. There are no use-case descriptions, exclusions, or mention of different tool behaviors.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• zos_connect_call_serviceB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It mentions a configuration requirement but does not disclose whether the call may mutate mainframe state, potential side effects, error behaviors, or response handling. For a tool that invokes services, this is a significant transparency gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence that includes useful context (mainframe program types) and a key prerequisite. It avoids fluff, though it slightly repeats the tool name.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema and no annotations, and the description does not address response behavior, error scenarios, or potential side effects. For a service-calling tool, this is insufficient for an agent to use it safely and effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all parameters are already documented in the schema. The description adds nothing beyond the schema—it only names the service generically. Given the high coverage, a baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Call a z/OS Connect service' with a specific verb and resource, and clarifies the interaction with mainframe programs (CICS, IMS, batch). This distinguishes it from sibling tools like zos_connect_list_services and zos_connect_get_service_info, which are for discovery/inspection.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no explicit guidance on when to use this tool versus alternatives, nor any exclusions. The only usage hint is the configuration prerequisite 'Requires ZOS_CONNECT_URL', but it does not explain scenarios where this tool is appropriate or when other z/OS Connect tools should be used instead.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• key_protect_list_keysB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does not mention that this is a read-only operation, what the response includes, pagination behavior, or any access requirements. The parenthetical about HSM backing is contextual, not behavioral.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, front-loaded with the action and resource, with no wasted words. It is appropriately concise for a straightforward list operation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema and no annotations, so the description should explain return values or important behavioral details. It does neither, leaving the agent without information about what the response looks like or any caveats.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% (both limit and offset have descriptions), so the baseline is 3. The description adds no extra parameter context, but the schema already documents the parameters adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('List') and resource ('encryption keys from IBM Key Protect'), which clearly distinguishes it from sibling operations like create/get/wrap/unwrap/delete. The parenthetical adds helpful context about the service type, though it does not explicitly name alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus alternatives (e.g., get_key for a single key). The description only states what it does, not when to select it, and lacks any prerequisite or exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• watsonx_embeddingsB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description would need to disclose behavioral traits such as output format or side effects, but it only states the core action. No additional context is provided.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with no fluff, making it easy to parse and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with two parameters and no output schema, but the description doesn't explain the return format or provide usage context, leaving some ambiguity for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with both texts and model_id having descriptions, so the baseline is 3. The description adds no parameter-specific meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool generates text embeddings using watsonx.ai models, which is a specific verb and resource that distinguishes it from sibling tools like watsonx_chat and watsonx_generate.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives, nor any exclusions or prerequisites. It is a bare statement of functionality.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• key_protect_delete_keyA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses the most critical behavioral trait: the deletion is 'irreversible'. This provides a safety warning, but it does not mention permissions, what happens to associated resources, or the effect of the 'force' parameter. Since no annotations are provided, the description carries the full burden and only partially meets it.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence that front-loads the action and the key warning. No irrelevant information, perfectly concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple deletion tool with schema-covered parameters and no output schema, the description is mostly complete. However, it could mention the 'force' parameter behavior for associated resources, which is only in the schema, and it lacks information about potential failures or permissions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers both parameters with descriptions at 100% coverage, so the description does not need to add parameter information. The description itself does not reference parameters, but the schema already provides adequate semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action: 'Delete an encryption key from IBM Key Protect', with a specific verb and resource. It distinguishes itself from sibling tools like key_protect_list_keys and key_protect_create_key, and adds the unique irreversibility warning.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives, nor any exclusions or prerequisites. The description simply states the action without contextual decision-making information, e.g., when to use force or what to consider before deletion.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• key_protect_get_keyA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears the full burden of behavioral disclosure. It merely states 'Get details' without explaining whether this is a read-only operation, what specific details are returned, or any potential errors (e.g., key not found). The verb implies read-only, but there is no additional context about response structure or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that directly states the tool's purpose. It is front-loaded with the action and resource, and there is no redundant or extraneous information. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter get operation, the description is minimally sufficient: it identifies the action and requires key_id. However, there is no output schema and the description does not specify what 'details' means (e.g., metadata, policies, or key material). Given that the tool could return security-sensitive data, more detail about the response would improve completeness. The missing usage guidance also lowers the score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (key_id is fully described as 'The ID of the key to retrieve'). The description does not add any extra parameter meaning beyond what the schema already provides, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get'), the resource ('details of a specific key'), and the system ('IBM Key Protect'). It distinguishes itself from sibling tools like key_protect_list_keys (listing) and key_protect_delete_key (deletion) by specifying retrieval of a single key's details.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The purpose is implied: use this when you need details for a specific key, identified by key_id. However, there is no explicit guidance on when not to use it or how it compares to alternatives like listing keys or wrapping/unwrapping. The context of Name and sibling tools suggests the use case, but the description does not articulate it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• key_protect_unwrap_keyA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of disclosing behavioral traits. It only says 'Unwrap (decrypt) data', which merely restates the function. It does not mention whether the operation is read-only, requires special permissions, has side effects, or what happens on AAD mismatch. This lack of non-obvious behavioral disclosure is a gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is direct and front-loaded with the main action. No wasted words. It is appropriately concise for a straightforward tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has only 3 parameters all fully described in the schema, and no output schema or annotations, the description adequately states the core purpose. It could mention the return value (plaintext) or error conditions, but these are not strictly necessary for a simple operation. The description is sufficient for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds no additional meaning beyond what the schema already provides for parameters like key_id, ciphertext, and aad. It does not explain parameter relationships or provide usage examples, so it neither enhances nor detracts from the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the operation: 'Unwrap (decrypt)' with a specific resource 'data using a root key'. This distinguishes it from sibling tools like key_protect_wrap_key, which performs the inverse operation. The verb and resource are specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage (unwrap data) but does not explicitly state when to use this tool versus alternatives like key_protect_wrap_key. It provides no context such as 'use this for data wrapped with wrap_key' or any exclusions. The usage is somewhat inferred from the name and description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• key_protect_create_keyA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It adds useful security context ('stored in FIPS 140-2 Level 3 HSM') and implies a persistent write operation, but it does not disclose permissions, idempotency, return behavior, or rate limits. This is a moderate gap for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no filler. Every word contributes meaning: the action, the resource, the service, and the security storage context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is quite brief for a create operation with no output schema and no annotations. It covers the fundamental purpose and adds security context, but it omits expected return values, next steps, or any caveats. Schema covers parameters well, but the overall description could be more complete for a tool that creates a persistent resource.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema fully describes all four parameters including defaults and enum values. The description adds no parameter-level detail beyond what the schema already provides, so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Create'), the resource ('encryption key'), and the service ('IBM Key Protect'). This distinguishes it from sibling tools like key_protect_list_keys, key_protect_wrap_key, and key_protect_delete_key.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use when a new encryption key is needed, but it does not explicitly state when to use this tool over alternatives or mention any exclusions. There is no guidance on prerequisites or when not to use it, though the create action is self-evident among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• key_protect_wrap_keyA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only restates the operation ('wrap (encrypt) data') and gives the envelope encryption context, but fails to mention that this operation does not modify the root key (i.e., it's a safe read-like operation), that it returns ciphertext, or any required permissions. This is a significant gap for a cryptographic tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that clearly states the action and purpose. There is no redundant wording or filler, and it earns its place by adding the envelope encryption context beyond the tool name.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 3 parameters, no annotations, and no output schema. The description provides some context (envelope encryption) but does not explain what the function returns (wrapped ciphertext), the role of AAD, or any behavioral expectations. While the schema covers parameters, the lack of output/behavioral detail leaves gaps especially for a crypto operation. It is adequate but not complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with each parameter (key_id, plaintext, aad) having a description. The tool description adds the high-level concept of envelope encryption but no additional parameter-level meaning. Baseline 3 is appropriate since the schema already documents the parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific verb 'Wrap (encrypt)' and the resource 'data using a root key in IBM Key Protect', and adds the purpose 'for envelope encryption'. This distinguishes it from sibling tools like key_protect_unwrap_key and key_protect_list_keys.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'for envelope encryption' provides clear context for when this tool is appropriate. However, it does not explicitly mention alternatives (e.g., using unwrap_key for decryption) or when not to use this tool. The sibling list implies the encryption/decryption pairing, but the description itself lacks exclusionary guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• watsonx_list_modelsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosing behavior. 'List' implies a read-only operation, but the description does not explicitly confirm non-mutation, mention pagination, rate limits, or output format. For a zero-parameter listing tool, the risk is low, but the lack of explicit safety disclosure is a minor gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, self-contained sentence with no filler or repetition. It is front-loaded and efficiently conveys the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool (no parameters, no output schema, no annotations), the description is nearly complete. It states the resource and action, though it could improve by noting what kind of data is returned (e.g., model IDs) or its role relative to sibling generation tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has zero parameters, so the description need not explain parameter semantics. The baseline of 4 is appropriate because there is nothing to add beyond the existing schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and resource ('available foundation models in watsonx.ai'), clearly distinguishing it from sibling tools like watsonx_generate, watsonx_embeddings, and watsonx_chat which operate on models rather than list them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. It does not state that this is a prerequisite for model-consuming tools or mention any exclusions, leaving the agent to infer context from sibling names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• zos_connect_get_service_infoA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden of behavioral disclosure. It does add value by stating the output includes the OpenAPI specification, which is useful. However, it does not explicitly confirm read-only status or mention any prerequisites or potential errors, leaving some gaps in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, succinct sentence of 14 words. It is front-loaded with the core action ('Get detailed information') and immediately clarifies the scope ('including its OpenAPI specification'). No unnecessary words or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one simple parameter and no output schema, the description is mostly complete. It explains both the general purpose (detailed info) and the specific output (OpenAPI specification), which helps the agent understand what to expect. However, it does not mention any usage limitations or explicitly differentiate from list_services, which would make it fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already has a full description for the only parameter (service_name: 'Name of the z/OS Connect service'), providing 100% coverage. The tool description does not add any additional meaning or constraints for this parameter, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb action ('Get') and the specific resource ('detailed information about a z/OS Connect service') along with a key output detail ('including its OpenAPI specification'). This distinguishes it from sibling tools like zos_connect_list_services (list) and zos_connect_call_service (call) by focusing on retrieval of detailed spec info for a single service.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool is for obtaining detailed service information, but it does not explicitly state when to use it over alternatives or mention any exclusions. For example, it could have said 'Use this instead of zos_connect_list_services when you need the OpenAPI specification.' Without such guidance, the usage context is only implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• zos_connect_list_servicesA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosure. It mentions the environment configuration requirement, which is useful, but it does not describe the return format, error behavior, or explicitly state that the operation is read-only. For a simple list operation, this is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences each add value: one states the purpose, the other states a critical prerequisite. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter list tool with no output schema, the description is mostly complete. It identifies the resource and environment requirement. However, it could briefly mention what a successful response contains (e.g., list of service names), since there is no output schema to infer this from.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. The description correctly adds no parameter-specific details, as none are needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and resource ('available z/OS Connect services') with a clarifying parenthetical ('RESTful APIs to mainframe programs'). It clearly distinguishes from sibling tools like zos_connect_get_service_info and zos_connect_call_service.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly states a prerequisite: ZOS_CONNECT_URL must be configured. This gives context on when the tool can be used. However, it does not explicitly mention alternatives or exclusions, such as using get_service_info for individual service details.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.