Server Quality Checklist

Tool Scores

• list_experiment_templatesB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action ('List') and return type ('JSON string'), but lacks details on permissions, rate limits, pagination, or error handling. This is inadequate for a tool that likely interacts with AWS services, where such context is critical.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with the core purpose in the first sentence and parameter/return details in a structured format. It avoids unnecessary verbosity, though the 'Args' and 'Returns' sections could be more integrated into the flow for slightly better readability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of AWS FIS operations, no annotations, and no output schema, the description is incomplete. It doesn't explain the structure of the returned JSON, error cases, or dependencies like required IAM permissions. For a tool in this context, more behavioral and output details are needed to be fully helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful context for the single parameter ('region') by specifying its purpose ('AWS region to query') and default value ('us-east-1'), which compensates for the 0% schema description coverage. Since there's only one parameter, this is sufficient to achieve a high score, though it doesn't elaborate on format or constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('AWS FIS experiment templates') with scope ('in the specified region'), making the purpose specific and understandable. However, it doesn't explicitly differentiate from sibling tools like 'list_experiments' or 'list_action_types', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives such as 'list_experiments' or 'get_experiment_template'. It mentions the region parameter but offers no context about prerequisites, timing, or exclusions, leaving usage decisions ambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_experiment_templateB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. While 'Create' implies a write/mutation operation, the description doesn't mention permission requirements, whether this is idempotent, what happens on failure, rate limits, or any side effects. It provides basic functional information but lacks critical behavioral context for a creation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured with a clear purpose statement followed by organized parameter documentation. Every sentence serves a purpose, though the 'Returns' section could be slightly more informative given there's no output schema. The formatting with Args/Returns sections is helpful.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex creation tool with 7 parameters (6 required), nested objects, no annotations, and no output schema, the description provides adequate functional coverage but lacks important contextual information. It explains what parameters are needed but doesn't address behavioral aspects, error handling, or relationship to sibling tools that would help an agent use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description compensates well by listing all 7 parameters with brief explanations of their purpose. It clarifies that 'targets' and 'actions' are dictionaries, 'stop_conditions' is a list, and provides the default for 'region'. This adds meaningful semantic context beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Create') and resource ('new AWS FIS experiment template'), making the purpose immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'generate_template_example' or explain how this creation differs from other template-related operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'generate_template_example' or 'get_experiment_template'. There's no mention of prerequisites, typical use cases, or scenarios where this tool would be preferred over other template-related operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_experimentsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions the return format ('JSON string') but lacks critical behavioral details: whether this is a read-only operation, if it requires authentication, rate limits, pagination, or error handling. For a cloud service tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded with the core purpose. The structured 'Args' and 'Returns' sections are clear, though slightly redundant with the opening sentence. Every sentence adds value, but minor trimming could improve efficiency.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 1 parameter, no annotations, and no output schema, the description is minimally adequate. It covers the basic operation and parameter but lacks details on authentication, error cases, or output structure beyond 'JSON string'. For a cloud API tool, this leaves gaps in usability for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With only 1 parameter and 0% schema description coverage, the description compensates well by explaining the parameter's purpose ('AWS region to query') and providing a default value. It adds meaningful context beyond the schema's basic type and title, though it doesn't detail format constraints or valid region values.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('AWS FIS experiments') with scope ('in the specified region'), making the purpose specific and understandable. It distinguishes from siblings like 'get_experiment' (single item) and 'list_experiment_templates' (different resource), though not explicitly named. A 5 would require explicit sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'list_experiment_templates' or 'get_experiment'. The description implies usage for listing experiments but lacks context on prerequisites, filtering, or exclusions. This leaves the agent with minimal direction for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• stop_experimentB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states the tool stops an experiment and returns JSON, but doesn't cover critical aspects like whether this is a destructive/mutative action (implied but not explicit), potential side effects, authentication needs, rate limits, or error conditions. This leaves significant gaps for agent understanding.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded with the core purpose in the first sentence. The Args/Returns sections are structured efficiently, though the 'Returns' line could be more specific about the JSON content. No wasted sentences, but minor room for refinement.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (stopping AWS experiments), lack of annotations, and no output schema, the description is minimally adequate. It covers basic purpose and parameters but misses behavioral details like what 'stop' entails operationally, success/failure states, or how it interacts with sibling tools. More context would help the agent use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful context for both parameters beyond the schema's 0% coverage: it explains 'experiment_id' identifies the experiment to stop and 'region' specifies AWS region with a default. This compensates well for the schema's lack of descriptions, though it doesn't detail format constraints (e.g., ID patterns or valid regions).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Stop') and target resource ('a running AWS FIS experiment'), making the purpose immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'get_experiment' or 'start_experiment' beyond the obvious verb difference, which keeps it from a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_experiment' for checking status or 'start_experiment' for initiating experiments. It lacks context about prerequisites (e.g., experiment must be running) or exclusions, leaving usage decisions to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_experimentB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only states it retrieves information without disclosing behavioral traits like authentication needs, rate limits, error handling, or data freshness. It mentions the return format as 'JSON string,' which adds some context but lacks depth for a read operation in a cloud service.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with the core purpose stated first, followed by clear sections for Args and Returns. Every sentence earns its place by providing essential information without redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (read operation with 2 parameters), no annotations, and no output schema, the description is minimally adequate. It covers the purpose and parameters but lacks details on authentication, errors, or sibling tool differentiation, leaving gaps for an AI agent to infer usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful semantics beyond the input schema, which has 0% coverage. It explains that 'experiment_id' is for retrieving a specific experiment and 'region' is the AWS region to query with a default, clarifying their roles. Since there are only 2 parameters and the schema lacks descriptions, this compensation is effective.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as 'Get detailed information about a specific AWS FIS experiment,' which includes a specific verb ('Get') and resource ('AWS FIS experiment'). It distinguishes from siblings like 'list_experiments' by focusing on a single experiment rather than listing multiple, but doesn't explicitly mention all sibling differences.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when to choose 'get_experiment' over 'list_experiments' for overviews or other siblings for related operations, leaving usage context implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_action_typesB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions the return format ('JSON string') but lacks behavioral details like whether this is a read-only operation, if it requires specific permissions, rate limits, or pagination. For a tool with no annotations, this is insufficient disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with the core purpose, followed by clear sections for Args and Returns. Every sentence adds value without redundancy, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description provides basic purpose and parameter info but lacks details on behavior, error handling, or output structure. It's minimally adequate for a simple list tool but could be more complete, especially regarding the JSON return format.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description compensates by explaining the 'region' parameter's purpose ('AWS region to query') and default value. This adds meaningful context beyond the bare schema, though it could elaborate on region format or valid options.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List' and resource 'AWS FIS action types', making the purpose specific and understandable. However, it doesn't explicitly differentiate from sibling tools like 'list_experiments' or 'list_experiment_templates', which would require mentioning it's specifically about action types rather than experiments or templates.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description doesn't mention sibling tools or contexts where listing action types is appropriate, such as before creating an experiment template. It simply states what it does without usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_experiment_templateB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool deletes a template, implying a destructive mutation, but lacks details on permissions required, whether deletion is reversible, potential side effects, or rate limits. The return message mention is vague, offering minimal behavioral insight.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with the core purpose, followed by parameter and return details. It uses bullet points for clarity, with no redundant sentences. However, the return statement is somewhat vague, slightly reducing efficiency.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's destructive nature, no annotations, and no output schema, the description is minimally adequate. It covers the basic operation and parameters but lacks critical context like authentication needs, error conditions, or confirmation prompts. For a deletion tool, this leaves significant gaps in safety and usability guidance.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful context for both parameters: 'template_id' is explained as 'ID of the experiment template to delete', and 'region' specifies 'AWS region to use (default: us-east-1)'. With 0% schema description coverage, this compensates well by clarifying purpose and defaults, though it could detail format constraints (e.g., ID structure).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Delete') and resource ('AWS FIS experiment template'), making the purpose immediately understandable. It distinguishes this tool from siblings like 'create_experiment_template' or 'get_experiment_template' by specifying the destructive operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention prerequisites (e.g., ensuring the template is not in use), exclusions, or compare it to sibling tools like 'list_experiment_templates' for selection. Usage is implied but not explicitly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_template_exampleB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions the tool generates an example template and returns a JSON string, but lacks details on behavioral traits such as whether it's read-only, if it has side effects (e.g., simulating actions), rate limits, or authentication needs. This is a significant gap for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, followed by a structured 'Args' and 'Returns' section. Every sentence adds value without redundancy, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (3 parameters, no output schema, no annotations), the description is adequate but incomplete. It covers the purpose and parameters well, but lacks behavioral context (e.g., safety, side effects) and doesn't detail the output format beyond 'JSON string', which could be more specific for an example template.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful semantics beyond the input schema, which has 0% description coverage. It explains that 'target_type' is the 'Target resource type', 'action_type' is the 'Action type to perform', and 'region' is the 'AWS region to use', including default values. This compensates well for the schema's lack of descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Generate an example AWS FIS experiment template for a given target and action type.' It specifies the verb ('generate'), resource ('AWS FIS experiment template'), and scope ('example'), though it doesn't explicitly differentiate from siblings like 'create_experiment_template' beyond the 'example' qualifier.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied by the description as a tool for generating examples, but there's no explicit guidance on when to use this versus alternatives like 'create_experiment_template' for actual creation or 'get_experiment_template' for retrieving existing ones. The context suggests it's for learning or testing, but this isn't stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_experiment_templateB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states it retrieves detailed information but doesn't disclose behavioral traits like authentication needs (AWS credentials), rate limits, error handling, or whether it's a read-only operation. The description adds minimal context beyond the basic action, leaving significant gaps for a tool interacting with AWS services.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with the core purpose, followed by clear sections for Args and Returns. Every sentence earns its place: the first states the action, the Args explain parameters concisely, and the Returns sets expectations. No wasted words or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is moderately complete. It covers the basic purpose and parameters adequately but lacks context on authentication, error handling, or detailed return structure (only stating 'JSON string'). For a tool with AWS integration and 2 parameters, it should provide more operational guidance to be fully helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaningful semantics for both parameters: template_id is described as 'ID of the experiment template to retrieve', and region as 'AWS region to query (default: us-east-1)'. This clarifies purpose and default values beyond the schema's basic types, though it doesn't detail format constraints (e.g., region naming conventions).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the resource 'detailed information about a specific AWS FIS experiment template', making the purpose explicit. It distinguishes from siblings like list_experiment_templates (which lists multiple) and create_experiment_template (which creates new ones). However, it doesn't explicitly contrast with get_experiment (which retrieves experiment instances rather than templates), leaving slight room for improvement.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you need detailed info for a specific template ID, but doesn't explicitly state when to use this vs. alternatives like list_experiment_templates (for browsing) or get_experiment (for experiment instances). No exclusions or prerequisites are mentioned, such as authentication requirements or region availability.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• start_experimentA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions 'idempotency' for client_token, which adds some behavioral context, but lacks details on permissions needed, rate limits, what 'start' entails (e.g., immediate execution, costs), or error handling. For a mutation tool with no annotations, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, followed by a structured Args/Returns section. Every sentence adds value: the first states the action, and the parameter explanations are necessary given low schema coverage. No wasted words or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 3 parameters with 0% schema coverage and no output schema, the description is incomplete. It explains parameters but lacks details on return values beyond 'JSON string containing the started experiment information' (e.g., structure, fields). For a mutation tool with no annotations, more behavioral context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaning by explaining template_id ('ID of the experiment template to use'), region ('AWS region to use'), and client_token ('Optional client token for idempotency'), which clarifies purpose beyond schema titles. However, it doesn't specify format or constraints for template_id or region.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Start a new AWS FIS experiment') and resource ('based on an experiment template'), distinguishing it from siblings like create_experiment_template (creates templates) or stop_experiment (stops experiments). The verb 'Start' is precise and the AWS FIS context is explicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by specifying it starts experiments 'based on an experiment template', suggesting it should be used when a template exists, but it doesn't explicitly state when to use this vs. alternatives like create_experiment_template or list_experiments. No exclusions or clear alternatives are provided, leaving some ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.