Server Quality Checklist

Tool Scores

• adobe_get_userA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the transparency burden. It discloses a rate limit and that the input can be an email or Adobe username, but it doesn't mention error behavior, permissions, or explicitly state it's a read-only operation (though 'get' implies it). The rate limit adds some behavioral context, but other aspects are missing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with three sentences covering purpose, rate limit, and parameter usage. It's front-loaded with the main action and does not include excessive detail, though the rate limit might be optional for tool selection.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-get tool, the description provides the core purpose and a key parameter detail, and the output schema presumably handles return structure. However, it lacks explicit guidance on alternative tools and edge-case behavior, so it's not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description expands on the schema's bare 'email' parameter by explaining it can be an Adobe username string as well, adding meaningful semantics beyond the schema. This is the only parameter, and the description compensates for the 0% schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'gets a single user's details' with a specific verb and resource, and distinguishes from sibling 'adobe_list_users' by focusing on a single user. The inclusion of 'group/product profile memberships' adds specific scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for retrieving one user's information, contrasting with the 'list' sibling tools, but it doesn't explicitly tell the agent when to choose this over others or mention any exclusions. This is implied usage based on the word 'single'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• adobe_remove_licenseA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It adds a rate limit (10/min per client, 100/min globally) and implies the action is a removal, but it does not mention side effects, idempotency, or error behavior (e.g., if the user is not in the profile).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded with the primary action, then a rate limit note, then an Args section. Every sentence contributes value, and the structure is easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the core purpose, rate limit, and parameter meanings, and references a sibling tool for obtaining valid values. Given that an output schema exists, return values need not be explained. However, it omits edge-case behavior such as handling missing users, which prevents a perfect score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains both parameters: email is 'the user's email address to revoke the license from' and product_profile is 'the exact product profile name (see adobe_list_product_profiles).' This adds meaningful usage guidance beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Revoke a license by removing a user from a product profile,' which is a specific verb+resource statement that clearly differentiates it from siblings like adobe_assign_license. It conveys both the action and the mechanism.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides context by noting the exact product profile name should be sourced from adobe_list_product_profiles, which is a useful usage hint. However, it does not explicitly contrast with alternatives or state when not to use it, though the purpose makes the primary use case evident.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• adobe_assign_licenseA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavior. It includes a rate limit (10 requests/min per client, 100/min globally) which is useful. However, it does not disclose side effects (e.g., consuming a license seat, irreversibility) or permission requirements, leaving a significant gap for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the main action, followed by a rate-limiting note and concise parameter explanations. No wasted words; the example and cross-reference earn their place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple two-parameter tool with an output schema present, the description covers purpose, rate limit, and parameter semantics. It lacks error-case behavior or idempotency notes, but for this complexity level it is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no parameter descriptions (0% coverage), but the description explicitly explains both parameters: email is the user's email to grant the license, and product_profile must be the exact name, referencing adobe_list_product_profiles. This adds clear semantics beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Assign a license by adding a user to a product profile'), names the resource (license) and mechanism (adding to profile), and gives an example (Acrobat Pro profile). This distinguishes it from sibling tools like adobe_remove_license and adobe_list_users.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool: whenever a license needs to be granted. It also provides a cross-reference to adobe_list_product_profiles for obtaining the exact profile name, which is helpful prerequisite guidance. However, it does not explicitly exclude cases (e.g., user already has a license) or directly compare with adobe_remove_license.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• adobe_list_usersA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full responsibility for behavioral disclosure. It adds valuable context by disclosing rate limits and pagination mechanics beyond a simple 'list users' statement. It does not cover auth or error behavior, but for a simple list operation this is reasonably transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-structured: a one-line summary, a rate-limit note, and an Args block. Every sentence earns its place without redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a low-complexity tool with one optional parameter and an output schema present. The description covers rate limits and pagination semantics comprehensively, and the output schema handles return value documentation. No significant gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides only a 'page' integer with no description (0% coverage), but the description fully explains it: zero-indexed, default 0, and how to use the 'lastPage' field to continue pagination. This adds critical meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'List users in the Adobe organization' with a specific verb ('List') and resource ('users'), and it naturally distinguishes from siblings like adobe_get_user (single user retrieval) and adobe_list_product_profiles (different resource).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides concrete usage context, including rate limits (25/min client, 100/min global) and pagination instructions referencing the 'lastPage' field. It implies this is the tool for enumerating users, but does not explicitly mention alternatives or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• adobe_list_product_profilesA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses behavior: it filters API responses to type=='PRODUCT_PROFILE' due to Adobe's API limitation, and provides rate limits (5/min per client, 100/min globally). This is valuable insight beyond a simple listing operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a clear one-line purpose, followed by necessary behavioral notes and parameter details. Every sentence earns its place without unnecessary fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with one optional parameter and an existing output schema, the description covers the essential aspects: purpose, filtering logic, rate limits, and parameter semantics. No significant gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides only a default value with no description. The description's Args section adds meaning: 'Page number to fetch, zero-indexed (default: 0)', which is essential for correct usage. This fully compensates for the 0% schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'List product profiles available for license assignment' with examples (Acrobat Pro/Standard), using a specific verb+resource. It distinguishes from sibling tools like adobe_list_users by focusing on product profiles.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The tool's purpose for license assignment is clear and the context implies when to use it. However, it does not explicitly state alternatives or when-not-to-use scenarios, though the filtering note provides helpful context about the API's behavior.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.