Server Quality Checklist

Tool Scores

• list-usersB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions pagination, which is useful, but fails to address critical aspects like authentication requirements, rate limits, error handling, or the structure of returned data. For a tool with no annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('Lista todos os usuários') and adds essential context ('cadastrados no Clerk com paginação'). There is no wasted verbiage, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (3 parameters, pagination), no annotations, and the presence of an output schema (which handles return values), the description is minimally adequate. It covers the basic purpose but lacks details on behavioral traits, usage context, and parameter semantics, leaving room for improvement in completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds no parameter-specific information beyond implying pagination (related to limit and offset). With 0% schema description coverage and 3 parameters, the schema documents constraints (e.g., min/max, enums, defaults), but the description doesn't compensate by explaining parameter meanings or usage. Baseline 3 is appropriate as the schema handles documentation, but the description adds minimal value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Lista') and resource ('usuários cadastrados no Clerk'), specifying pagination as a key feature. It distinguishes from siblings like delete-user, lock-user, and unlock-user by focusing on retrieval rather than modification, though it doesn't explicitly name alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving all users with pagination, but provides no explicit guidance on when to use this tool versus alternatives or any prerequisites. The context suggests it's for read operations, but lacks clear exclusions or comparisons with sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• unlock-userB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states the action is to unlock a user, implying a mutation, but doesn't disclose behavioral traits like required permissions, whether it's reversible, side effects, or rate limits. The description is minimal and lacks crucial operational context for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that states the action and outcome without unnecessary words. It's front-loaded with the core purpose and appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has an output schema (which handles return values) and minimal parameters, the description covers the basic purpose. However, as a mutation tool with no annotations, it lacks details on permissions, error cases, or behavioral nuances, making it incomplete for safe operation despite the output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage and only 1 parameter, the description doesn't add specific parameter details, but the tool's purpose inherently clarifies that 'userId' identifies the user to unlock. Since there's only one required parameter and the action is straightforward, the description provides adequate semantic context without needing explicit parameter explanation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Desbloqueia' - unlocks) and the resource ('um usuário do Clerk'), specifying that it allows the user to log in again. It distinguishes from 'lock-user' by being the opposite operation, though it doesn't explicitly differentiate from other siblings like 'delete-user' or 'list-users'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a user needs to be unlocked to regain login access, but doesn't provide explicit guidance on when to use this versus alternatives like 'delete-user' or prerequisites. It mentions the outcome ('permite que ele faça login novamente') which gives some context, but lacks clear when/when-not instructions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lock-userA
  3.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states the action (blocks user, prevents login) which implies a destructive/mutative operation, but doesn't disclose important behavioral traits like whether this is reversible, what permissions are required, if it affects existing sessions, or what the response looks like. For a mutation tool with zero annotation coverage, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that communicates the core functionality without any wasted words. It's appropriately sized for a simple tool and front-loads the essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given this is a mutation tool with no annotations but with an output schema (which handles return values), the description provides the basic purpose but lacks important context about behavioral implications, permissions, and reversibility. It's minimally adequate but has clear gaps for a tool that modifies user access.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage and only 1 parameter, the description doesn't explicitly mention the 'userId' parameter. However, the context ('um usuário do Clerk') implies what needs to be identified. For a single-parameter tool, this provides adequate semantic context despite not naming the parameter directly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Bloqueia' - blocks) and resource ('um usuário do Clerk'), and distinguishes it from sibling tools like 'unlock-user' by specifying it prevents login. It goes beyond just restating the name/title by explaining the functional outcome.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context (when you need to prevent a user from logging in), but doesn't explicitly state when to use this vs alternatives like 'delete-user' or 'unlock-user'. It provides clear functional intent but lacks explicit comparative guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete-userA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses critical behavioral traits: the action is permanent and irreversible, which is essential for a destructive operation. However, it doesn't mention authentication requirements, rate limits, or what happens to associated data, leaving some gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core action and followed by a critical warning. Every word earns its place, with no redundancy or unnecessary details, making it highly efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (destructive operation), no annotations, and an output schema present, the description is mostly complete. It covers the irreversible nature but lacks details on permissions, error handling, or output format. The output schema reduces the need to explain return values, but more behavioral context would help.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 1 parameter with 0% description coverage, so the description must compensate. It adds meaning by specifying that the parameter is the user ID for deletion, though it doesn't detail format or constraints. Since there's only one parameter, the baseline is high, and the description provides adequate context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('deleta permanentemente') and the resource ('um usuário do Clerk'), specifying it's done by ID. It distinguishes from siblings like list-users, lock-user, and unlock-user by focusing on permanent deletion rather than listing or state changes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when permanent deletion is needed, but doesn't explicitly state when to use this tool versus alternatives like lock-user for temporary deactivation or list-users for checking users first. No explicit exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.