Server Quality Checklist

Tool Scores

• get_service_usageC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states this is a 'Get' operation, implying read-only behavior, but doesn't clarify authentication needs, rate limits, error conditions, or what the statistics include (e.g., time range, metrics). This leaves significant gaps for a tool that likely returns data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's function without unnecessary words. It is appropriately sized and front-loaded, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete for a tool that returns usage statistics. It doesn't explain what the statistics include (e.g., counts, dates, units), potential limitations, or how to interpret results, leaving the agent with insufficient context for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the 'service' parameter fully documented via its enum and description. The tool description adds no additional parameter information beyond what the schema provides, so it meets the baseline for high schema coverage without compensating value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the resource 'usage statistics for a specific service', making the purpose immediately understandable. However, it doesn't distinguish this tool from its sibling 'get_usage_stats', which appears to be a similar usage-related tool, preventing a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_usage_stats' or 'check_alerts'. It lacks context about prerequisites, timing, or exclusions, leaving the agent with minimal usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_subscription_tierC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only states what the tool does, not how it behaves. It doesn't disclose if this is a mutation requiring admin rights, if it's reversible, what side effects occur, or error conditions. For a write operation with zero annotation coverage, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's front-loaded with the core action and includes all necessary tier values without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations and no output schema, the description is incomplete. It lacks information on permissions, side effects, return values, or error handling. Given the complexity of changing subscription tiers, more context is needed for safe and effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the parameter 'tier' fully documented in the schema including its enum values. The description adds no additional parameter semantics beyond restating the tier options, so it meets the baseline of 3 where the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Set') and the resource ('subscription tier'), with specific tier values listed. It distinguishes from siblings like 'get_subscription_info' (read vs write) but doesn't explicitly differentiate from other write operations like 'update_usage'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives is provided. The description doesn't mention prerequisites, permissions needed, or when not to use it (e.g., vs 'update_usage'). It's a basic functional statement without contextual usage advice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_usage_statsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool retrieves data ('Get'), implying a read-only operation, but doesn't mention any constraints like authentication requirements, rate limits, or whether the data is real-time or cached. This leaves significant gaps for a tool that likely accesses sensitive usage information.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's function without any fluff. It's front-loaded with the core action and resource, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It doesn't specify what 'usage statistics and limits' include (e.g., API calls, storage, user counts) or the format of the response, which is critical for an agent to use this tool effectively in a context with sibling tools that might overlap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description appropriately doesn't discuss parameters, focusing instead on the tool's purpose. A baseline of 4 is applied since the schema fully handles the parameter aspect.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the resource 'current Cursor Pro usage statistics and limits', making the purpose unambiguous. However, it doesn't explicitly differentiate from sibling tools like 'get_service_usage' or 'get_subscription_info', which likely provide related but different information.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_service_usage' or 'get_subscription_info'. It lacks context about what specific statistics or limits are included, leaving the agent to guess based on tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_alertsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action ('Check') but doesn't describe what 'approaching or exceeding limits' means operationally, whether this is a read-only operation, what permissions are needed, or how results are returned. This leaves significant gaps for a tool that likely interacts with system limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without any wasted words. It is appropriately sized for a zero-parameter tool and front-loaded with the essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with no output schema, the description is minimally adequate but lacks detail on what 'check' entails (e.g., returns alerts, thresholds, or just status). Given the sibling tools suggest a usage/limits context, more specificity about behavioral aspects would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so the schema fully documents the absence of inputs. The description appropriately doesn't add parameter details, maintaining focus on the tool's purpose without redundancy. Baseline 4 is correct for zero-parameter tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Check') and resource ('services approaching or exceeding limits'), making it immediately understandable. It doesn't explicitly distinguish from siblings like 'get_service_usage' or 'get_usage_stats', but the focus on 'limits' provides some differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'get_service_usage' or 'get_usage_stats'. The description implies usage for monitoring limit thresholds but doesn't specify prerequisites, frequency, or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_subscription_infoB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool retrieves information ('Get'), implying a read-only operation, but doesn't mention potential side effects, authentication requirements, rate limits, or error handling. This leaves significant gaps in understanding how the tool behaves in practice.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without any unnecessary words. It is front-loaded with the core action and resource, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no output schema, no annotations), the description is adequate as a basic read operation. However, it lacks details on return values (e.g., format of subscription info), error cases, or dependencies, which could be important for an AI agent to use it correctly in a broader context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and the input schema has 100% description coverage (though empty). The description doesn't need to add parameter details, as there are none to explain. A baseline score of 4 is appropriate since the schema fully covers the lack of parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get') and resource ('current subscription tier and limits'), making it easy to understand what it does. However, it doesn't explicitly differentiate from sibling tools like 'get_service_usage' or 'get_usage_stats', which might also retrieve subscription-related information.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_service_usage' or 'get_usage_stats'. It lacks context about prerequisites, such as whether authentication is needed, or exclusions, such as when other tools might be more appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_usageB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden but lacks critical behavioral details. 'Update' implies mutation, but it doesn't disclose permissions needed, whether changes are reversible, rate limits, or what the response looks like. The testing/manual context hints at caution but is insufficient for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste—every word contributes to purpose and context. It's appropriately sized and front-loaded with the core action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations and no output schema, the description is incomplete. It lacks details on behavioral traits, response format, and error handling, leaving significant gaps for an agent to use it safely and effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so parameters are fully documented in the schema. The description adds no parameter-specific information beyond the general purpose, meeting the baseline for high schema coverage without adding extra value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Update') and resource ('usage statistics'), and specifies the context ('for testing or manual updates'). It distinguishes from sibling tools like 'get_usage_stats' (read) and 'set_subscription_tier' (different resource). However, it doesn't explicitly contrast with all siblings like 'check_alerts'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context ('for testing or manual updates'), suggesting this is not for routine operations. However, it doesn't provide explicit when-to-use rules, alternatives (e.g., vs. 'set_subscription_tier'), or prerequisites, leaving some ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.