Server Quality Checklist

Tool Scores

• cycles_list_reservationsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint=true, so the description adds minimal behavioral context. Does not disclose pagination (cursor/limit) or any other traits beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no wasted words. Front-loaded with action and resource, then filters, then use cases. Excellent structure.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema, the description does not cover the majority of input parameters or their semantics. With 10 parameters, this is insufficient for an agent to select and invoke correctly without external knowledge.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%. Description only mentions 'status' and 'subject fields' vaguely, but fails to explain 8 other parameters (app, agent, limit, cursor, etc.). Heavily under-documented for a 10-param tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states verb (List), resource (reservations), and optional filter (status). Immediately distinguishes from sibling tools by its read-only nature and specific use cases.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states use cases: 'debugging stuck reservations or auditing budget usage.' No explicit when-not-to-use, but context implies read-only. Sibling differentiation is indirect.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cycles_create_eventA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Adds value beyond annotations: 'applied atomically to all derived scopes' and 'fire-and-forget'. Annotations already indicate idempotent and non-destructive. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences, no redundancy. Every sentence adds meaningful information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 8 parameters, 3 required, nested objects, and no schema descriptions, the description lacks sufficient detail for correct invocation. Missing parameter guidance.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Description does not explain any parameters despite 0% schema description coverage and complex nested objects. Agent relies solely on schema structure without contextual intent.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verb 'Record' and resource 'usage event'. It distinguishes from siblings by explicitly contrasting with the reserve/commit lifecycle.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Describes when to use: 'fire-and-forget metering of completed operations where pre-estimation is not available'. Implicitly excludes the reserve/commit lifecycle. Lacks explicit exclusion of alternatives but clear context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cycles_extendA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds context beyond annotations by stating that the reserved amount is unchanged and that it acts as a heartbeat, which aligns with the idempotent and non-destructive hints. It does not contradict annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, with two sentences that front-load the core action and then provide additional context. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description provides the essential purpose and a key usage scenario, but lacks details on parameter semantics (e.g., extendByMs limits, idempotencyKey purpose) and does not mention whether the operation can fail or any side effects. The presence of an output schema partially mitigates return value documentation, but parameter documentation is entirely absent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description does not explain individual parameters (e.g., what extendByMs represents, what idempotencyKey is for) beyond the implied context from the main description. The agent must infer parameter meanings from the tool title and description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Extend the TTL') and the resource ('active reservation'), and explicitly notes what it does not change, distinguishing it from sibling tools like cycles_reserve.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides a specific use case ('heartbeat for long-running operations') and a clear behavior note ('does not change the reserved amount'), but does not explicitly contrast with alternatives like cycles_reserve or cycles_commit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cycles_commitA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate idempotency and non-destructiveness. Description adds that it must follow a reservation, commits actual usage, and releases unused amount, providing lifecycle context beyond structured annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences: purpose, usage order, and effect. Every sentence adds value with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers core behavior and workflow dependency. Lacks details on error conditions or idempotency handling, but has output schema to document return values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain any parameters (e.g., reservationId, idempotencyKey, actual, metrics, metadata). Agent must infer from names alone, which is insufficient for correct invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it commits actual usage after an operation completes, linking to cycles_reserve. Distinguishes from siblings by specifying it finalizes the charge and releases unused reserved amount.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says to call after cycles_reserve regardless of success/failure. Lacks mention of when not to use (e.g., alternatives like cycles_release), but the context is clear for the intended workflow.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cycles_decideA
  4.1/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description transparently states it does not create a reservation, aligning with annotations (destructiveHint=false, idempotentHint=true). It clarifies the read behavior beyond the annotations, ensuring the agent understands it is a non-destructive check.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise with two short sentences, front-loading the core purpose and usage guidelines. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having a clear purpose, the description completely omits parameter semantics for a tool with 5 parameters (3 required) and nested objects. The agent lacks crucial information to correctly construct inputs.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, meaning the JSON schema has no parameter descriptions. The description provides no explanation of any parameters (action, subject, estimate, metadata, idempotencyKey), leaving the agent to infer from schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it is a 'preflight check' to ask if an action is allowed without reserving budget, and explicitly says it does not create a reservation. This distinguishes it from sibling tools like cycles_reserve.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance: 'Use at workflow start to decide strategy' and directs to cycles_reserve for concurrency-safe budget locking, giving a clear when-to-use and when-not-to-use with an alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cycles_get_reservationA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark readOnlyHint: true, indicating safe read. The description adds value by detailing the returned fields (status, subject, action, etc.) and the debugging/monitoring use case, going beyond what annotations provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with purpose, and every word adds value. No redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple retrieval tool with one parameter, existing annotations, and an output schema, the description covers purpose, returned data, and use case adequately.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% (no parameter descriptions), and the description only says 'by ID' without explaining how to obtain or format the reservationId. This is insufficient compensation for the lack of schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action 'Get details of a specific reservation by ID' and lists the specific attributes returned. The tool is distinct from siblings like cycles_list_reservations (listing all) and cycles_reserve (creating).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description notes that it is 'useful for debugging and monitoring long-running operations,' providing context. However, it does not explicitly exclude alternatives or state when not to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cycles_reserveA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate idempotentHint=true and destructiveHint=false, so the description adds context about returning a reservation_id for later commit/release and the ALLOW/not ALLOW decision. However, it does not describe any potential side effects like holding budget or timeout behavior, which would be valuable for transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, highly concise, with key information front-loaded. Every sentence earns its place: first defines the purpose, second provides usage scope and alternative.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (9 parameters, nested objects, enums, and a sibling set of 8 other tools), the description is too brief. It covers the core purpose and usage guidelines but lacks details on parameter semantics and expected behavior. The output schema exists, but the description should at least hint at what parameters are essential.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema coverage is 0%, meaning the description does not explain any of the 9 parameters. While the schema itself defines constraints and enums, the description adds no guidance on how to set parameters like estimate, overagePolicy, or idempotencyKey. This is a significant gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action ('Reserve budget'), the resource ('before a costly operation'), and the outcome ('returns a reservation_id'). It also distinguishes from the sibling cycles_decide by explicitly naming it and explaining the difference (reserving vs. lightweight preflight check).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool ('before a costly operation') and when not to use it ('For lightweight preflight checks without reserving, use cycles_decide instead'). It also provides a clear instruction: 'If decision is not ALLOW, do not proceed.'

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cycles_releaseA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide idempotentHint=true and destructiveHint=false. The description adds value by explaining the effect (returns released budget amount back to pool) and the non-committing nature. It does not contradict annotations and provides context beyond them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise: two sentences, front-loaded with the primary action. Every sentence adds value, with no redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that an output schema exists (not shown), the description is largely complete: it states purpose, usage, and effect. It lacks error conditions or prerequisites (e.g., reservation must exist), but these are minor gaps. The sibling tools provide context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. However, the description does not explain any parameters. While parameter names are somewhat self-explanatory, the lack of any parameter details leaves room for ambiguity, especially for the 'reason' parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description directly states that the tool releases a reservation without committing, clearly distinguishing it from cycles_commit and cycles_reserve. The verb 'Release' combined with 'without committing' precisely conveys the action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The explicit instruction 'Use when an operation is cancelled, skipped, or fails before execution' provides clear when-to-use guidance. It also implies when not to use (e.g., when committing, use cycles_commit instead).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cycles_check_balanceA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint=true and openWorldHint=false. The description adds value by explaining that balances can change between check and action, which is important behavioral context beyond what annotations provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two sentences. The first sentence states the main purpose, and the second adds a constraint and warning. No superfluous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema (not shown but indicated), the description sufficiently covers the tool's purpose and key constraints. It lacks details on all parameters, but the essential requirement (at least one filter) is clearly stated. The warning about balance changes adds important context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It clarifies that at least one of the listed filters (tenant, workspace, app, workflow, agent, or toolset) is required, but does not explain other parameters like limit, cursor, or includeChildren. Partial compensation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool checks the current budget balance for a scope and lists the specific amounts returned (remaining, reserved, spent, allocated, debt). It also distinguishes itself from the sibling tool cycles_reserve by warning not to use it as a substitute.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states that at least one subject filter is required (tenant, workspace, app, workflow, agent, or toolset) and warns not to use this tool as a substitute for cycles_reserve. This provides clear when-to-use and when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.