Server Quality Checklist

Tool Scores

• destroy_rentalA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing behavior. It clearly states that the tool stops the GPU, stops billing, is safe to call twice (idempotent), and requires user confirmation. This is strong behavioral disclosure, though it doesn't mention reversibility or what happens to data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three short sentences, front-loaded with the core purpose ('Stop a rental'), then adding critical behavioral notes. Every sentence adds value and there is no filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema), the description covers the essential context: what happens, idempotency, and the need for confirmation. It doesn't describe return values or error conditions, but these are less critical for a straightforward destroy action.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has one parameter (rental_id) with no description (schema description coverage is 0%). The description does not mention or explain the parameter at all, leaving the agent to infer meaning from the name only. Since coverage is low and the description provides no compensation, the parameter semantics are under-specified.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Stop') and resource ('a rental'), and clearly states the main effects: stopping the GPU and stopping billing. This distinguishes it clearly from siblings like list_rentals, get_rental, create_rental, and get_balance.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for when to use the tool: as a 'money brake' to stop a rental, and explicitly instructs to confirm with the user unless they already asked to stop it. It doesn't mention alternative tools, but the use case is evident and the confirmation guidance is valuable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_rentalsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral disclosure burden. It explicitly states 'Read-only,' which is a key safety trait, and clarifies that 'active' means 'what's running and billing.' This adds meaningful context beyond the bare schema, though it does not mention pagination or return format, which are common for list tools.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: four short sentences that front-load the purpose, then efficiently cover defaults and the read-only nature. Every sentence contributes value without repetition, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has only one optional parameter, no output schema, and no annotations, the description covers all essential aspects: purpose, default behavior, parameter usage, and safety. It does not describe return values or pagination, but for a simple list operation this is a minor omission, and the description is otherwise complete enough for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already defines the status parameter with an enum and default, but the description adds semantic meaning by explaining that the default includes only active rentals and that 'all' gives full history. This directly clarifies what each value represents, going beyond the schema's generic description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description begins with 'List the account's rentals,' which uses a specific verb and resource, clearly distinguishing it from siblings like get_rental (single rental) and create_rental/destroy_rental. The scope ('account's rentals') and default behavior ('active ones') further clarify its purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: by default only active rentals are returned, and passing status='all' retrieves full history. It does not explicitly mention alternatives, but the sibling names make the appropriate tool obvious, and the guidance on the status parameter is practically sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_balanceA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and explicitly discloses 'Read-only, no cost,' indicating no side effects or charges. It does not mention auth or return format, but for a simple balance check this is meaningful and sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that states the action first and then adds concise, relevant context. Every word earns its place with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter read-only balance tool, the description is fully complete: it explains what is being retrieved, how it relates to rentals, and that the call is safe and free. No output schema is necessary for this simple case.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and an empty input schema, so the baseline is 4. The description confirms no input is needed and clarifies the operation, adding no unnecessary parameter detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly names the action ('Get the account's credit balance') and adds billing context ('rentals are billed per minute against it'), clearly distinguishing this from the GPU/rental sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context that this relates to rental billing, implying it should be used to check available credit before or during rental operations. It does not explicitly name alternatives or exclusions, but sibling tools are obviously different.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_gpusA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explicitly states 'Read-only, no cost,' which is a key safety signal. It also indicates real-time availability ('right now') and the action to take with results, adding meaningful behavioral context beyond a simple list.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four short sentences, each with a distinct purpose: main action, usage flow, safety indication, and selection heuristic. No redundancy or filler, and the most important information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a parameterless list tool with no output schema, the description covers all necessary context: what it lists, what data it provides, how to use the result (node_id), safety profile (read-only), and a practical usage guideline. This is complete for an agent to invoke and interpret the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and an empty schema, so there is no parameter-specific information to provide. The baseline of 4 for no parameters is appropriate; the description doesn't need to compensate for any schema gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists GPU nodes available to rent, enumerating specific fields (price, model, VRAM, CUDA compute capability, country). It distinguishes itself from sibling tools like list_rentals by focusing on available inventory rather than user's current rentals.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context that this is a discovery step before renting ('Rent by the returned node_id') and offers selection guidance for LLM work (compute_cap >= 8.9). However, it doesn't explicitly say when not to use it or contrast with alternatives, so it stops just short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_rentalA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does an excellent job. It discloses the blocking/waiting behavior, the ~24s internal timeout, the total provisioning time (~20-40s), the connection_pending readiness flag, and declares the tool 'Read-only.' This gives an agent a clear mental model of side effects and timing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Every sentence adds unique value: purpose, usage timing, wait behavior, anti-loop guidance, timing estimate, readiness condition, and read-only flag. The content is front-loaded and the length is justified for the behavioral nuance it conveys.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-param tool with no output schema, the description covers what the tool returns (status, Jupyter URL, token), when it's ready, how long to wait, and how to interact with the rental lifecycle (create → get → destroy). It is fully complete for the agent's needs.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must explain rental_id. It does so indirectly: 'Get one rental's status' and 'After create_rental, call this' implies the rental_id comes from create_rental. While not a formal param definition, this is sufficient context for a single obvious parameter. It earns above baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Get') and resource ('one rental's status and connection details (Jupyter URL + token)'), clearly distinguishing this from sibling list_rentals. It immediately conveys the tool's unique purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly provides when-to-use guidance: 'After create_rental, call this to get the URL.' It also gives strong behavioral guidance: warns against tight loops, explains the internal wait (~24s), and states the readiness condition. This is far beyond a minimal description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_rentalA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully discloses critical behavior: per-minute billing, asynchronous pending status, provisioning time, idempotency key generation, and default open-ended duration. This exceeds transparency expectations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but every sentence serves a purpose—cost, confirmation, async flow, timing, next steps, duration policy, idempotency. It is front-loaded with the core action and wastes no words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (billing, async provisioning, open-ended vs. capped duration) and lack of output schema, the description covers all essential aspects: return shape (rental_id/status), follow-up action, cost warning, and stopping mechanism. Nothing critical is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although the schema already documents all parameters, the description adds meaningful context: how 'hours' behaves relative to the default, that node_id comes from list_gpus, and that type only supports 'jupyter'. This enriches the agent's understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Start renting a GPU', which clearly identifies the action and resource. It distinguishes itself from sibling tools focused on listing, getting, or destroying rentals.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit operational guidance: confirm with user, use open-ended by default, only pass 'hours' when user requests a limit, and call get_rental once instead of polling. This is exactly the kind of when/how guidance needed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.