Server Quality Checklist

Tool Scores

• genie_recipesA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly, idempotent, and openWorld hints. The description adds valuable behavioral context by noting that the stage field is authoritative and only stage=commercial represents live inventory, going beyond the annotations. No contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, immediately states the action, and contains zero filler. It is perfectly concise and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the small number of optional parameters (3), complete schema coverage, and rich annotations, the description provides sufficient context. It covers search dimensions and the key stage nuance. The lack of an output schema is not an issue since this is a search tool, and return format is implied. Minor room for improvement could be mentioning pagination or result scope.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers all parameters with 100% description coverage, so the baseline is 3. The description adds semantic value by mapping 'maturity, category, or text' to parameters and specifically highlighting that stage is authoritative, which enriches the stage parameter's meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Search the public recipe menu by maturity, category, or text' which is a specific verb+resource. It does not explicitly distinguish from sibling tools like genie_catalog, so it falls short of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool ('Search the public recipe menu') and adds important guidance about the stage field being authoritative, with only stage=commercial being live inventory. However, it does not mention alternatives or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• genie_quoteA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly, idempotent, and non-destructive, but the description adds valuable context beyond them: 'does not reserve capacity, create an account, or spend funds.' This clarifies side effects and safety, aligning with annotations without contradiction. The 'live' qualifier indicates real-time behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the main action, and every word adds value. It avoids redundancy and clearly communicates purpose and behavioral boundaries in a compact form.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with 3 optional parameters, good annotations, and no output schema, the description is largely complete. It covers purpose, safety, and what the quote is not. However, it does not explicitly state what the returned quote contains (e.g., total price in currency/credits), which could be useful given no output schema exists. Minor gap keeps it from a 5.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% parameter description coverage, so the baseline is 3. The tool description does not add any additional meaning about parameters (minutes, envelope, placement) beyond what the schema provides, but the schema itself already explains each parameter well, including defaults and enums.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Get a live, non-binding price quote.' The verb 'get' and resource 'price quote' are specific, and the title 'Quote a private inference locker' adds context. It distinguishes from siblings by emphasizing non-binding and read-only nature, setting it apart from status, catalog, recipes, and credit pricing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: use this tool to obtain a live, non-binding quote without reserving capacity or spending funds. However, it does not explicitly name alternatives or state when not to use it, such as 'use genie_credit_pricing for credit prices' or 'use genie_status for current state.' The context implies usage but lacks explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• genie_statusA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false, and the description adds the behavioral detail 'This never creates or purchases a locker,' which goes beyond annotations by explicitly ruling out purchase side effects. It also lists the specific data points included, providing useful context for what the read operation covers. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences: the first states the tool's function and lists outputs, the second clarifies the non-mutating behavior. It is front-loaded with the verb 'Read' and contains no filler or repetition. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only status tool with no parameters and no output schema, the description adequately covers the key information: what data is accessible (catalog, capacity, sellability, locker counts, payment-rail status) and the non-mutating guarantee. It does not describe the output format, but the enumerated list gives a strong expectation. The rich annotations compensate for any missing behavioral details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the input schema is empty. Per the baseline for 0 params, the description need not add parameter details. The description does not mention any parameters, which is appropriate and adds no confusion.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action: 'Read current catalog, capacity, sellability, locker counts, and payment-rail status.' It uses the verb 'read' and enumerates the resources, making it distinct from sibling tools like genie_catalog or genie_quote. The additional sentence 'This never creates or purchases a locker' further differentiates it from mutation-oriented tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for checking service status by saying 'Read current... status.' It also provides an explicit exclusion: 'never creates or purchases a locker,' signaling it is not for purchasing or creation tasks. However, it does not explicitly name alternative tools or state 'use this when you need an aggregate status view,' so it lacks full when/when-not guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• genie_catalogA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint, idempotentHint, and destructiveHint=false, so safety is clear. The description adds valuable context about 'lifecycle-verified' and 'currently eligible for purchase,' implying the catalog is dynamic and curated. It also clarifies the commercial inventory scope beyond what annotations convey.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no redundancy, front-loaded with the key purpose. Every word contributes: 'Read only' reinforces safety, 'lifecycle-verified' and 'eligible for purchase' define scope, and the exclusion sentence removes ambiguity without extra length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a simple parameterless list tool. The description fully explains what is and isn't included, and annotations cover safety and volatility. The sibling tools for recipes and pricing complete the context, so no important behavior is left undisclosed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so the schema is trivially fully covered. The description doesn't need to explain parameters. It provides semantic context about the output (eligible SKUs) which is sufficient given the tool's simplicity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists 'lifecycle-verified SKUs that are currently eligible for purchase' with a specific verb (read/list) and resource (commercial GenieLocker inference SKUs). It distinguishes from siblings by explicitly excluding designed/benchmarked recipes, which aligns with the genie_recipes sibling.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool: when you need commercial, purchasable SKUs, not recipes. It explicitly states recipes are not represented, providing a clear exclusion. It doesn't name alternative tools directly, but the context is sufficient for an agent to differentiate from siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• genie_credit_pricingA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true. The description adds specific behavioral context by stating it never creates a wallet or buys credits, and notes the data is 'current,' which aligns with openWorldHint. This goes slightly beyond the annotations without contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose and followed by a concise safety clarification. Every sentence earns its place with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only tool with no parameters or output schema, the description is fully complete. It enumerates exactly what data is returned (credit definition, burn multipliers, tier pricing, examples) and is backed by comprehensive safety annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and schema coverage is 100%. Per the rubric, the baseline for 0 params is 4. The description adds no parameter-level detail (not needed), but the baseline applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads credit definition, burn multipliers, tier pricing, and examples. It uses a specific verb ('Read') and resource ('GenieLocker prepaid credit pricing'), and distinguishes itself from siblings by emphasizing it never creates wallets or buys credits.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool (to read current credit pricing) and an explicit when-not ('never creates a wallet or buys credits'). However, it does not name alternative sibling tools, so it lacks the explicit alternative naming that would earn a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.