Server Quality Checklist

Tool Scores

• get_accountA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint=true, destructiveHint=false, idempotentHint=true, openWorldHint=true. The description adds 'does not sign anything that mutates state' and 'Requires SIGNER_API_TOKEN', which provide extra behavioral context beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with purpose, then usage context. No fluff, every sentence earns its place. Under 30 words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the single parameter and rich annotations, the description covers purpose, usage context, and auth. Missing details about return format could be improved, but overall sufficient for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with a clear description for the 'venue' parameter (enum, must match list_venues). The tool description does not add additional meaning beyond what the schema already provides, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states 'Return equity, free margin, and open positions for a venue.' The verb 'return' and resource 'account snapshot' are specific. It distinguishes from siblings like place_order or list_venues.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use BEFORE place_order to confirm the account has margin.' Provides clear context and a specific use case. Also mentions read-only and auth requirement.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_venuesA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, destructiveHint=false, etc. The description adds context: 'Read-only static manifest — does NOT need the Signer gateway to be reachable.' This provides useful behavioral insight beyond annotations without contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no fluff, front-loaded with the core purpose and key differentiators. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters, no output schema, and rich annotations, the description adequately covers the tool's purpose, behavior, and usage context. It mentions the returned fields, which is sufficient even without a structured output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters (100% schema coverage by default), so baseline is 4. Description adds no parameter details, which is appropriate as none exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists venues the Signer can sign trades for, specifies the output fields (venue id, asset class, auth scheme), and distinguishes it from siblings by noting it's a read-only static manifest that doesn't need the Signer gateway to be reachable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises to 'Call this first to discover what's signable' and notes the tool doesn't require the Signer gateway, implying safe usage even if gateway is down. However, it doesn't explicitly contrast with sibling tools or state when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• place_orderA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate destructiveHint=true. The description adds valuable context: the Signer enclave signs the payload with a never-exported key, policy enforcement, and the side effect of real or testnet trade. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three concise sentences, front-loaded with the primary purpose, and no filler. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 7 parameters, no output schema, and annotations present, the description covers purpose, key behaviors, and parameter nuances. Could mention rate limits or error handling, but overall complete enough for a trading tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline 3. The description adds meaning beyond the schema, such as examples for symbols, side definitions (long/short), quantity base asset, and policy_id override guidance.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Place') and resource ('a single order on the named venue'), clearly distinguishing it from siblings like cancel_order, list_venues, get_account, and get_attestation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when the tool can be used (single order) and mentions policy enforcement and side effects (real/testnet trade). It does not explicitly state when not to use it, but the context is clear enough for an AI agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cancel_orderA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses behavioral traits beyond annotations: it explains the return value (cancellation receipt), confirms idempotency behavior with ok=false and venue reason, and notes the enclave signing. These details add value over the idempotentHint annotation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences long, each serving a distinct purpose: stating the action, providing context, and explaining idempotency. No wasted words, and key information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description explains return value (cancellation receipt) and idempotent behavior, which is appropriate for a tool without an output schema. However, it does not detail the receipt's structure, which might be needed for full completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the baseline is 3, but the description significantly enriches parameter understanding: it links venue to list_venues, order_id to place_order, and clarifies symbol requirement per venue. This goes well beyond the schema's descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool cancels an outstanding order by its venue order_id, which is a specific verb+resource combination. It distinguishes itself from sibling tools like place_order and get_account.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides usage context by mentioning 'Signed inside the enclave just like place_order' and describes idempotent behavior including handling of non-existent orders. However, it does not explicitly state when not to use this tool or name alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_attestationA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds value beyond annotations by explaining that the signing key never leaves attested code, and that the attestation proves code matches published source. It does not contradict any annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: first sentence states what is returned, second sentence explains significance and provides verification instructions. Every sentence is meaningful and there is no waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description clearly states the contents (PCR0, PCR1, PCR2 measurements + signature), explains the trust chain, and provides a verification URL. For a parameterless tool, this is complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With zero parameters and 100% schema coverage, the baseline is 4. The description adds no parameter info but is not needed. It effectively describes what the returned document contains.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool returns the AWS Nitro attestation document with specific PCR measurements and signature. It explains the purpose in the context of proving code integrity for signing. This distinguishes it from sibling tools like cancel_order or place_order.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises to use this tool before trusting place_order and provides a verification step using a URL. While it gives good context for when to use, it doesn't explicitly list when not to use or mention alternatives, though none seem needed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.