Server Quality Checklist

Tool Scores

• preflight_rulesB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden. It reveals that an environment variable can change each threshold and that this tool exposes those thresholds, which is useful. However, it does not explicitly state whether the operation is read-only, what the output contains, or how the environment variable affects the response.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The first sentence is informative and front-loaded. The second sentence is rhetorical and metaphorical, adding emphasis but no concrete operational detail; it does not fully 'earn its place' in an AI-facing description.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with no parameters, but there is no output schema and no annotations. The description states that thresholds and their environment-variable dependencies are shown, yet it omits the actual environment variable name and the return shape, leaving minor but real gaps for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is no schema burden to compensate for. The mention of an environment variable that changes thresholds is the only semantic detail relevant to invocation, and it is adequate for a parameterless tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The title 'Show the active rules' provides a clear verb and resource, and the description adds that these are the 'thresholds every decision is measured against.' It is distinguishable from siblings like preflight_trade by focusing on rules/thresholds rather than execution, though it does not explicitly name an alternative.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the rules matter for decisions ('every decision is measured against'), but it gives no explicit guidance on when to call this tool versus preflight_trade or market_snapshot. There are no stated exclusions, prerequisites, or recommended call contexts.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• market_snapshotA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full behavioral burden and does well: it discloses that field ages are stated, unreadable fields return null and are listed in 'missing', and nulls are never replaced with zeros. It does not explicitly say the operation is read-only or mention authentication or rate limits, but 'snapshot' and the absence of mutation signals make the safety profile reasonably clear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no filler. The first sentence front-loads the purpose and scope, the second adds a crucial data-quality caveat about null handling. Every clause earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter snapshot tool with no output schema, the description covers the important output behaviors: what market facts are included, that ages are stated, and how missing fields are represented. It does not describe exact response formatting, but the core information an agent needs to select and interpret the call is present.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description does not add parameter-level meaning beyond the schema, but the schema already fully documents symbol and depthLimit with constraints and examples, so the agent is not left guessing.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as a market snapshot and enumerates specific contents: top of book, spread, 24h range, book depth, and lot/notional filters. It lacks a single strong verb like 'retrieves' or 'gets', but 'The market facts a trading decision rests on' combined with the tool name makes the purpose clear and implicitly distinguishes it from the preflight siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrasing 'a trading decision rests on' implies this is the tool to consult for current market conditions before trading, but it does not explicitly state when to prefer it over preflight_trade or preflight_rules, nor does it mention any exclusions. Usage context is present but not directly actionable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• preflight_tradeA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and does well: it discloses the outcome values, lists the supporting data returned, and explicitly defines the meaning of UNREADABLE to prevent dangerous misinterpretation. It does not detail side effects or rate limits, but this is a preflight check and the description's framing makes non-execution clear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences with no filler: the core purpose is front-loaded, the return semantics are listed compactly, and the critical UNREADABLE warning and invocation rule each earn their place. Nothing extraneous remains.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema and no annotations, the description supplies the essential return values, their meaning, and the required call context. It could name the sibling tools to help distinguish from market_snapshot and preflight_rules, but the description is otherwise sufficient for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 80%, so the schema already documents most parameters. The description adds value by indicating which data feeds the decision (spread, slippage, quote age, position size, venue filters) but does not meaningfully elaborate on individual parameter semantics beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb ('Decide whether an order should be placed') and a specific resource (a trade before execution), and clearly differentiates from siblings by focusing on pre-execution decision rather than snapshotting or rules. The three return statuses make the tool's purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs 'Call this before every order placed by an execution tool', giving clear when-to-use guidance. It also explains how to interpret UNREADABLE as refusal. It does not explicitly name the sibling tools as alternatives, so it misses a small opportunity to state when NOT to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.