Server Quality Checklist

Tool Scores

• list_setupsC
  2.7/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It does not state whether the call is read-only, whether it has side effects, what response shape to expect, or any other runtime behavior. The description only characterizes content, not behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single short sentence with no filler words. It front-loads the core concept of the setup taxonomy, though it would be clearer if it began with an action verb.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a parameterless listing tool with no output schema, the description conveys the main content of the result. However, it lacks any usage context, behavioral guarantees, or relationship to sibling tools, leaving the agent to infer when and why to call it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. The description adds useful context about what the returned data represents, which is more than the empty input schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description identifies the resource (the setup taxonomy and thesis-word mappings) but uses a noun phrase rather than an action verb like 'lists' or 'returns.' It is not a tautology and gives more specific content than the name alone, but the agent must infer the action from 'list_setups'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance about when to use this tool instead of base_rates, cost_estimate, audit_log, or second_opinion. The mention of 'Second Opinion' hints at a relationship to a sibling tool, but no explicit condition, alternative, or exclusion is stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• base_ratesB
  3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations present, the description must carry the burden of behavioral disclosure. It does reveal meaningful behavioral context: it returns historical forward-return statistics and identifies active setups based on the latest completed daily bar. However, it does not explicitly state that this is a read-only operation, describe how statistics are computed, or clarify timezone/session details. The description is transparent about output scope but not exhaustive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, tight sentence with no filler. It front-loads the main purpose (historical forward-return statistics) and appends the secondary output (active setups). Every phrase adds value, and it avoids repeating the tool name or schema details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is no output schema, no annotations, and 0% schema parameter coverage, the description leaves important gaps. It does not explain how to use as_of or horizon_days, does not describe the shape of the returned statistics, and provides no indication of typical use cases or limitations. An agent would need to guess several details before confidently invoking the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for the input schema's lack of explanations. It only clarifies the symbol context ('on a symbol') and indirectly references the daily bar timing. The as_of and horizon_days parameters are not explained at all, despite horizon_days having a default and as_of being a required-independent string. This leaves significant ambiguity for an agent choosing parameter values.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool's main function: providing historical forward-return statistics for setups on a symbol, plus which setups are active on the latest completed daily bar. It is specific about the resource (setups on a symbol) and the two kinds of information returned. It does not explicitly contrast with sibling tools like list_setups, but the core purpose is unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no guidance on when to use this tool versus alternatives such as list_setups or second_opinion. There are no stated conditions, exclusions, or references to sibling tools. While the description implies it is useful for forward-return statistics, it does not say when a user should prefer it over related tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• audit_logB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It states that the tool surfaces recent decisions and reports whether the hash chain verifies, which is useful behavioral context. However, it does not explicitly state that the operation is read-only or describe behavior when the chain fails verification, leaving some ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one short, front-loaded sentence with no filler. It states the core content ('Recent Second Opinion decisions') and the unique verification aspect compactly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema and no annotations, the description is the only source of return-value and safety information. It communicates the general content of the result but not its shape, ordering, or what 'chain verifies' means in the output. For a simple one-parameter tool this is mostly adequate, but there are gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter `limit` has no schema description and the description does not mention it directly. 'Recent' hints at limiting the number of entries, and the parameter name plus default value make it reasonably inferable, but the description adds almost no explicit meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the resource (hash-chained audit log of Second Opinion decisions) and the distinctive output (whether the chain verifies). It implicitly distinguishes itself from the sibling `second_opinion` by focusing on audit history rather than creating/getting a decision, though it lacks an explicit verb like 'returns' or 'lists'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no explicit guidance on when to use this tool versus alternatives such as `second_opinion` or `list_setups`. The context can be inferred from the tool name and sibling names, but the description itself provides no when-to-use or when-not-to-use conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cost_estimateA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It explains the calculation methodology thoroughly: taker fees and order-book impact on both legs, measured against mid from live Binance depth, with output in basis points. It does not discuss latency, error cases, or whether the estimate is indicative, but it gives a strong account of what the tool does.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence with no filler. Every clause adds useful information: output unit, input notional, fee components, impact components, and data source. It is concise without sacrificing meaningful detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a two-parameter tool with no output schema and no annotations, the description covers the central semantics well: it defines the output, the unit, the calculation inputs, and the data source. It is missing minor but useful details such as edge-case behavior, but overall an agent has enough context to invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It clarifies that 'notional' is the quantity for which the cost is estimated and implies 'symbol' selects the Binance market. However, it does not explicitly describe either parameter, their formats, or unit constraints beyond what the parameter names already suggest.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states what the tool computes: a round-trip cost in basis points for a given notional, specifying the components (taker fees and order-book impact on both legs) and the reference (live Binance depth mid). It is specific enough to distinguish the tool from generic siblings, though it does not explicitly differentiate from sibling tool names like base_rates.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'for a given notional' implies the tool should be used when the user wants a cost estimate for a specific trade size. However, there is no explicit guidance on when to use this tool versus alternatives, no exclusion criteria, and no mention of sibling tools or fallback conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• second_opinionA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral disclosure burden. It discloses that the tool is deterministic, that it analyzes the current chart and order book, and that it returns a verdict with supporting numbers. The phrase 'pre-trade check' and 'before any order tool' strongly imply no order is placed, though it could have stated the read-only nature more explicitly.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two dense sentences front-load the core purpose, then describe the computed metrics and output verdict, ending with a clear call-to-action. No redundant phrasing or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description explains what the tool computes and what it returns, which is important because there is no output schema. It covers the main workflow context and the as_of replay case is handled in the schema. It stops short of describing edge cases like low sample sizes or exact verdict thresholds, but the returned numbers are meant to inform the agent's decision.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 83%, so the schema already documents most parameters. The description adds context about how thesis is checked and that costs come from the live order book and fees, but it does not add detail that the schema lacks for individual parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific function: a deterministic pre-trade check that classifies chart setups, computes historical base rates and round-trip costs, and returns an APPROVE/CAUTION/VETO verdict. This clearly distinguishes it from the sibling tools base_rates and cost_estimate, which only provide individual pieces.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The instruction 'Call this before any order tool' gives explicit placement in the agent's workflow, and the pre-trade framing makes the intended context clear. It does not spell out when to prefer the sibling tools instead, but the integration role is evident.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.