Server Quality Checklist

Tool Scores

• paper_trade_closeA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description properly discloses critical behavioral traits: it is simulation-only, never sends real orders, and does not close immediately. It also mentions the 16-minute SIP safety window, adding context beyond the schema. However, it does not detail behaviors for invalid parameters or duplicate requests, which are somewhat covered by schema descriptions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the main purpose and key caveats. Every sentence adds essential information without waste, maintaining high readability and brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the existence of an output schema and sibling tools, the description covers the essential behavior and workflow. It explains the delay and estimation step but could be more explicit about how to use it in conjunction with other tools. Overall, it provides sufficient context for an agent to understand the tool's role, though not exhaustive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides descriptions for 4 out of 5 parameters, and the tool description does not add parameter-level details. The backtest parameter lacks a schema description and is also not explained in the tool description, leaving a gap. While the description's 'simulation-only' hints at backtest usage, it does not fully compensate for the missing parameter context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific verb 'record' and resource 'simulation-only market DAY exit intent', distinguishing it from opening trades (paper_trade_record) and evaluation (paper_trade_evaluate). It also clarifies it does not send live orders or close immediately, leaving no ambiguity about the tool's function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a clear workflow: use this to record an exit intent, then evaluate later after a 16-minute safety window. It does not explicitly name alternative tools or state when not to use it, but the context and sibling listings make the intended usage evident.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• paper_trade_getA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description takes on the burden of behavioral disclosure. It explicitly states that no market-data or broker call is made, indicating a read-only, non-side-effect operation. It also describes the return content in rich detail, helping agents understand what to expect, though it does not cover error cases or authentication requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the primary action and resource, then elaborates with meaningful content details. No words are wasted; every clause adds value, including the explicit 'No market-data or broker call is made' behavioral note. It is exemplary in its economy and clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read operation with one parameter and an output schema, the description is quite complete. It discloses the scope ('one'), the nature ('simulated', 'durable'), the contents, and a key behavioral guarantee (no external calls). However, it does not mention what happens if the trade_id is invalid or not found, which is a minor gap that could be inferred from the output schema. Overall, it is sufficient for an agent to invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers 100% of parameters and already provides a clear description for trade_id: 'Trade UUID returned by paper_trade_record.' The tool description adds no additional parameter-level semantics beyond this. Since schema coverage is high, the baseline of 3 applies, and the description does not enhance or clarify the parameter further.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Read'), the resource ('one durable simulated trade'), and the specific content included (intents, fills, fees, quotes, snapshots). It also distinguishes from siblings by emphasizing 'one' trade and explicitly noting that no market-data or broker call is made, which differentiates it from potential alternatives like paper_trade_list or paper_trade_record near the same domain.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool: when a single trade's full details are needed, and explicitly notes that no external data calls are made, implying it is safe for quick lookups. However, it does not explicitly mention alternatives or exclusions (e.g., 'use paper_trade_list for multiple trades'), relying on the 'one' keyword and sibling names to convey scope distinction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• paper_trade_listA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It explicitly states 'No market-data or broker call is made' which is a valuable behavioral disclosure about side effects and external dependencies. It does not specify pagination behavior or state-value meanings, but these are partially covered by the output schema and enum values. Overall, it provides useful transparency beyond the bare function.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single 16-word sentence, front-loaded with the action and resource, with no filler words. It earns every word.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has an output schema that can document return values, and its simple parameters (limit/state) are captured in the schema. The description adds 'durable' and 'no market-data or broker call' which gives important context about scope and side effects. It is sufficiently complete for a list operation with a small parameter set.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema describes limit but not state. The description adds 'optionally filtered by state', clarifying the state parameter's role. However, it does not explain the meaning of each state enum value, and the schema itself lacks descriptions for those values, leaving a known gap. With 50% schema coverage, the description partially compensates but not fully.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and identifies the resource ('durable simulated trades') and optional filter ('by state'), clearly distinguishing it from sibling tools like paper_trade_get or paper_trade_record.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about what the tool does (lists trade records) and notes that no market-data or broker call is made, implying it is safe and fast. However, it does not explicitly mention when to prefer this over siblings or provide exclusions, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• paper_trade_recordA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It clearly explains safety (no live order), durability, timing behavior (pre-open waits, after-close expires), and the submitted_at/backtest constraint. It does not mention idempotency handling or what happens on successful creation, but the output schema covers return values. The description adds valuable behavioral context beyond the schema, so it merits a high score, though it is not exhaustive, preventing a 5.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exceptionally concise, using only three sentences to convey the core purpose and critical behavioral constraints. It is front-loaded with the primary action and immediately distinguishes simulation from live execution. Every sentence adds unique value with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 10 parameters, 5 required, an output schema, and multiple conditional behaviors, the description covers the most critical operational aspects: the simulation-only nature, the DAY order semantics, regular-hours restrictions, and submitted_at handling. It does not explain idempotency details or the conditional limit_price requirements, but these are partially covered by schema descriptions. Given the complexity and existing schema support, the description is quite complete but leaves some gaps that prevent a 5.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is low (40% per context, though actual text has more). The description adds meaning to submitted_at and backtest ('Without submitted_at, the server timestamps receipt. Historical submitted_at requires backtest=true') and clarifies DAY as time_in_force. However, it does not explain key parameters like order_type, limit_price, or position_side, which have sparse schema descriptions. The description partially compensates for the schema gap but leaves several parameters under-explained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Durably record') and a precise resource ('simulation-only whole-share US-equity DAY entry intent'). It explicitly distinguishes from live/broker-paper orders and is clearly differentiated from the sibling tools (e.g., paper_trade_close for closing positions) through the term 'entry intent'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool: it is for simulation-only entry recording, and it explicitly states 'never sends a live or broker-paper order', which guides the agent against using it for live trading. It also mentions the backtest requirement for historical submitted_at. However, it does not explicitly mention alternatives such as paper_trade_close for closing positions, so it lacks a full when-not-to-use comparison against siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• paper_trade_evaluateA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full responsibility for behavioral disclosure. It richly details execution semantics: buys use ask, sells use bid, latency and size enforced, fees itemized, fill time calculation (submitted_at + latency or session open), quote_as_of significance, and the 16-minute historical delay when as_of is omitted. It also warns against assuming a fill from indeterminate outcomes, adding exceptional transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is five sentences, but each sentence carries critical, non-redundant information. The first sentence is front-loaded with the core purpose and scope, followed by execution details and usage cautions. There is no fluff or repetition; it is appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema (not shown), the description still guides the agent on what to inspect (outcome_reason, state_after, next_action, warnings) and how to interpret results. It covers default time behavior, fill timing, and safety caveats. For a tool with this complexity, the description is remarkably complete and leaves few unanswered questions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 67% (top-level), with as_of's description nested in anyOf. The description adds value by explaining how as_of affects fill evaluation and position marking, and reiterates the default behavior (now minus 16 minutes) clearly. It also ties trade_id to paper_trade_record and request_id to idempotency, though those are also in the schema. The added context on quote_as_of and execution time goes beyond the schema, so a 4 is warranted.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource: 'Evaluate a pending simulated entry/exit or mark an open position using delayed historical consolidated SIP top-of-book quotes; no order is sent.' This clearly distinguishes it from siblings (record, get, list, close) by focusing on evaluation of hypothetical fills and position marks, and explicitly notes no real order is placed.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly indicates when to use it (evaluate pending order or mark open position) and provides a caution about reading outcome fields and not inferring fills from indeterminate results. However, it does not explicitly name alternatives or state 'when not to use,' only implying the context. This is clear context without explicit exclusions, so a 4 is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.