Server Quality Checklist

Tool Scores

• cancel_orderB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description implies a destructive action but provides no additional behavioral context such as whether cancellation is reversible, any confirmation steps, or side effects. With no annotations, the description carries full burden and is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise with a single sentence and no extraneous information. However, additional details could be included without sacrificing brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, no annotations, and a mutation tool, the description is too sparse. It fails to explain return values, error conditions, or expected behavior after cancellation, leaving significant gaps for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so parameters are fully described in the schema. The description adds no extra meaning beyond what the schema already provides, meeting the baseline for this metric.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (cancel), the resource (a single working order), and the identifier method (by id). It effectively distinguishes from sibling tools like cancel_all_orders and replace_order.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives such as cancel_all_orders or replace_order. There is no mention of prerequisites (e.g., order must be in 'working' status) or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_net_liq_historyB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, and the description does not disclose behavioral traits such as data availability limits, potential latency, or what happens for invalid parameters. The bare description is insufficient for a tool with no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single, clear sentence with no redundancy. Could be slightly more structured, but highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 2-parameter tool, the description is adequate but lacks explanation of net liquidating value and how it relates to sibling tools. No output schema, so return format is left implicit.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with clear parameter descriptions. The tool description adds no additional meaning beyond the schema, but per guidelines, baseline is 3 given high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (Get historical net liquidating value), the resource (account), and the scope (over a time period). It distinguishes from sibling tools as no other tool retrieves net liquidating value history.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives like get_balances or get_positions. No mention of prerequisites or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_transactionsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description alone must disclose behavioral traits. It does not mention pagination, date range handling, rate limits, or whether the operation is read-only (though implied by 'Get'). The description adds minimal behavioral context beyond the name.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, front-loaded with the key action and resource. Every word is necessary, and there is no redundancy or unnecessary information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the 7 parameters and no output schema, the description is insufficient. It does not explain pagination, default date ranges, ordering, or what the return data looks like. Agents need more context to use this tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds limited extra meaning by specifying 'trades, money movements, fees', but does not elaborate on parameter syntax or usage beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the resource 'transaction history', specifying the scope as 'trades, money movements, fees'. This distinguishes it from sibling tools like get_accounts or get_positions, which retrieve different data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives, nor are there any exclusions or conditions stated. The description simply states what it does, leaving the agent without context for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_positionsB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully convey behavioral traits. It only states it lists open positions (a read operation) but lacks details on safety, rate limits, or error handling. The description adds minimal value beyond the obvious.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no wasted words. It efficiently conveys the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema and no annotations, the description is minimal. It does not explain return format, pagination, error conditions, or authentication requirements, leaving gaps for an agent needing full context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description mentions optional filtering by symbol or underlying, which aligns with the schema but does not add extra meaning beyond the parameter descriptions themselves.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'list', the resource 'open positions', and the scope 'for an account', with optional filters. It is specific and distinguishes itself from sibling tools like get_orders or get_quotes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage (listing positions) but does not explicitly state when to use this tool versus alternatives, nor does it provide exclusions or prerequisite conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_accountsA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It only states 'list all accounts for the authenticated customer,' implying a read operation, but does not disclose authentication requirements, rate limits, or potential errors. More behavioral context would be beneficial.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no unnecessary words. It is appropriately front-loaded and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a parameterless tool that returns a list of accounts, the description provides sufficient information despite lacking output schema and return format details. It is complete enough for an AI agent to understand and invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so schema coverage is complete. The description does not need to add parameter details. Baseline for 0 parameters is 4, and the description aligns with that.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List' and the resource 'tastytrade accounts', making the purpose explicit. It distinguishes from sibling tools like get_balances and get_positions by focusing specifically on accounts.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives such as get_balances or get_positions. The description lacks context on prerequisites or typical use cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_balancesA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description carries the full burden. It indicates a read operation ('get current balances') but does not disclose potential side effects, rate limits, permissions, or data caching. Listing example outputs adds some transparency but incomplete.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, front-loaded with the primary purpose. No redundant words, efficient and clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with no output schema, the description covers what it returns (various balances). However, the 'etc.' is vague, and there is no mention of the return format or structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema coverage and only one parameter, the description adds no new meaning beyond 'for an account'. The schema already documents account_number format. Baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and specifies the resource 'current balances for an account' with examples like cash, net liquidating value, buying power, margin usage, etc. This distinguishes it from sibling tools like get_accounts or get_positions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives. It implies use for an account but does not provide context on when to prefer get_balances over get_accounts or other similar tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_orderA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description bears full burden. It states 'Get the current state', which implies a read-only operation with no side effects. However, it lacks explicit statements about safety, authorization needs, or rate limits. It adds minimal context beyond the obvious.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, no fluff. It is concise and front-loaded with the key information (verb, resource, identifier).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and low complexity, the description should hint at return format or error handling. It simply says 'current state', which is vague. The agent lacks information about what the response will contain or what happens if the order is not found.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% (both parameters have descriptions). The tool description adds no additional meaning beyond what the schema already provides. Baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (Get), resource (current state of a single order), and identification method (by id). It effectively distinguishes from siblings like list_orders (list multiple) and cancel_order (different action).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool should be used when you need the state of a specific order, but it does not explicitly state when to use it versus alternatives or provide any exclusions. No mention of prerequisites or context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_quotesA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses the data source (dxLink streamer) and return types, but does not explicitly state that the operation is read-only, nor does it discuss rate limits, error handling, or behavior with invalid symbols.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with colon-separated detail, effectively front-loading the main purpose. It is concise and to the point, though a slight restructuring could improve readability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description should clarify the return shape. It lists data types but does not mention that the response is likely an array or map keyed by symbol. It covers symbol constraints and timeout, but omits error handling and response structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds substantial value by detailing the allowed symbol formats (equities, options, futures, etc.) and the data fields returned. For timeout_seconds, the schema already explains default and range, so the description does not need to repeat it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get a live market data snapshot') and lists the specific data types (bid/ask, last trade, OHLC, Greeks). It also enumerates accepted symbol formats, distinguishing it from siblings like get_option_chain or get_market_metrics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for real-time snapshots but provides no explicit guidance on when to use this tool versus alternatives, nor does it state when not to use it. No mention of prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_ordersA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must provide behavioral context. It only states the action without mentioning read-only nature, rate limits, pagination behavior, or error handling, leaving significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the main action followed by a useful tip. No fluff or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the rich input schema (100% parameter coverage) and no output schema, the description is adequate for basic use but lacks details on default behavior, sorting, or edge cases that an agent might need.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the description adds minimal extra value. The tip about live_only is marginally helpful but largely duplicates the schema description. No additional context for other parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists orders for an account, with a specific verb and resource. It distinguishes from siblings like get_order (singular) by implying plural listing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Only one hint is provided: 'Use live_only for today's working orders.' No explicit guidance on when to use this tool vs alternatives like get_order, nor any when-not or exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_margin_requirementsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description only indicates a read operation ('Get'), which is standard. It does not disclose any additional behavioral details such as required permissions, rate limits, or potential side effects, though none are expected for a simple query.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence of 9 words, efficiently conveying the tool's purpose and scope without any redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While the tool is simple with one parameter and no output schema, the description lacks any detail about the return format (e.g., structure of the report) or additional context like whether it reflects real-time data or historical. A brief mention of output would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for the single parameter 'account_number', described as 'The account number'. The tool description adds no extra meaning beyond what the schema already provides, so baseline score applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the resource 'margin/buying-power requirements report for an account', distinguishing it from sibling tools like get_accounts or get_balances which cover different financial data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for obtaining margin requirements, but does not provide explicit guidance on when to use it over alternatives (e.g., get_balances for cash balances), nor does it state when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_market_metricsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description partially discloses behavioral traits by listing the returned metrics, but fails to mention the read-only nature, authentication needs, rate limits, or behavior on invalid symbols.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that efficiently lists all key features without waste, front-loading the purpose and key data points.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one required parameter, no output schema), the description is fairly complete, though it could mention behavior for invalid symbols or empty results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description does not add additional meaning beyond the schema for the 'symbols' parameter; it only lists output fields, which is not parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Get') and resource ('market metrics'), and lists the exact data points (IV rank, IV percentile, etc.), clearly distinguishing it from sibling tools like get_quotes (price quotes) or get_option_chain (options data).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for volatility analysis by listing relevant metrics, but does not explicitly state when to use this tool versus alternatives, nor does it provide context on prerequisites or excluded scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_instrumentA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description carries the full burden. It mentions returned data types and symbol formats, but does not disclose read-only nature, error conditions, or rate limits. This is adequate for a simple read operation but lacks extra context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loading the core purpose and then adding key format guidance. Every sentence adds value with no repetition or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema exists, and the description only hints at return values with a parenthetical list. It does not explain the structure or format of the full response, which is a gap for a tool that returns a 'definition.'

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions for both parameters. The description adds significant meaning by providing concrete symbol format examples (OCC format, prefixes for futures and crypto), which goes beyond the schema's generic descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves 'full instrument definition' with specific examples (tick size, streamer symbol, etc.) and distinguishes symbol formats. It is distinct from sibling tools like get_option_chain and search_symbols.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when needing instrument details for a symbol, but does not explicitly state when to use this tool vs. alternatives like search_symbols or get_option_chain. No exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_symbolsA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It states 'Search' but doesn't disclose whether it's read-only, auth requirements, or result limitations. Adequate for a simple search but leaves gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence front-loaded with verb and resource, plus an example. No wasted words; efficient and clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given simplicity (1 param, no output schema), description covers purpose and gives example. Lacks details on case sensitivity, exact vs partial, max results, but sufficient for basic use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with one parameter. The description adds value with an example ('AAP' matches AAPL), clarifying partial matching behavior beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches for tradeable symbols by ticker or partial ticker. It distinguishes itself from sibling tools which deal with orders, accounts, and other functions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use vs alternatives, but sibling tools are in different domains so context implies usage. Lacks when-not-to-use or alternative references.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cancel_all_ordersA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It explains that the tool 'fetches live orders and cancels every cancellable one', revealing the internal process and the limitation that only cancellable orders are affected. It could also mention that this is a destructive (write) operation, but the name and emergency tone imply that.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exceptionally concise at two sentences, with the first sentence front-loading the critical 'Emergency' context and the core purpose. Every word earns its place, with no redundancy or unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description covers the essential aspects: purpose, optional filter, and behavior (fetches live orders, cancels cancellable ones). It is sufficient for an agent to understand the tool's scope and side effects, though it could mention rate limits or authorization requirements.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Both parameters are fully described in the input schema (100% schema description coverage), so the baseline is 3. The description adds minimal extra value by restating the optional filter, but does not provide additional details like format or constraints beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool cancels ALL working orders for an account, with an optional symbol filter. It uses specific verb and resource ('cancel all orders', 'account', 'underlying symbol') and distinctly differentiates from the sibling tool cancel_order, which operates on a single order.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description begins with 'Emergency:' to indicate urgent bulk use, and explains the optional filtering by 'underlying_symbol'. While it doesn't explicitly say when not to use it, the sibling tool list strongly implies that single-order cancellation should use cancel_order. The guidance is clear and contextually adequate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• health_checkA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It transparently states it verifies credentials and reports environment, implying a read-only, safe operation. Could add that it has no side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, no wasted words. Every word contributes to clarity. Front-loaded with action verb.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters, no output schema, and simple purpose, the description is nearly complete. Could briefly note that it is safe to call repeatedly, but not necessary.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist; schema coverage is 100%. Description adds no parameter info, which is acceptable since there are none. Baseline of 4 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it verifies tastytrade API credentials and reports environment. Verb 'Verify' and specific nouns 'credentials' and 'environment' make purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use versus sibling tools. While health check is distinct from trading operations, the description does not advise calling it before other operations or mention alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_option_chainA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that unfiltered calls return only expirations, and adding filters expands the data. It also notes that returned symbols are usable as order legs and quotes, which is useful context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences front-load the purpose, then explain filter behavior and practical use of results. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 7 parameters and no annotations or output schema, the description covers the essential behavioral flow and practical utility. It lacks details on output format or error handling, but is reasonably complete for a fetch operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents each parameter. The description adds value by explaining how parameters interact (e.g., expiration or DTE triggers strikes) and the incremental effect of strike filters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it retrieves option chains for a symbol, with specific behavior under different filter conditions. It distinguishes from sibling tools like get_quotes or search_symbols by focusing on options.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use each filter: no filters returns expirations, expiration/DTE yields strikes, and strike filters narrow further. While it lacks explicit 'when not to use' or alternatives, it provides clear contextual guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• replace_orderA
  4.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description must fully disclose behavior. It states 'Only the provided fields change' and highlights the leg restriction, but does not mention prerequisites (e.g., order must be working), permissions, or error cases. Partial transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tightly written sentences. First sentence states purpose and scope. Second sentence provides a critical constraint. No unnecessary words, well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 7 parameters, 3 enums, and no output schema, the description covers the main functional constraints. It lacks details on order state requirements but is otherwise adequate for a replace-order operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with individual parameter descriptions. The description adds overarching context: it modifies price, type, or time in force, and that legs are not modifiable. This adds value beyond the schema's per-parameter details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool modifies a working order's price, type, or time in force, and distinguishes from siblings like create_order (creation) and cancel_order (cancellation). It also explicitly notes that legs cannot be modified, adding precision.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use: modifying a working order's price, type, or time in force. Also gives clear when-not-to-use: legs cannot be modified, advising to cancel and re-create instead. This effectively guides the agent away from incorrect usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_orderA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden. It discloses the dry-run default, the leg count constraint, and that the preview returns buying-power effect, fees, and warnings. However, it does not describe the return value for a live order or address authentication/rate limits, which slightly limits transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences efficiently convey purpose, default behavior, and usage advice. No superfluous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 11 parameters and no output schema, the description covers the core workflow (dry-run vs. live) and leg constraints. It mentions the preview output, but lacks detail on the live order response or error handling. Still, it provides sufficient context for an agent to use the tool safely.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value by explaining the dry_run parameter's role and the types of legs. It also notes OCC format for equity options, which goes beyond the schema. This extra context justifies a score above baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description explicitly states the tool's function: 'Create an order with 1-4 legs', covering multiple instrument types. This clearly distinguishes it from sibling tools like cancel_order or list_orders.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description provides clear guidance: default behavior is a dry run, and users must set dry_run=false to actually submit. It instructs to always dry-run first and confirm with the user, which is excellent usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.