Server Quality Checklist

Tool Scores

• validate_ordersC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions what is validated but does not describe the tool's behavior, such as whether it performs read-only checks, returns validation results, requires specific permissions, or has side effects like logging. For a validation tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the main purpose ('Validate a set of orders') and lists specific checks. It avoids unnecessary words, but could be slightly improved by structuring the checks more clearly (e.g., with bullet points or commas). Overall, it is appropriately concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (3 parameters, 0% schema coverage, no annotations, no output schema), the description is incomplete. It does not explain what the tool returns, how validation results are presented, or error handling. For a validation tool with multiple inputs and no structured output, more context is needed to guide the agent effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for undocumented parameters. It does not explain the meaning or usage of 'order_ids', 'symbol', or 'status' parameters, nor how they relate to the validation checks mentioned. The description adds no parameter semantics beyond the bare schema, failing to address the coverage gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Validate a set of orders' with specific checks (symbol validity, venue status, duplicate ClOrdIDs, client status). It uses a specific verb ('validate') and resource ('orders'), but does not explicitly differentiate from sibling tools like 'query_orders' or 'check_ticker', which prevents a score of 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It lists what the tool checks but does not mention prerequisites, context, or exclusions, such as whether it should be used before sending orders or as a diagnostic step. Without such usage context, the agent lacks clear direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cancel_replaceC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions the technical protocol ('via FIX 35=F or 35=G'), which adds some context about implementation, but fails to describe critical behaviors: whether this is a destructive mutation (implied by 'cancel' but not explicit), what permissions are needed, how errors are handled, or what the response looks like. For a tool that modifies financial orders, this is inadequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero wasted words. It front-loads the core purpose ('Cancel or replace an existing order') and adds technical specificity ('via FIX 35=F or 35=G') only where necessary. Every element earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (financial order modification), lack of annotations, no output schema, and 0% schema description coverage, the description is incomplete. It omits critical context: error handling, side effects, response format, and differentiation from sibling tools. The technical FIX reference is useful but insufficient for safe and effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for all 6 parameters. It only vaguely references parameters via 'cancel or replace' (mapping to 'action') and 'existing order' (mapping to 'order_id'), but provides no semantic context for 'new_venue', 'new_quantity', 'new_price', or 'new_symbol'. The description adds minimal value beyond what the bare schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Cancel or replace') and resource ('an existing order'), making the purpose unambiguous. It distinguishes from siblings like 'send_order' or 'query_orders' by focusing on modification rather than creation or querying. However, it doesn't explicitly differentiate from 'release_stuck_orders' or 'modify_algo', which could have overlapping functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing an existing order ID), exclusions (e.g., not for algo orders), or when to choose 'cancel' vs 'replace'. Given siblings like 'cancel_algo' and 'modify_algo', this lack of differentiation is a significant gap.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• load_tickerC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions two actions (loading and releasing) but doesn't specify whether this is a read-only or mutating operation, what permissions are required, whether it's idempotent, what happens on failure, or what the expected response looks like. For a tool that appears to modify system state, this is a significant gap in behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise - a single sentence that efficiently communicates the core functionality. It's front-loaded with the primary action and avoids unnecessary words. Every part of the sentence contributes to understanding the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 6 parameters (4 required), 0% schema coverage, no annotations, and no output schema, the description is inadequate. It explains what the tool does at a high level but provides insufficient context about parameters, behavior, output, or usage relative to siblings. The agent would struggle to use this tool correctly without additional information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage for 6 parameters (4 required), the description provides no information about any parameters. It doesn't explain what 'symbol', 'cusip', 'name', 'listing_exchange', 'lot_size', or 'tick_size' represent, their formats, constraints, or relationships. The description fails to compensate for the complete lack of schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('load', 'release') and resources ('symbol', 'reference store', 'orders pending that symbol'). It distinguishes from siblings like 'check_ticker' or 'update_ticker' by focusing on initial loading rather than querying or updating. However, it doesn't explicitly differentiate from 'release_stuck_orders' which might share some functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'update_ticker' or 'check_ticker'. It doesn't mention prerequisites, timing considerations, or specific scenarios where this tool is appropriate versus other ticker-related tools. The agent must infer usage from the description alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cancel_algoC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool cancels an algo and sends cancel requests for child slices, implying a destructive mutation. However, it lacks critical details: whether this requires specific permissions, if the cancellation is reversible, what happens to partially filled orders, or what the response looks like (success/failure indicators). For a mutation tool with zero annotation coverage, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action ('Cancel an active algo') and adds necessary detail about child slices. Every word earns its place, with no redundancy or fluff. It's appropriately sized for a tool with 2 parameters and clear purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (destructive mutation with cascading effects), lack of annotations, no output schema, and incomplete parameter documentation, the description is inadequate. It should explain behavioral implications (e.g., idempotency, error handling) and expected outcomes. For a tool that likely impacts trading systems, more context is needed for safe agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 50% (only 'reason' has a description). The description doesn't mention parameters at all, so it adds no semantic value beyond the schema. However, with 2 parameters (one required), the baseline is 3 since the schema partially documents them. The description doesn't compensate for the coverage gap (e.g., explaining 'algo_id' format or 'reason' constraints).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Cancel') and target ('an active algo'), and specifies the secondary effect ('send OrderCancelRequest for all open child slices'). This distinguishes it from siblings like 'modify_algo' or 'release_stuck_orders' by focusing on termination rather than modification or recovery. However, it doesn't explicitly differentiate from 'cancel_replace', which might have overlapping functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'cancel_replace' or 'modify_algo'. It mentions 'active algo' but doesn't specify prerequisites (e.g., algo must be running) or exclusions (e.g., cannot cancel completed algos). Without such context, the agent must infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_algo_statusC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but offers minimal behavioral insight. It mentions what is checked (e.g., schedule deviation) but doesn't disclose permissions needed, rate limits, response format, or whether it's read-only or has side effects. For a status-checking tool with zero annotation coverage, this is inadequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that lists key status aspects without fluff. It's front-loaded with the core purpose and uses a colon to introduce details, making it easy to parse. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (status checking with 3 parameters), lack of annotations, and no output schema, the description is incomplete. It doesn't explain what the check returns, how results are formatted, or behavioral constraints, leaving significant gaps for the agent to operate effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is low at 33% (only 'algo_id' has a description), but the description adds no parameter-specific information beyond what the schema provides. It doesn't explain the meaning of 'symbol' or 'status' enum values, nor does it clarify interactions like omitting 'algo_id' for all active orders. Baseline 3 is appropriate as the description doesn't compensate for the coverage gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'check' and resource 'algo orders', listing specific status aspects like schedule deviation and execution quality. It distinguishes from siblings like 'cancel_algo' or 'modify_algo' by focusing on status monitoring rather than modification, though it doesn't explicitly contrast with similar tools like 'query_orders'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'query_orders' or 'check_ticker'. The description implies usage for monitoring algo order health but lacks explicit context, prerequisites, or exclusions, leaving the agent to infer based on tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_tickerC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the return content but doesn't address critical aspects like whether this is a read-only operation, potential rate limits, authentication requirements, error conditions, or data freshness. For a lookup tool with zero annotation coverage, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise—just one sentence that efficiently conveys the core functionality and return values. Every word earns its place with no wasted text, making it easy to parse and understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations, 0% schema description coverage, and no output schema, the description is incomplete. It doesn't explain return formats, error handling, or operational constraints needed for reliable tool invocation in a trading context with sibling tools like 'send_order' and 'query_orders'.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description implies that either 'symbol' or 'cusip' can be used for lookup, which adds some semantic context beyond the bare schema (which has 0% description coverage). However, it doesn't specify format requirements, mutual exclusivity, or examples, leaving significant gaps in parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Look up') and resource ('a symbol or CUSIP'), and specifies what information is returned ('full record, pending corporate actions, and affected open order count'). However, it doesn't explicitly differentiate from sibling tools like 'load_ticker' or 'update_ticker', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'load_ticker' or 'update_ticker'. It doesn't mention prerequisites, exclusions, or specific contexts for usage, leaving the agent without clear decision-making criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• dump_session_stateC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool returns diagnostics but does not specify if it's read-only, has side effects, requires permissions, or details about response format (e.g., structured data, error handling). For a diagnostic tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the key action ('return full session diagnostics') and lists the returned information clearly. There is no wasted text, making it appropriately concise for its purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of session diagnostics and the lack of annotations and output schema, the description is incomplete. It mentions what information is returned but does not explain the format, potential errors, or how to interpret the diagnostics (e.g., what 'heartbeat age' indicates). For a tool that likely provides critical operational data, more context is needed to use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the 'venue' parameter fully documented in the schema. The description does not add any additional meaning or context beyond what the schema provides, such as examples or constraints. With high schema coverage, the baseline score of 3 is appropriate as the description doesn't compensate but also doesn't detract.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('return full session diagnostics') and resources ('for a venue'), including what information is returned ('sequence numbers, latency, heartbeat age, and associated orders'). However, it does not explicitly differentiate from sibling tools like 'check_fix_sessions' or 'session_heartbeat', which may have overlapping diagnostic functions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With sibling tools like 'check_fix_sessions', 'session_heartbeat', and 'update_venue_status' that might relate to session monitoring or diagnostics, there is no indication of when this specific diagnostic tool is preferred or what scenarios it addresses uniquely.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fix_session_issueC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions three actions but doesn't explain what each does operationally (e.g., whether 'reset_sequence' is destructive, if 'reconnect' requires downtime, or what side effects occur). For a tool that likely modifies session state, this lack of detail is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise—a single sentence that efficiently states the purpose and lists the three actions. There's no wasted language, and it's front-loaded with the core function. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of FIX session management, no annotations, no output schema, and 0% schema description coverage, the description is inadequate. It doesn't explain what 'resolving' entails, potential impacts, or expected outcomes. For a tool with two required parameters and likely critical operations, more context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description mentions 'resend_request (gap recovery), reset_sequence, or reconnect,' which partially explains the 'action' parameter's enum values. However, with 0% schema description coverage, it doesn't clarify the 'venue' parameter or provide deeper context for the actions. The description adds some value but doesn't fully compensate for the schema's lack of descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Resolve a FIX session issue' with three specific actions. It uses a specific verb ('Resolve') and identifies the resource ('FIX session issue'), but doesn't explicitly distinguish it from sibling tools like 'reset_sequence' or 'dump_session_state' which might handle similar concerns.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It lists three actions but doesn't explain when to choose 'resend_request' over 'reset_sequence' or 'reconnect', nor does it mention when to use this tool instead of sibling tools like 'check_fix_sessions' or 'reset_sequence' (which appears as both an action here and a separate tool).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• grep_logsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the basic operation but doesn't cover important aspects like whether it's read-only (implied but not stated), performance characteristics, error handling, or what happens with large files. For a tool with zero annotation coverage, this leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that communicates the core functionality without any wasted words. It's appropriately sized for a simple search tool and gets straight to the point.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations and no output schema, the description is insufficient. It doesn't explain what format the matching lines are returned in, whether there's pagination for large results, error conditions, or performance considerations. Given the context of sibling tools that suggest a trading/algo system environment, more operational context would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so both parameters are well-documented in the schema. The description doesn't add any meaningful parameter semantics beyond what the schema already provides, but it doesn't need to compensate for gaps either. This meets the baseline for good schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('search', 'return') and resources ('log files', 'matching lines'). It distinguishes itself from sibling tools by focusing on log file pattern matching, though it doesn't explicitly differentiate from 'tail_logs' which might also work with logs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'tail_logs' or other log-related operations. It doesn't mention prerequisites, constraints, or typical use cases beyond the basic functionality.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• modify_algoC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It implies mutation ('modify'), but doesn't specify permissions needed, side effects (e.g., if pausing affects orders), rate limits, or error conditions. This is a significant gap for a tool that modifies active systems.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the purpose and lists actions clearly. Every word earns its place with no redundancy or unnecessary details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of modifying an active algorithm, no annotations, and no output schema, the description is incomplete. It lacks details on behavioral traits, error handling, and return values, making it inadequate for safe and effective use by an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 33% (only 'new_pov_rate' has a description). The description adds minimal value by mentioning 'update POV participation rate', which loosely relates to 'new_pov_rate', but doesn't explain 'algo_id' or 'action' beyond what the enum implies. With low coverage, it partially compensates but leaves key parameters undocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'modify' and the resource 'active algo', with specific actions listed (pause, resume, update POV participation rate). It distinguishes from siblings like 'cancel_algo' or 'check_algo_status' by focusing on modification rather than cancellation or status checking, though it doesn't explicitly contrast with all siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., algo must be active), exclusions, or compare with siblings like 'cancel_algo' for stopping versus pausing. The description only lists actions without contextual usage advice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_ordersC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions returns order details but does not disclose behavioral traits such as whether this is a read-only operation, potential rate limits, authentication needs, or how results are structured (e.g., pagination). For a query tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the purpose and key details. It avoids unnecessary words, though it could be slightly more structured by separating usage guidance from purpose. Overall, it earns its place without waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (query with filters), no annotations, and no output schema, the description is adequate but incomplete. It specifies what is returned but not the format or limitations. For a query tool, more context on result structure or constraints would be beneficial, though the purpose is clear.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 5 parameters with clear descriptions. The description adds minimal value by noting 'optional filters' but does not provide additional semantics beyond what the schema specifies, such as filter combinations or default behaviors. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('query') and resource ('OMS orders'), specifying it returns order details including notional value and SLA countdowns for institutional orders. However, it does not explicitly differentiate from sibling tools like 'validate_orders' or 'release_stuck_orders', which might involve order-related operations but with different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions 'optional filters' but provides no guidance on when to use this tool versus alternatives like 'validate_orders' or 'send_order'. It lacks explicit when/when-not scenarios or prerequisites, leaving usage context implied rather than clearly defined.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reset_sequenceC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states the action is a reset, implying a mutation, but doesn't disclose behavioral traits such as permissions required, whether it's destructive to session data, side effects, or response format. This is a significant gap for a mutation tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without any wasted words. It is appropriately sized and front-loaded, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a mutation operation with no annotations and no output schema), the description is incomplete. It lacks details on behavior, side effects, or return values, which are critical for safe and effective use. The high schema coverage doesn't compensate for these gaps in a mutation context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with the parameter 'venue' fully documented in the schema. The description adds no additional meaning beyond implying the venue is for a session, which is already clear from the schema. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Reset') and the resource ('FIX sequence numbers for a venue session'), making the purpose understandable. It distinguishes from siblings like 'check_fix_sessions' or 'fix_session_issue' by focusing on resetting rather than checking or troubleshooting, though it doesn't explicitly contrast with them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., when sequence numbers need resetting), exclusions, or refer to sibling tools like 'fix_session_issue' for related issues, leaving usage context unclear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• session_heartbeatC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions 'return the heartbeat status' but doesn't disclose what that status entails (e.g., success/failure, latency metrics), whether this is a read-only or state-changing operation, potential side effects (e.g., keeping a session alive), or any rate limits or authentication needs. For a tool with no annotation coverage, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action ('Send a heartbeat') and purpose ('return the heartbeat status'). There's no wasted verbiage or redundancy, making it appropriately concise for a simple tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of trading/venue management (implied by sibling tools), no annotations, and no output schema, the description is incomplete. It doesn't explain what a 'heartbeat' means in this context, what the returned status includes, or how this tool fits into broader session management workflows. For a tool with rich contextual signals but minimal disclosure, it should do more to compensate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the single parameter 'venue' documented as 'Venue name (e.g. NYSE, BATS, ARCA)'. The description adds no additional parameter semantics beyond what the schema provides, such as clarifying the format of venue names or how they relate to sessions. With high schema coverage, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Send a heartbeat') and the target resource ('for a specific venue session'), with a specific verb+resource combination. However, it doesn't differentiate this tool from its many siblings in the trading/venue management context, such as 'check_fix_sessions' or 'update_venue_status', which might involve similar session-related operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With 21 sibling tools including session-related ones like 'check_fix_sessions' and 'dump_session_state', there's no indication of when a heartbeat is appropriate versus other session checks or updates, nor any prerequisites or exclusions mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tail_logsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions returning lines but does not cover aspects like error handling (e.g., if the file doesn't exist), performance implications, or output format details. This leaves significant gaps in understanding the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's function without unnecessary words. It is front-loaded and appropriately sized, making it easy to grasp quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It does not address behavioral traits, error cases, or return values, which are crucial for a tool that interacts with log files. This leaves the agent with insufficient context for reliable use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema already documents both parameters ('file' and 'lines') with descriptions. The description adds no additional meaning beyond implying the tool tails logs, which aligns with the schema but doesn't provide extra context like file path examples or line count constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('return') and resource ('last N lines of a log file'), making the purpose specific and understandable. However, it does not explicitly differentiate from sibling tools like 'grep_logs', which might search logs rather than tail them, leaving some ambiguity in sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives, such as 'grep_logs' for searching or other log-related tools. It lacks explicit context, prerequisites, or exclusions, offering minimal usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• send_orderB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions validation, corporate action checks, and auto-routing behavior, which adds useful context. However, it doesn't cover critical aspects like error handling, response format, rate limits, authentication requirements, or whether this is a read-only or destructive operation, leaving significant gaps for a tool that sends orders.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that packs multiple operational details (validation, corporate action checks, auto-routing) without unnecessary words. It's appropriately sized and front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex order-sending tool with 7 parameters, 0% schema coverage, no annotations, and no output schema, the description is insufficient. It lacks details about return values, error conditions, security requirements, and comprehensive parameter explanations that would be needed for reliable agent usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage for all 7 parameters, the description doesn't add meaningful semantic information about any parameters. It mentions 'symbol' and 'venue' indirectly in the validation and auto-routing context, but doesn't explain what 'client_name' represents, the meaning of 'order_type' values, or other parameter details needed for proper usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('send a new order') and the mechanism ('via FIX NewOrderSingle'), which is specific and actionable. However, it doesn't explicitly distinguish this from sibling tools like 'send_algo_order' or 'validate_orders', which would require more differentiation for a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by mentioning validation, corporate action checks, and auto-routing, but doesn't explicitly state when to use this tool versus alternatives like 'send_algo_order' or 'query_orders'. It provides some operational context but lacks clear when/when-not guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_tickerB
  3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds context beyond basic functionality by mentioning that it 'bulk-update all open orders' and 'Flags stop orders for manual review,' which are important behavioral traits. However, it lacks details on permissions, rate limits, error handling, or what happens to non-open orders, leaving gaps in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is highly concise and front-loaded, consisting of a single sentence that efficiently communicates the core actions and a key behavioral note. Every word earns its place, with no redundant or unnecessary information, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a tool that performs bulk updates and flags orders for review, with no annotations, no output schema, and low schema coverage, the description is incomplete. It lacks information on return values, error conditions, side effects beyond the mentioned flagging, and how it interacts with other tools in the sibling list, leaving significant gaps for an AI agent to understand its full context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, so the description must compensate. It does not explain the parameters 'old_symbol', 'new_symbol', or 'reason' beyond what the schema provides (e.g., the enum values for 'reason'). The description mentions renaming a symbol and updating orders, which hints at the purpose of 'old_symbol' and 'new_symbol', but adds minimal semantic value, failing to fully compensate for the low schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Rename a symbol' and 'bulk-update all open orders') and identifies the resource ('symbol' and 'open orders'). It distinguishes itself from siblings like 'check_ticker' or 'load_ticker' by focusing on updates rather than queries or loading. However, it doesn't explicitly differentiate from potential alternatives like 'modify_algo' or 'cancel_replace' in the sibling list.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It mentions that 'Flags stop orders for manual review,' which implies a specific behavior but doesn't clarify when this tool is appropriate compared to other order-related tools like 'cancel_replace' or 'modify_algo'. There are no explicit when-to-use or when-not-to-use instructions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• send_algo_orderB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool 'creates' a parent order, implying a write/mutation operation, but doesn't mention critical behavioral aspects like required permissions, whether the order is immediately active, rate limits, error handling, or what happens if parameters are invalid. The description is insufficient for a mutation tool with 10 parameters.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the key action and algorithm types. However, it could be more structured by separating the algorithm list or adding brief usage context, but it avoids unnecessary verbosity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex mutation tool with 10 parameters, no annotations, and no output schema, the description is inadequate. It doesn't explain the return value (e.g., order ID, status), error conditions, or how the order interacts with the system (e.g., execution timing, sibling tools like 'check_algo_status'). The lack of behavioral and parameter details leaves the agent poorly informed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 40% (only 4 of 10 parameters have descriptions in the schema). The description adds minimal value: it lists algorithm types (already in the enum) and mentions 'execution schedule' and 'child slices' (hinting at 'end_time' and 'slice_count' but not explaining them). It doesn't clarify the purpose of 'client_name', 'venue', or other parameters, leaving significant gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Submit a new algorithmic order') and lists the exact algorithm types (TWAP, VWAP, POV, IS, DARK_AGG, ICEBERG), distinguishing it from sibling tools like 'send_order' (likely a regular order) and 'modify_algo' (modification rather than creation). It also specifies the outcome: 'Creates a parent algo order with execution schedule and initial child slices.'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'send_order' (for non-algorithmic orders) or 'modify_algo' (for modifying existing algo orders). It mentions algorithm types but doesn't explain when to choose one over another or any prerequisites (e.g., market hours, venue availability).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_fix_sessionsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states what metrics are checked but lacks behavioral details such as whether this is a read-only operation (implied by 'check'), potential performance impact, error handling, or output format. This leaves gaps for a tool that likely interacts with live trading systems.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the purpose and lists key metrics without unnecessary words. Every element earns its place by specifying the tool's focus clearly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of FIX sessions in trading contexts and the lack of annotations or output schema, the description is incomplete. It does not address critical aspects like whether this tool is safe for live environments, what the return data looks like, or how to interpret the health metrics, leaving significant gaps for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the parameter 'venue' documented as 'Specific venue, or omit for all'. The description does not add any additional meaning beyond this, such as examples of venue values or implications of omitting it. Baseline 3 is appropriate since the schema adequately covers the single parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'check' and the resource 'FIX session health', with specific metrics listed (status, sequence numbers, heartbeat age, latency). It distinguishes from siblings like 'dump_session_state' (which likely provides raw data) or 'session_heartbeat' (which likely sends heartbeats) by focusing on health assessment.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives is provided. While the purpose implies it's for monitoring FIX session health, there's no mention of prerequisites, timing (e.g., during trading hours), or comparisons to siblings like 'check_algo_status' or 'run_premarket_check'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_premarket_checkB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions checking various components but doesn't disclose behavioral traits like whether this is a read-only diagnostic, if it triggers any actions (e.g., auto-fixing issues), what permissions are required, or how results are presented. The term 'Flagship' suggests importance but lacks operational clarity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that lists all key components of the health check without unnecessary words. It's appropriately sized for a tool with no parameters and front-loaded with the main purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (comprehensive health check) and lack of both annotations and output schema, the description is moderately complete. It outlines what areas are checked but doesn't explain the return format, severity levels, or actionable insights. For a diagnostic tool in a trading context, more detail on output expectations would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so the schema fully documents the absence of inputs. The description appropriately doesn't add parameter details beyond what the schema provides, maintaining a baseline score for tools with no parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs a 'pre-market health check' with specific components (sessions, corp actions, stuck orders, SLA deadlines, validation summary). It uses a specific verb ('Flagship') and identifies the resource (pre-market health), but doesn't explicitly distinguish it from sibling tools like 'check_fix_sessions' or 'check_ticker' that might cover overlapping areas.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With siblings like 'check_fix_sessions', 'check_ticker', 'validate_orders', and 'release_stuck_orders' that might handle specific aspects of the health check, there's no indication of when this comprehensive check is preferred over targeted tools or what prerequisites might exist.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• release_stuck_ordersB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only states the action ('release stuck orders') without disclosing behavioral traits like whether this is a safe read-only operation, if it requires specific permissions, potential side effects (e.g., order resubmission), or system impact. It mentions the mechanism ('removing venue_down flags') but doesn't explain what that entails.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It is front-loaded with the core action and avoids redundancy, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a system-level recovery tool with no annotations and no output schema, the description is insufficient. It doesn't explain what 'stuck orders' are, how 'venue_down flags' work, what happens after release (e.g., do orders resume processing?), or potential risks. For a tool that likely affects multiple venues, more context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so the baseline is 4. The description adds no parameter-specific information, which is appropriate since there are no parameters to document.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('release all stuck orders') and the mechanism ('by removing venue_down flags'), with the resource being 'orders across all venues'. It distinguishes from siblings like 'cancel_algo', 'send_order', or 'validate_orders' by focusing on system-level recovery rather than order manipulation or validation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'fix_session_issue', 'reset_sequence', or 'update_venue_status', which might address similar problems. It lacks explicit context about when this operation is appropriate or what conditions indicate 'stuck orders'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_scenariosA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It describes the two actions ('list' and 'load') and their effects, but lacks details on permissions, rate limits, side effects (e.g., if loading a scenario affects other operations), or response format. For a tool with no annotations, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded, with two sentences that directly explain the tool's functionality and usage. Every sentence earns its place by providing essential information without redundancy, making it efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description covers the basic actions and parameters but lacks completeness. It does not explain return values, error conditions, or how this tool interacts with sibling tools in the trading context. For a tool with two actions and no structured behavioral data, it is adequate but has clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with clear descriptions for both parameters in the schema. The description adds minimal value beyond the schema by reiterating the action purposes and noting that 'scenario_name' is required for 'load'. Since the schema already documents this well, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'List all available trading scenarios or load one into the runtime.' It specifies the verb ('list' or 'load') and resource ('trading scenarios'), but does not explicitly differentiate it from sibling tools like 'load_ticker' or 'update_ticker', which might handle similar resources. This makes it clear but not fully sibling-distinctive.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use each action: 'Use action='list' to see all scenarios with their context summaries. Use action='load' with scenario_name to switch the active scenario.' It specifies usage based on the action parameter, but does not mention when not to use this tool or alternatives among siblings, such as 'load_ticker' for ticker-related operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_venue_statusA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states the tool changes status but doesn't disclose behavioral traits like required permissions, whether changes are reversible, side effects on related systems, or error conditions. The mention of recovery via another tool hints at consequences but lacks detail.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with zero waste: the first states the purpose, and the second provides usage guidance. It's front-loaded and appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and 2 parameters with partial schema coverage, the description is minimally adequate. It covers purpose and usage but lacks details on behavior, parameters, and outputs, leaving gaps for a mutation tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 50% (only 'venue' has a description). The description adds no parameter-specific information beyond what's in the schema or enum. With moderate schema coverage, the baseline is 3, as the description doesn't compensate for the gap but doesn't detract either.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Change venue status') and specifies the resource ('venue') with the possible status values ('active/degraded/down'). It doesn't explicitly differentiate from sibling tools, but the specific focus on venue status makes the purpose clear and non-tautological.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly provides an alternative tool ('fix_session_issue') for recovering degraded/down venues, giving clear guidance on when to use this tool vs. another. It establishes a specific context for usage without being misleading.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.