Server Quality Checklist

Tool Scores

• agent_propose_rebalanceB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate a non-destructive write operation. Description adds little beyond stating 'proposes' - no mention of side effects, storage of proposals, or behavior based on confidence. With annotations present, the description offers minimal additional behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One sentence of 10 words, no fluff. Conveys the core purpose efficiently. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 3 parameters, no output schema, and minimal annotations, the description is adequate but not comprehensive. Lacks details on return value, required parameters, or what happens after the proposal. However, given the simplicity, it meets a minimum viable level.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema descriptions cover all 3 parameters with basic info (e.g., 'confidence'). Description adds no new parameter semantics beyond mentioning confidence and rationale in the purpose. Baseline of 3 is appropriate given full schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the action (propose rebalance), actor (robo-advisor), and key outputs (confidence score, rationale). It distinguishes from sibling tools like agent_execute_preapproved_order which executes orders.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool vs alternatives. Does not mention prerequisites or exclusion criteria. The description implies it's for proposing rebalances, but fails to cue the agent about when to propose versus execute.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• agent_fetch_market_signalA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations show readOnlyHint=false (write) and destructiveHint=false. The description adds 'records', confirming a write operation, but does not disclose side effects, idempotency, or other behavioral traits. With annotations already covering basic behavior, the description adds modest value.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, no wasted words. Front-loaded with the key action and resource.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple record-creation tool with 4 parameters and no output schema, the description is adequate but lacks usage guidelines and deeper behavioral context. It does not clarify whether the operation is idempotent or what happens on duplicate assetId+kind.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 100% coverage with minimal descriptions. The description adds examples for 'kind' (price/volume/news), providing a bit of context beyond the schema, but does not explain constraints or formats for 'value' or 'source'. Baseline 3 is appropriate due to high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'records' and resource 'market signal', with examples (price/volume/news). It clearly distinguishes from sibling tools like 'agent_execute_preapproved_order' or 'agent_flag_anomaly'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives, nor any prerequisites or context. The description merely states the action without contextual cues.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• agent_generate_reportA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate readOnlyHint=false and destructiveHint=false. The description adds that the report is a 'recommendation' but does not disclose whether it creates a persistent object, requires permissions, or has side effects beyond generation. Minimal behavioral context beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence of 10 words, highly concise. It front-loads the core action and resource with no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with 2 params and no output schema. The description does not specify the output format or behavior (e.g., synchronous, return type). While sufficient for basic understanding, it lacks details on what the agent receives after generation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema coverage is 100% with descriptions for portfolioId and reportType. The description adds no additional meaning beyond the schema. Baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool generates a portfolio-level performance/risk report as a recommendation. The verb 'generates' and resource 'report' are specific. It distinguishes from siblings like agent_execute_preapproved_order (execution) and agent_propose_rebalance (rebalancing proposal).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for generating reports but provides no explicit guidance on when to use this tool versus alternatives. No exclusions or conditions are mentioned, leaving the agent to infer context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• agent_flag_anomalyA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations mark readOnlyHint false and destructiveHint false; description adds that it raises an alert. No extra behavioral context beyond what annotations and name imply.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence with no redundant words. Front-loaded and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Tool is simple with two string params and no output schema. Description adequately covers purpose and behavior; minor gap on expected severity values but acceptable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and description adds no extra meaning beyond parameter names. Baseline score is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description states specific verb 'raises' and resource 'portfolio-level alert' with severity. Distinguishes from siblings which involve orders, signals, reports, rebalancing, risk scores.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use or when-not-to-use guidance. Implied by purpose but not stated; sibling tools are distinct but no alternatives mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• agent_execute_preapproved_orderA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations only indicate readOnlyHint=false and destructiveHint=false. Description adds that it executes a market order, is subject to preapproval limits, and may fail on invariant checks. This adds significant behavioral context beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences plus a bullet point, no waste. Front-loaded with the main action, efficiently conveying constraints and failure conditions.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers the core action, preapproval guard, and domain invariants. No output schema, but description doesn't mention return value; however, the context is sufficient for an agent to understand constraints. Minor gap: what does the tool return upon success/failure?

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but parameter descriptions are minimal (e.g., 'α' unexplained). The description does not add further meaning to parameters beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states 'executes a market order' with specific verb and resource. Distinguishes from sibling tools like agent_fetch_market_signal or agent_propose_rebalance by its action of executing orders.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Describes preapproval guard conditions and domain invariants, providing context for when the tool may fail. However, no explicit guidance on when to use vs. alternatives or prerequisites like 'ensure preapproval first'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• agent_recompute_risk_scoreA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations show readOnlyHint=false and destructiveHint=false; the description adds minimal extra context. It doesn't disclose side effects like database updates or performance implications, but it aligns with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, front-loaded with key action, no wasted words. Efficient and to the point.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one optional parameter and no output schema, the description provides essential information: what it does and the scope behavior. No gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers portfolioId with a description, but the description adds that omitting it recomputes for all portfolios—valuable context beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('recomputes the risk profile') and the resource ('portfolio'), and distinguishes from siblings like agent_execute_preapproved_order or agent_propose_rebalance.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for recomputing risk scores but lacks explicit guidance on when to use vs alternatives or when to omit portfolioId. The mention of omitting portfolioId is a helpful scope detail.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.