Server Quality Checklist

Tool Scores

• get_portfolio_positionsB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, and the description does not disclose whether the tool is read-only, any potential side effects, latency, or whether data is real-time or cached. For a read operation, the description carries full burden and falls short.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that is front-loaded with the action. Every word adds value; no wasted or redundant phrasing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of an output schema and annotations, the description should provide more detail about the return format (e.g., array of objects), whether positions are current or historical, and any limitations. It is insufficient for an agent to correctly interpret the output.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With no parameters (100% schema coverage), the description does not need to add parameter information. The baseline of 3 applies, and the description adds some value by explaining what the output contains, though this is more about return semantics than parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verb 'Get' and resource 'portfolio positions', and lists concrete examples of returned data (ticker, quantity, cost, account details). It clearly distinguishes from siblings like get_portfolio_pnl (profit/loss) and get_position_details (more detailed).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool is for obtaining basic position overviews, but does not explicitly state when to use it instead of siblings or provide any exclusions. The usage context is implied by the word 'basic' and the sibling tool names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_position_detailsA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. Only states 'get detailed information' implying read-only, but lacks disclosure of side effects, auth requirements, or rate limits. Minimal behavioral insight.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, front-loaded with verb and resource, no wasted words. Efficiently conveys core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema provided, and description omits what 'detailed information' includes or return structure. Sibling differentiation is missing. Incomplete for a tool with no output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a description for 'tickers'. Description adds value with examples (["AAPL", "NVDA", "7940.T"]) and clarifies format, exceeding bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Get detailed information for specific portfolio positions by ticker symbols', with a specific verb and resource. It distinguishes from siblings like get_portfolio_positions (likely lists without details) and get_portfolio_pnl (P&L).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies usage for fetching details on specific tickers, but no explicit when-to-use or alternatives. No guidance on when not to use or when siblings are preferred.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_portfolio_pnlA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It mentions current market values but does not disclose whether data may be stale without refresh, nor any side effects like external API calls. Adequate but lacks depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence front-loads the core purpose and is concise (20 words). Could be slightly restructured for readability, but no unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one optional parameter and no output schema, the description is sufficient. It states what the tool returns, though it could hint that refresh triggers a slower operation. Generally complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers the only parameter (refresh) with a description, so baseline is 3. The description adds no additional semantic value for parameters beyond what schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the resource 'portfolio profit and loss analysis', specifying it includes summary totals and individual position performance. It effectively distinguishes from siblings like get_portfolio_positions and get_position_details.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance is given on when to use this tool versus alternatives. The description implies it is for P&L analysis, but does not mention when not to use it or provide context like refresh usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• refresh_portfolio_dataA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description says 'force refresh' implying a mutation action, but provides no details on side effects, rate limits, or what happens to existing data. With no annotations, more context about the refresh process would help.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that conveys the tool's purpose without unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description lacks information about return value (no output schema), error handling, or whether the refresh is asynchronous. For a mutation tool, these details would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist (input schema is empty and schema coverage is 100%). The description does not need to add parameter information, so a high baseline score is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool refreshes all portfolio price data from Yahoo Finance, including current and historical prices. It uses a specific verb ('refresh'), identifies the resource ('portfolio price data'), and distinguishes from sibling read-only tools like get_portfolio_pnl.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool vs alternatives. Siblings are query tools for reading data, but the description does not mention scenarios like stale data needing refresh or any prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.