Server Quality Checklist

Tool Scores

• sector_correlationA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It transparently explains that holdings without a sector label are ignored, weights default to equal weighting and are normalized, and correlations can legitimately return null when a sector's series has zero variance. It also clarifies that the portfolio is user-supplied. This covers key behavioral edge cases, though it does not discuss error conditions or data requirements beyond this.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured: a direct first sentence states the core action, followed by a use-case paragraph, then important caveats. Each sentence adds value, and the text is free of fluff. It front-loads the essential purpose while keeping technical details succinctly organized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the main computation, user-supplied portfolio behavior, and a critical edge case (null correlations), and an output schema exists to specify return values. However, it omits the lookback_days parameter, which is essential for understanding the time horizon of the correlation. This omission makes the description incomplete for full autonomous invocation, despite the otherwise rich context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful context for holdings and weights (e.g., 'holdings without one are ignored', 'default to equal weighting'), supplementing the schema's structure. However, the lookback_days parameter is never mentioned, leaving its role and impact undocumented. Given that schema descriptions are absent for lookback_days (only a default of 365), this is a notable gap, but the description does compensate somewhat for the other parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource: 'Compute the correlation matrix between sectors of a portfolio you supply.' It clearly distinguishes itself from sibling tools (portfolio_beta, revalue_positions) by focusing on sector-level correlation and diversification analysis. The use cases ('are my sectors moving together', 'where is the concentration risk') reinforce its unique purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool ('Use this to answer how diversified a book actually is') but does not explicitly mention when not to use it or name alternative sibling tools. It sets expectations that the user supplies the portfolio and that nothing is looked up, which helps avoid misuse. However, it lacks explicit exclusions or comparisons.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• portfolio_betaA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and excels. It discloses that the tool cannot access brokerage accounts, that weights need not sum to 1, that non-US listings require an exchange code, and that returned observation counts/date ranges may be shorter than requested. It also details the null-and-note behavior for insufficient data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured in three distinct paragraphs: purpose/use-case, requirements/constraints, and return behavior/edge cases. Every sentence contributes substantive information; nothing is redundant or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity and the presence of an output schema, the description is remarkably complete. It covers input requirements, methodology, return values (portfolio beta, individual betas, observations, date range), and edge-case handling (note and null when observations are too few) without needing to restate the output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides detailed descriptions for all parameters (weights normalized, exchange needed for non-US, lookback_days default). The tool description adds brief context like 'beta is estimated from daily returns over the lookback window' but does not significantly expand beyond schema semantics. Schema coverage is high, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a clear verb+resource statement: 'Compute the beta of a portfolio you supply against a benchmark.' It also gives concrete example questions ('what is my portfolio's beta to the S&P'), making the purpose unmistakable and distinct from sibling tools like sector_correlation and revalue_positions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says when to use it ('Use this to answer how sensitive a set of holdings is to a market index') and states a critical prerequisite ('You must pass the holdings in; this tool has no access to any brokerage account'). It does not name alternatives explicitly, but it clearly frames the tool for benchmark sensitivity analysis, which differentiates it from siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• revalue_positionsA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full behavioral burden and excels: it discloses FIFO matching mechanics, symmetric open-position behavior, FX conversion at closing fill's rate, and the null-for-unmarked-symbols behavior. These details go far beyond a generic tool description.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Every sentence earns its place. Main purpose is front-loaded, then use cases, then behavioral details, then parameter explanation. The length is justified by the complexity of FIFO matching and FX conversion, with no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema (which covers return values), the description covers inputs, processing algorithm, edge cases (opposite-direction opens), and limitations (cannot fetch trades). It is fully contextualized for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema descriptions are rich, but the tool description adds semantic context beyond them, e.g., that `marks` maps symbol to current price and that missing marks yield null. It explains the effects of executions as a list but doesn't enumerate each field—still adds meaning above schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource: 'Match buys and sells FIFO and compute realised and unrealised P&L.' It clearly distinguishes itself from siblings (portfolio_beta, sector_correlation) by focusing on trade history and P&L, not portfolio statistics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance: 'Use this to turn a list of fills into a trade history' and states a clear limitation: 'this tool cannot fetch anyone's trade history.' It also explains when to pass marks for unrealized P&L, giving actionable usage criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.