Server Quality Checklist

Tool Scores

• calculatePortfolioValueC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool calculates total value but doesn't explain how (e.g., using real-time prices, cached data, or requiring external price lookups), what happens with invalid symbols, or if there are rate limits or authentication needs. For a calculation tool with zero annotation coverage, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It is front-loaded with the core action ('Calculate total value') and appropriately sized for its simple function, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a calculation requiring external data), lack of annotations, and no output schema, the description is incomplete. It doesn't address how the calculation is performed (e.g., integration with price data), error handling, or return format, leaving the agent with insufficient context for reliable use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with the 'holdings' parameter fully documented in the schema as an array of objects with 'symbol' and 'shares'. The description adds no additional parameter semantics beyond implying the tool uses these holdings to compute a total. Since the schema does the heavy lifting, the baseline score of 3 is appropriate, as the description doesn't enhance parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as 'Calculate total value of a portfolio given holdings', which specifies the verb ('calculate') and resource ('portfolio value'). It distinguishes from siblings like 'getStockPrice' or 'convertCurrency' by focusing on portfolio aggregation rather than individual stock data or currency conversion. However, it doesn't explicitly differentiate from potential portfolio-related tools not listed, keeping it at 4 instead of 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, such as needing current stock prices from other tools, or compare to siblings like 'getStockPrice' for individual values. Without any usage context or exclusions, the agent must infer based on tool names alone, which is insufficient for optimal selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• convertCurrencyC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions using 'latest exchange rate', which hints at real-time or recent data, but doesn't disclose critical behavioral traits such as rate limits, error handling (e.g., invalid currency codes), data sources, or whether the conversion is approximate or exact. This is a significant gap for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's function without any wasted words. It is appropriately sized and front-loaded, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of currency conversion (which involves external data sources and potential errors) and the lack of annotations and output schema, the description is incomplete. It doesn't cover return values, error cases, or behavioral constraints, leaving the agent with insufficient information to use the tool effectively in a real-world context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with clear descriptions for all three parameters (amount, from_currency, to_currency). The description adds minimal value beyond the schema, as it only reiterates the conversion action without providing additional context like currency code formats or examples beyond what's implied. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('convert') and resource ('amount from one currency to another'), and mentions the use of 'latest exchange rate'. However, it doesn't explicitly differentiate from sibling tools like 'calculatePortfolioValue' which might also involve currency conversion, though the distinction is somewhat implied by the tool name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention any prerequisites, limitations (e.g., supported currencies), or comparisons with sibling tools that might handle financial data. This leaves the agent without context for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• getStockNewsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions using the Alpha Vantage API, which hints at external data sourcing, but fails to disclose critical traits like rate limits, authentication needs, error handling, or the format of returned news. This is a significant gap for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's function without any wasted words. It is appropriately sized and front-loaded, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (external API call with no annotations or output schema), the description is incomplete. It lacks details on behavioral aspects like rate limits, error cases, and the structure of returned news, which are crucial for effective tool use. The high schema coverage doesn't compensate for these missing contextual elements.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds minimal meaning beyond the input schema, which has 100% coverage and fully documents the 'symbol' parameter. It implies the parameter is used to fetch news, but doesn't provide additional context like valid symbol formats or examples beyond what's in the schema. With high schema coverage, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get') and resource ('latest news for a stock ticker symbol'), making it easy to understand what it does. However, it doesn't explicitly differentiate from sibling tools like 'getCompanyOverview' or 'getIntraday', which might also provide stock-related information, so it doesn't reach the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention any context, prerequisites, or exclusions, such as when to prefer 'getCompanyOverview' for fundamental data or 'getIntraday' for price updates, leaving the agent with no usage instructions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• getCompanyOverviewB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the API source (Alpha Vantage), which hints at external dependencies, but doesn't cover critical aspects like rate limits, authentication needs, error handling, or response format. For a tool with zero annotation coverage, this leaves significant gaps in understanding its operational behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's front-loaded with the core action and includes the API source for context. Every part of the sentence contributes meaning, making it appropriately concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no nested objects) and high schema coverage, the description is minimally adequate. However, with no annotations and no output schema, it doesn't fully compensate for missing behavioral and output details. The description covers the basic purpose but leaves gaps in usage and operational context, making it just sufficient for a simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the single parameter 'symbol' fully documented in the schema. The description adds no additional parameter details beyond what the schema provides (e.g., no examples or constraints). According to the rules, when schema coverage is high (>80%), the baseline is 3 even with no param info in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get company information') and resource ('for a given stock ticker symbol'), with the specific API source ('using Alpha Vantage API'). It distinguishes from siblings like getStockPrice or getStockNews by focusing on comprehensive company information rather than specific data points. However, it doesn't explicitly contrast with all siblings, keeping it at 4 rather than 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like getStockPrice or getStockNews. It doesn't mention prerequisites, exclusions, or comparative contexts. The agent must infer usage from the tool name and description alone, which is minimal guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• getDividendHistoryB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only states it uses Alpha Vantage API without detailing rate limits, authentication needs, data freshness, or return format. It mentions 'recent' but doesn't define timeframe, leaving behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, front-loaded with core purpose, zero waste. Efficiently conveys key information without unnecessary details, making it easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 1-parameter tool with no output schema and no annotations, the description is minimally adequate but lacks details on return values, error handling, or API constraints. It covers the basic purpose but leaves contextual gaps for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the parameter 'symbol' fully documented in the schema. The description adds no additional parameter semantics beyond implying ticker symbols like AAPL or MSFT, which is already covered, so baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get') and resource ('dividend payout history for a ticker symbol'), specifying the data source (Alpha Vantage API). It distinguishes from siblings like getStockPrice or getCompanyOverview by focusing on dividends, but doesn't explicitly contrast them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives like getCompanyOverview (which might include dividend info) or other financial data tools. The description implies usage for dividend history but lacks explicit context or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• getStockPriceB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions the API source (Alpha Vantage) which adds some context, but doesn't describe rate limits, authentication requirements, error conditions, response format, or whether this is a read-only operation. For a tool with zero annotation coverage, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that states exactly what the tool does without unnecessary words. It's appropriately sized for a simple lookup tool and front-loads the core functionality ('Get the latest stock price'). Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-parameter lookup tool with no output schema, the description provides basic functionality but lacks important context. Without annotations, it should ideally mention that this is a read-only operation, describe the return format (e.g., price as number with currency), and note any API limitations. The current description is minimally adequate but has clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the single parameter 'symbol' well-documented in the schema as 'The stock ticker symbol, e.g., AAPL, MSFT'. The description adds no additional parameter information beyond what's already in the schema. According to scoring rules, when schema coverage is high (>80%), the baseline is 3 even with no param info in description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('latest stock price'), and specifies the data source ('using Alpha Vantage API'). It distinguishes from siblings like getCompanyOverview or getDividendHistory by focusing specifically on price data rather than company information or historical dividends. However, it doesn't explicitly differentiate from getIntraday which might also provide price data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context through 'latest stock price' and 'Alpha Vantage API', suggesting this is for current price lookups. However, it provides no explicit guidance on when to use this versus alternatives like getIntraday (for intraday data) or getCompanyOverview (for broader company data). No when-not-to-use scenarios or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• getIntradayA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions the data source (Alpha Vantage API) but does not disclose critical behavioral traits such as rate limits, authentication requirements, data freshness, error handling, or response format. This is a significant gap for an API-dependent tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads key information (action, resource, granularity, target, API) with zero wasted words. Every element earns its place by clarifying the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (API integration, no output schema, no annotations), the description is incomplete. It lacks details on behavioral traits, response structure, and usage guidelines, which are essential for effective tool invocation by an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the single parameter 'symbol' fully documented in the schema. The description adds no additional parameter semantics beyond what the schema provides, such as format examples or constraints, so it meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Get'), resource ('recent intraday stock prices'), and granularity ('5-minute intervals') for a specific target ('ticker symbol') using a named API ('Alpha Vantage'). It distinguishes from siblings like 'getStockPrice' by specifying the intraday time interval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context for intraday price data but does not explicitly state when to use this tool versus alternatives like 'getStockPrice' or 'getCompanyOverview'. No exclusions or prerequisites are mentioned, leaving the agent to infer appropriate usage scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.