Server Quality Checklist

Tool Scores

• market_quoteA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint, idempotentHint, openWorldHint, and destructiveHint=false. The description adds value by detailing behavior for unknown symbols: 'Returns 'no data' for unknown/delisted symbols (Finnhub returns zeros for unknowns — we treat that as no data, never fabricate).' This provides concrete error-handling context beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise: a one-sentence summary followed by essential details, an Args section, and examples. It is front-loaded with the core purpose and avoids unnecessary exposition. Every sentence adds value, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has an output schema (not shown but flagged), which likely documents return values. The description provides supplementary information: data fields returned, error handling for unknown symbols, and backing service. For a simple quote tool with good annotations and output schema, the description is complete enough, missing only explicit prerequisites (e.g., API key).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already describes both parameters (symbol and response_format) with decent detail. The description adds an 'Args' section that restates types and adds examples mapping natural language to symbols. Since schema coverage is complete, this adds marginal value, meriting a baseline score of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get a real-time stock quote: price, % change, day high/low, and previous close.' It specifies the verb 'Get', the resource 'real-time stock quote', and the fields returned. This distinguishes it from sibling tools like market_analyze or market_news, which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides examples and mentions backing by Finnhub, but does not explicitly guide when to use this tool versus alternatives like market_analyze or market_scan. There is no 'when to use' or 'when not to use' advice. The examples imply use for simple quote requests, but no exclusions or alternatives are stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• market_analyzeA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read-only and non-destructive behavior. Description adds value by explaining graceful degradation (if components are unavailable), score range (-100 to +100), and the composite nature, providing useful behavioral context beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is well-structured with an overview, behavioral details, and arguments/examples. It is somewhat lengthy but front-loaded with key information; every sentence serves a purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's composite nature, schema coverage, annotations, and output schema existence, the description adequately covers return values, parameter usage, and graceful degradation. No major gaps identified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema already provides clear descriptions for both parameters (symbol and response_format). Description adds examples and clarifies response_format options ('markdown' vs 'json'), but does not add significant meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it performs composite market analysis combining momentum, news, and earnings. Verb ('analyze') and resource ('market') are specific, and the composite nature distinguishes it from sibling tools like market_quote or market_news.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description explains it provides a combined score using quote, news, and calendar, implying use when a consolidated verdict is needed. However, it does not explicitly state when not to use it or point to alternatives like individual data tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• market_calendarA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnlyHint and idempotentHint annotations, the description adds that the endpoint may be premium-gated on the free tier and degrades gracefully, providing useful behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief yet informative, with a clear opening statement, a note on limitations, and concise examples that illustrate usage without unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity and the presence of an output schema, the description covers the key points: what it returns, the premium-gating behavior, and typical use cases, making it fully adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already fully describes both parameters with clear descriptions, so the description adds no new semantic information beyond restating the defaults and examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool fetches an earnings calendar for a specified number of days, distinguishing it from sibling tools like market_news or market_quote which focus on different market data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes example queries that imply when to use the tool (e.g., 'Which companies report earnings this week?'), but it does not explicitly compare with alternatives or state when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• market_newsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate read-only and non-destructive. Description adds behavioral details: top 5 results, 7-day lookback, and data source (Finnhub free tier). No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise and well-structured: single sentence intro, then behavior bullet, then Args, then examples. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers purpose, behavior, parameter usage, and examples. Output schema exists, so return format further documented. No missing information for an agent to use this tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema provides descriptions for both parameters, so the description's Args line adds little new semantic meaning. However, it clarifies usage in context (e.g., symbol optional, response_format enum values). Baseline 3 due to schema already covering parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it fetches market news, distinguishing between company-specific and general headlines. It uses specific verbs and resources, and differs from sibling tools like market_quote which provide stock quotes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly explains behavior with and without a symbol, including limits (top 5, past 7 days) and examples for common queries. Lacks explicit when-not-to-use or alternatives to siblings, but context and examples suffice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• market_scanA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Adds important behavior: ranking by % change, accepting up to 25 symbols, and gracefully skipping unknown/delisted tickers. Complements annotations (readOnlyHint, idempotentHint) without contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise with clear structure: purpose, constraints, then bulleted examples. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity, annotations, and output schema existence, the description fully informs the agent about behavior, constraints, and usage context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description repeats the max 25 symbols constraint already in the schema, but adds helpful examples. Schema coverage is effectively high due to nested parameter descriptions, so the description adds marginal semantic value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool scans a watchlist and ranks by absolute % change, distinguishing it from siblings like market_quote or market_analyze.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly mentions it's useful for monitoring portfolios or sector baskets, and provides multiple examples. However, it does not explicitly state when not to use or list alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• broker_positionsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false. The description adds that it never places orders or moves money, and details behavior when keys are missing (returns graceful message with setup instructions), which goes beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, well-structured with sections for description, instructions, args, and examples. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (1 param, read-only, has output schema), the description covers all needed context: purpose, prerequisites, behavior, and parameter usage. Output schema exists, so return format doesn't need separate explanation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite schema description coverage being 0% per context, the description explains the single parameter response_format with values 'markdown' or 'json', and provides examples. This compensates well for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Read open positions from an Alpaca paper trading account.' It specifies the resource (open positions) and verb (read), and distinguishes from sibling tools like market_analyze, market_calendar, etc., which are about market data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains that the tool is optional, requires API keys, and returns a graceful message if keys are absent. It also states read-only behavior. Examples illustrate typical usage. No explicit exclusions or alternatives, but context makes it clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.