Server Quality Checklist

Tool Scores

• leaderboardA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavior. It clearly indicates a read-only ranking operation and lists return fields. However, it does not mention rate limits, pagination, or side effects, which would be beneficial for a complete transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a brief purpose statement, usage context, and a clear args list. Every sentence adds value, with no redundancy or unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description lists return fields adequately. It covers parameter details and implies differentiation from siblings. However, it lacks information on error handling, empty results, or performance limits, leaving minor gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the parameter explanations in the description are critical. The description provides clear semantics for each parameter: sort options, category examples, filter choices, and limit range. This fully compensates for the missing schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool ranks Polymarket whales by performance metrics, with examples of sorting keys and filtering by category. It distinguishes from siblings such as 'wallet_overview' (individual wallet) or 'markets' (market data).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use the tool (find top traders) but does not explicitly list alternatives or when not to use it. Sibling tools like 'wallet_overview' or 'whale_alerts' are not mentioned, leaving some inference needed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• whale_alertsA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses return content (who traded, buy/sell, amount, etc.) and the non-error behavior without a key. However, it lacks details on rate limits, caching, or real-time latency, leaving some behavioral aspects implicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a one-line summary followed by a return value list and an Args section. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and four parameters, the description adequately explains inputs, Premium requirement, and return fields. It lacks explicit output structure details but lists the fields returned, which is sufficient for a feed tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description adds comprehensive meaning to all four parameters: minutes (lookback window, max 1440), min_usd (minimum trade size), category (restrict or None), limit (1-100). This fully compensates for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns 'Recent trades by smart-money whales' as a 'live alerts feed (Premium)', specifying the verb 'returns' and the resource 'whale trades'. It distinguishes from siblings by emphasizing its real-time nature and Premium requirement, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states the requirement for a Premium API key via ORCALAYER_API_KEY and describes the behavior without a key (returns a notice). It provides clear context for when to use the tool, though it does not explicitly mention when not to use or suggest alternative tools among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• marketsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Describes return fields (question, price, whale counts, volume, days left). No annotations, so description covers behavioral aspects well.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Efficiently written with summary and args list. Could be slightly shorter but well-structured and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 params, no output schema, no annotations, description is largely complete. Missing error handling but covers core usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All 5 parameters explained beyond schema: q accepts URLs/slugs, category lists options, min_volume/min_whales as filters, limit range. Compensates for 0% schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool searches Polymarket markets with optional smart-whale clustering. Distinct from siblings which focus on leaderboards or wallets.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says to use when finding markets by topic and smart-money interest. Does not explicitly exclude cases or mention alternatives, but context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• wallet_positionsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses sorting by current value, returns compact form with omitted count, and that wallet may hold more than limit. No annotation provided, so description handles burden well.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short paragraphs with an Args section. Every sentence adds value; no filler. Well-structured and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers inputs and some output behavior, but lacks description of individual position fields (especially given no output schema). Vague 'compact form' leaves agent guessing the structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% coverage; description adds full meaning: address accepts 0x address or nickname, limit is from 1 to 50 with default 15. Compensates completely.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states 'list a wallet's largest open positions by current value' with specific verb and resource. Distinct from siblings which are leaderboard, markets, overview, alerts.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explains inputs (address, limit) and output behavior (sorted, truncated with omitted count). No explicit when-not or alternatives to siblings, but context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• wallet_overviewA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, but description discloses possible computing status and retry logic, and clarifies it returns a summary rather than raw data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Front-loaded purpose, brief sentences with no redundancy, well-structured with Args section.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-param tool with no output schema, description covers input, output types, and edge case (computing status) comprehensively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Single parameter 'address' is fully explained in description as accepting a 0x address or nickname, compensating for the 0% schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it summarizes a wallet's trading profile and performance, distinguishing it from sibling tools like 'wallet_positions' which likely provide detailed positions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Specifies acceptable inputs (0x address or nickname) and mentions a retry behavior for computing stats, but does not explicitly compare to alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.