Server Quality Checklist

Tool Scores

• generate_batch_reportA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It adds useful context by noting that analysis happens 'concurrently' and that the result is a 'combined report' with specific metrics. However, it does not explicitly state whether the operation is read-only, the nature of the report structure, or any potential side effects or limitations (e.g., rate limits, failure handling).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that is front-loaded with the action ('Concurrently analyze') and resource ('multiple trader profiles'), followed by the output contents. Every word contributes, with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of an output schema, the description should clarify what the 'combined report' looks like (e.g., per-profile breakdown vs. aggregated summary) and any edge cases. It lists key fields but leaves structural ambiguity about how results are organized, making it minimally complete but not fully self-sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for the single parameter profile_ids, with a clear description in the schema. The tool description merely echoes 'multiple trader profiles,' adding little beyond the schema's 'Array of profile IDs (wallet addresses or usernames).' Thus the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Concurrently analyze multiple trader profiles and generate a combined report' with specific output components (PnL, risk, score, bot status). This distinguishes it from sibling tools like analyze_trader_strategy (single profile) and find_top_traders (discovery).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'multiple trader profiles' implies a batch use case, but it does not explicitly contrast with analyzing profiles individually via analyze_trader_strategy, nor does it state conditions for when this tool is preferred. Usage context is only implied, not explicitly guided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_trader_strategyA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of disclosing behavior. It mentions the methodology (trade history and LLM classification) and the return fields, which adds transparency. However, it does not disclose potential limitations (e.g., data freshness, latency, reliance on profile existence) or safety characteristics beyond the fact that it is an analysis function.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the core action and immediately specifies what the tool returns. Every word earns its place; there is no redundant phrasing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (one parameter) and the absence of an output schema, the description provides adequate context by listing the return categories (strategy type, risk level, success score, bot detection). It does not detail output structures or error scenarios, but for a simple analyzer this is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides a thorough description of the single parameter (profile_id) including format and examples. The description adds no additional parameter semantics, so the baseline score of 3 for high schema coverage applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb ('Analyze') and resource ('a trader's strategy'), and distinguishes it from siblings by focusing on individual trader analysis rather than discovery (find_top_traders) or batch reporting (generate_batch_report). It also enumerates the key outputs, leaving no ambiguity about the tool's scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use when a specific trader's strategy needs evaluation, but it does not explicitly state when to prefer this tool over alternatives or when not to use it. There is no mention of exclusions or comparisons with sibling tools, so the guidance is only implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_top_tradersA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses the bot-detection behavior based on trade frequency and volume, which is useful. However, it does not detail return format, whether bots are filtered or flagged, or other operational aspects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long with no unnecessary words. The primary action is front-loaded, and the second sentence adds valuable behavioral context about bot detection.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with two well-documented parameters and no output schema, the description is reasonably complete. It explains the main purpose and a key behavior, though it could clarify what data is returned and how bot detection affects the output.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with both parameters clearly documented in the schema. The description does not add additional parameter-specific meaning beyond what the schema already provides, so the baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool fetches top traders from the Polymarket leaderboard, with a specific verb and resource. It adds a unique capability (bot detection) that distinguishes it from siblings like analyze_trader_strategy and generate_batch_report.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The context is clear: use this to obtain leaderboard data. It implies a straightforward use case without explicit alternatives or exclusions, but the sibling tool names are distinct enough that the intended usage is evident.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.