Server Quality Checklist

Tool Scores

• get_signalsA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits. It only states that the tool retrieves signals with a limit parameter and returns formatted data. It does not mention whether the operation is read-only, any required authentication, rate limits, or side effects. The description lacks transparency about how 'recent' is defined or if the tool triggers state changes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is remarkably concise: a single-sentence summary followed by a short explanation and clear Args/Returns sections. Every sentence provides meaningful information without redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no required inputs, an output schema exists), the description covers the essentials: what signals are, the direction indicator, and the limit parameter. It does not define 'recent' or explain any pagination, but for a straightforward read tool this is mostly adequate. An output schema is present, so return details are not required.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has zero description coverage for the 'limit' parameter, but the description adds crucial semantics: it specifies the allowed range (1–10) and the default value (5). This goes beyond the schema's minimal definition, though it could elaborate further on how the limit affects results or edge cases.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool gets recent market signals showing direction of price moves. It specifies the verb 'Get' and the resource 'market signals'. The explanation that signals indicate shifts toward YES or NO distinguishes it from sibling tools like get_consensus or get_markets, providing good differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide any guidance on when to use this tool versus siblings like get_consensus, get_markets, get_opportunities, or get_settlements. There is no mention of prerequisites, contexts, or explicit recommendations, leaving the agent to infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_marketsA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description must disclose behavioral traits. It states markets are 'open and being monitored' and returns a 'formatted list.' However, it does not mention any potential side effects, rate limits, or auth needs—acceptable for a read operation but not exhaustive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is concise, front-loaded with core purpose, and structured with sections for Args and Returns. No redundant sentences, though slightly more verbose than minimal.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given output schema exists (context signal), description provides adequate context for a simple two-parameter tool. Explains parameters and return format, sufficient for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description must explain parameters. It describes 'sport' with explicit values (NBA, NFL, etc.) and 'limit' with range (1–20, default 10), adding significant value beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves 'active prediction markets currently being tracked,' with a specific verb and resource. It distinguishes from siblings like 'get_consensus' by focusing on market listing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises using this tool before querying consensus ('Use this to see what events are currently available before querying consensus'). Provides clear usage context, though lacks explicit when-not-to-use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_settlementsA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description does not fully disclose behavioral traits like rate limits, pagination, or how 'recently' is defined. It mentions 'verified outcome data' but lacks depth on what that entails—adequate but not thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, starting with a clear purpose, followed by a use-case sentence, and then structured Args/Returns sections. Every sentence serves a purpose with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with one optional parameter and an output schema (handling return values). The description covers purpose, parameter details, and a use case. Minor gap: no definition of 'recently', but overall complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds significant value beyond the input schema, which has 0% description coverage. It specifies the range '1–10' and default value for 'limit', along with its meaning. The schema only shows default and type.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Get recently settled prediction market events with verified outcomes,' using a specific verb and resource. It distinguishes from siblings like get_markets and get_consensus by focusing on settled events with outcomes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear use case: 'checking how recent consensus predictions compared to actual results.' However, it lacks explicit exclusions or alternatives, such as noting when to use get_markets instead for active events.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_consensusA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It mentions 'real-time' data and describes output fields (probability, trend direction, spread). It does not discuss rate limits, authentication, or error handling, but for a simple read-only tool, the coverage is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (~100 words) with a clear structure: a one-line purpose, a paragraph on what is returned, and labeled sections for args and returns. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has only 2 optional parameters, no nested objects, and an output schema exists, the description covers all essential aspects: purpose, parameter details, and return fields. No gaps are apparent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, so the description adds crucial meaning: it lists explicit enum values for 'sport' (NBA, NFL, etc.) and explains the 'limit' parameter range (1–20, default 10). This goes well beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Get real-time prediction market consensus probabilities', specifying both the action (get) and the resource (consensus probabilities). It distinguishes this tool from siblings like get_markets and get_opportunities by emphasizing that it returns aggregated consensus from multiple markets.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains what the tool returns but does not explicitly state when to use this tool versus alternatives (e.g., get_markets). It provides no 'when not to use' guidance or explicit comparison to siblings, sticking only to a general purpose statement.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_opportunitiesA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the full burden of behavioral disclosure. It explains that the tool returns divergence opportunities ranked by score and that higher scores indicate greater disagreement. It does not discuss authorization, rate limits, or destructive effects (not applicable). The description is transparent enough for a read operation, though it could mention if results are cached or if there are any known limitations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured. The purpose is stated in the first sentence, followed by a one-sentence explanation of divergence. The Args section clearly lists each parameter with its description, and the Returns section specifies the output format. Every sentence contributes meaning without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that the tool has an output schema (assumed structured), the description still explains the return format ('Formatted list of divergence opportunities ranked by score'). All three parameters are documented with defaults and valid values. Similarly, the sibling tools are listed for context (though not compared). The description is complete for a list-returning tool with no required parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, meaning the schema provides no documentation for parameters. However, the tool description fully compensates by listing each argument with its meaning, default values, and valid options (e.g., sport: 'NBA, NFL, MLB, NHL, MLS, POLITICS, or omit'; limit: '1–20, default 10'). This adds significant value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Get events where prediction markets show notable divergence.' It defines divergence opportunities concisely and distinguishes this from sibling tools by focusing on divergence and disagreement, which is unique among get_consensus, get_markets, get_settlements, and get_signals.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: it surfaces events with notable disagreements, possibly where information is still being incorporated. This implies when to use it (when seeking mispricings or inefficient markets), but it does not explicitly state alternatives or when not to use it. The sibling tool names are listed, but no direct comparison is made.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.