Server Quality Checklist

Tool Scores

• get_cityA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description adds valuable behavioral context: it is free, requires no wallet or API key, and it discloses what the response includes (current target market day and Polymarket market URL). This covers access requirements and expected output, though it does not mention rate limits or error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that front-loads the core action and resource, then adds key details. Every phrase earns its place, with no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter lookup tool with no output schema, the description provides enough context about the return contents ('current target market day' and 'Polymarket market URL'). It does not cover failure modes, but the simplicity and clear purpose make it nearly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already covers the 'slug' parameter with a description, but the tool description adds context that slugs come from 'get_cities', which helps the agent understand where the value originates. This goes beyond the schema's static type/example.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Get' with a clear resource 'one Isocast city by slug', and distinguishes itself from the sibling 'get_cities' by emphasizing 'one' and 'by slug'. This makes its purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implicitly indicates when to use this tool (when you have a slug for a single city) but does not explicitly mention alternatives like 'get_cities' for listing. It provides a usage prerequisite ('by slug') but lacks explicit 'vs alternative' guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_healthA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It discloses that the tool is free and requires no credentials, and enumerates the specific behavioral output (liveness, payment mode, OFAC status, signals freshness). It doesn't explicitly state it's read-only, but the 'Check' verb implies non-mutating behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exceptionally concise: two sentences that front-load the purpose, then specify the exact sub-checks and access requirements. Every clause adds useful information, with no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is no output schema, the description does a good job of informing the agent what the response will contain by listing the health indicators and the signals freshness block. It could be more explicit about the overall response structure or what 'stale' means, but for a simple health-check tool, it covers the essential context adequately.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters, so the description's lack of parameter details is perfectly acceptable. Per the baseline for 0 params, a score of 4 is appropriate. The description adds value by describing the output fields, though parameter semantics are not applicable here.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with a specific verb 'Check' and resource 'Isocast API health', clearly distinguishing it from sibling data-retrieval tools. It enumerates the exact health aspects covered (liveness, payment mode, OFAC, signals freshness), leaving no ambiguity about its function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool: it's a health check and explicitly notes that it is free and requires no wallet or API key, implying it can be used without authentication. It doesn't explicitly mention alternatives or exclusions, but the distinct purpose and self-contained context make usage obvious.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_citiesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It explicitly states 'Free, no wallet or API key required,' clarifying authentication needs, and lists the exact fields returned. It could define 'active' or mention pagination, but for a simple read-only list, this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the action and resource, and packs relevant details (fields, use case, auth) without fluff. Every clause earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (zero params, no output schema), the description covers purpose, use case, return fields, and access requirements. There are no significant gaps for an agent to invoke or interpret the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool accepts no parameters, so the baseline for parameter semantics is 4. The description adds value by detailing what the output contains, compensating entirely for the empty schema and absence of output schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists every active Isocast city, enumerating specific fields returned (slug, display name, country, etc.). This distinguishes it from siblings like get_city which presumably targets a single city.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides an explicit use case: 'Use this to discover which weather markets have signals.' It implies a broad listing role without naming alternatives, but the context is clear enough for an agent to select it appropriately.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_signal_metaA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses return contents (pricing/payment metadata, optional latestSeq), a key behavioral exclusion (no signal rows), and access requirements (free, no wallet/API key). This covers the main behaviors an agent needs, though it doesn't mention potential rate limits or error conditions. Still, the disclosure is transparent and valuable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with a clear front-loaded main clause, details, a negative exclusion, and access prerequisites. Every clause adds value: the field list, the optional behavior, the NEVER statement, and the free/no-auth note. No fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (1 optional parameter, no output schema), and the description fully covers purpose, parameter usage, behavioral exclusions, and access constraints. It is self-contained and leaves no significant gaps for an agent to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for the single optional 'city' parameter, so the baseline is 3. The description adds real semantics by explaining that passing a city includes its latestSeq, which goes beyond the schema's generic 'Optional city slug'. This clarifies the parameter's effect and justifies a score above baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verbs and resources: 'Get Isocast pricing and payment metadata' and lists exact fields (unit price, minimum spend, currency, network, payTo, volume tiers). It clearly distinguishes from sibling tools by stating 'NEVER returns any signal rows', which sets it apart from get_sample_signal. The purpose is unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: it explains the optional 'city' parameter to get latestSeq and explicitly states exclusions ('NEVER returns any signal rows'). It also notes no auth required. However, it does not explicitly name alternative tools or state 'use this for metadata', so it falls short of a 5 by not providing direct when/when-not comparisons.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_sample_signalA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully discloses key behaviors: always marks sample:true, never fabricated, never reveals latest signal, may substitute with another city's past signal labeled via sampleNote, and shows exact bucket-transition shape. This is rich behavioral context far beyond basic read-only hints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose in the first sentence, followed by essential behavioral details and alternatives. Every sentence contributes meaningful information—no fluff or repetition. Despite a longer first sentence, the overall structure is efficient for the complexity conveyed.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given only 1 parameter, no output schema, and no annotations, the description is remarkably complete. It explains the response shape (old/new reading, buckets with yes/no/midpoint, market URL), fallback logic, and differentiation from sibling tools. An agent can select and invoke this tool correctly without needing additional details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already has 100% coverage for the sole 'city' parameter with a clear description ('City slug, e.g. 'toronto''). The tool description adds minimal new parameter-specific semantics; it implies city is a slug but does not elaborate on format or constraints beyond schema. Baseline 3 is appropriate since schema handles the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states 'Get a FREE sample Isocast signal for a city' with clear verb and resource, and further distinguishes from live signals by contrasting with buy_signals + poll_signals. It also clarifies the fallback behavior and output shape, making the tool's purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly states when to use this tool (to get a sample without paying) and provides clear alternatives: 'For live entitled signals, use buy_signals + poll_signals.' It also notes no wallet/API key is needed, setting expectations for free usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.