Server Quality Checklist

Tool Scores

• get_forecastA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses the model version, error cases (404, 503), and output structure. It does not mention rate limits or authentication, but these are less critical for a read-only tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections: purpose, method details, arguments, returns, and raises. Every sentence adds value; no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the existing output schema, the description does not need to explain return values. It covers inputs thoroughly and lists error cases. It lacks mention of any side effects or prerequisites, but the tool is straightforward.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description fully compensates. It explains appid with an example, and describes wishlist and followers as optional overrides for what-if mode, adding meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool fetches a calibrated P10-P90 revenue cone for a Steam game by appid. It specifies the model version and that it powers the public site, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains what the tool does but provides no guidance on when to use it versus sibling tools like boxleiter_estimate or get_calibration_summary. Usage context is only implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_compsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the embedding-based similarity method and the return format, which is transparent. It does not mention authorization or error cases, but remains informative for a read-only tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is about 6 sentences, well-structured with a clear purpose, technical detail, usage context, and parameter/return specs. Every sentence adds value, with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (2 params, no nesting) and the presence of an output schema, the description covers purpose, method, usage, parameters, and return fields. It lacks error handling details, but the completeness is high for a straightforward tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, meaning no descriptions in the schema. The description compensates by explaining both parameters: appid is the Steam app ID, k is the number of comps (1-20, default 5). This adds full semantic value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it fetches top-K nearest-neighbor comparable Steam games for an appid. It specifies the resource (comps) and action (fetch), and distinguishes from sibling tools like boxleiter_estimate by focusing on comps rather than estimation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a specific use case: 'sanity-checking a forecast.' It explains why comps are useful but does not explicitly state when not to use it or mention alternative tools. However, the context is clear enough for appropriate selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_methodologyA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Although no annotations are provided, the description fully discloses the tool's behavior: it pulls the canonical llms.txt file, returns plaintext markdown content, and lists the type of URLs included. The read-only nature and lack of side effects are evident.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is reasonably concise with a clear title-like first sentence, a short explanation paragraph, and a Returns section. It could be slightly more terse by avoiding slight repetition, but overall it is well-structured and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters and an output schema exists (implied), the description sufficiently explains the return type (plaintext markdown), content (URLs for methodology, guides, reports, tools), and source. No additional information is needed for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters, and the input schema is empty (100% coverage vacuously). As per guidelines, baseline score is 4 when there are 0 parameters. The description does not need to add parameter info.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it returns the AI-crawler-friendly methodology summary (llms.txt) from steamforecast.app/llms.txt, which lists high-quality URLs for AI agents. It distinguishes itself from a single forecast tool (get_forecast) by emphasizing it provides a full sitemap of authoritative content.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description advises using the tool when a model wants the full sitemap of authoritative content rather than a single forecast, implicitly differentiating it from the sibling get_forecast. While it does not explicitly exclude other siblings, the context is clear and sufficient for an agent to decide.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_calibration_summaryA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses the return structure (dict with aggregate, per-stratum, sample sizes, links). With no annotations, this provides solid transparency, though it could mention idempotency or caching.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short, front-loaded sentences. No fluff; every sentence adds useful information. Efficiently structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With an output schema present, the description complements it by explaining the data source and linking to live data. For a zero-parameter tool, this is fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, and schema coverage is 100%. The description adds value by explaining what the output contains, justifying the lack of parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states verb 'Return' and resource 'calibration coverage summary'. Differentiates from sibling get_methodology by specifying that this tool returns quarterly report numbers while get_methodology provides canonical current values.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly mentions the data source (Q2 2026 quarterly report) and directs users to fetch get_methodology for current values, providing clear when-to-use and alternative guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• boxleiter_estimateA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses it's a heuristic, not a forecast, includes error rate from author. Describes return structure with low/median/high brackets and calibration warning, compensating for lack of annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two paragraphs plus Args/Returns section, front-loaded purpose, every sentence adds value. Efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given output schema exists, description explains return structure in detail. All parameters documented. Tool is simple; description is fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has no descriptions (0% coverage), but description fully explains both parameters: review_count as total Steam reviews, price_cents as list price in cents with example (2499 for $24.99). Adds meaning beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it applies the Boxleiter rule-of-thumb heuristic, distinguishes from get_forecast as a sanity check. Provides specific formula and author context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says when to use (quick heuristic) and when not (not calibrated), suggests comparing with get_forecast for outlier detection. Names sibling tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.