Server Quality Checklist

Tool Scores

• codeforces_rating_historyA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering safety and idempotency. The description adds the ordering and delta details, which is useful but minimal. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence conveys the core purpose without wasted words. It efficiently conveys the resource, ordering, and the deltas detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one required parameter, full schema coverage, and an output schema, the description sufficiently explains the purpose. The only minor gap is that it doesn't describe return format specifics, but the output schema covers that need.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and all parameters (handle, limit, response_format) already have descriptions. The tool description does not add meaning beyond the schema, so the baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool fetches contest-by-contest rating changes for a given handle, specifies ordering (oldest first), and includes deltas. This distinguishes it from sibling tools like user_profile (overall rating) and recent_submissions (submissions).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention when not to use it, nor does it reference sibling tools. Users are left to infer suitable contexts from the purpose alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• codeforces_user_profileA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, covering the safety profile. The description adds that the profile is 'public' and enumerates the data fields, which is slightly beyond annotations but does not disclose other behaviors like error handling or data freshness. With annotations present, the added value is moderate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single focused sentence that front-loads the core purpose and lists key fields. There is no redundancy or filler; every word contributes to understanding what the tool does.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only profile tool, the description is nearly complete. The output schema exists, so return values don't need explanation. The description could mention the lack of filtering or performance implications, but given the straightforward nature and other structured metadata, the information is sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides full descriptions for both parameters (handle with example, response_format with explanation). The description adds no additional parameter-specific meaning, so the baseline of 3 applies given 100% schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves a public profile for a Codeforces handle and lists the specific fields returned (rating, max rating, rank, organization). This is a distinct purpose, easily differentiated from siblings like rating_history or recent_submissions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for getting profile data but does not provide explicit when-to-use guidance or contrast with sibling tools such as codeforces_rating_history (for historical trends) or codeforces_search_problems. No exclusions or alternatives are mentioned, leaving the agent to infer context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• codeforces_upcoming_contestsA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already cover safety (readOnly, idempotent, non-destructive). The description adds behavioral context beyond those annotations by specifying that results are sorted soonest first and include start time and duration. This tells the agent exactly what to expect in the output, which is valuable. No additional behavioral disclosures (e.g., pagination, rate limits) are necessary for such a simple read-only tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that leads with the essential information ('Contests that have not started yet') and includes ordering and output fields. No fluff, no redundant wording. It is appropriately sized and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with an output schema present, so return values do not need to be described. Annotations cover safety, and the description covers the core behavior. Sibling tools are clearly differentiated. For an agent to invoke this tool correctly, it has all necessary information: what it returns, how it sorts, and the parameter constraints are in the schema. No gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%: both parameters (limit and response_format) have clear descriptions in the schema. The tool description does not add any extra meaning to the parameters. Since the schema already fully documents them, the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb+resource: it lists contests that haven't started, with a specific ordering (soonest first) and the fields included (start time and duration). This distinguishes it sharply from sibling tools like codeforces_search_problems or codeforces_rating_history, which serve entirely different purposes. There is no ambiguity about what this tool returns.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage: it is for upcoming contests. However, it does not explicitly mention when to use it versus alternatives or any exclusions. Sibling tools are clearly different, so the context is obvious, but there is no explicit routing guidance. This fits the 'implied usage' level.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• codeforces_recent_submissionsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already cover read-only and non-destructive behavior. The description adds useful behavioral context: results are ordered newest first, and each row links to the submission on Codeforces so the code can be read. This goes beyond what annotations alone provide and contains no contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a tight two sentences with no redundant words. The primary purpose is front-loaded in the first sentence, and the second sentence adds a focused filtering tip and a detail about links to code. Each sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the 100% schema coverage, existing annotations, and presence of an output schema, the description fills the remaining gaps: ordering, the verdict filter use case, and the link-to-code behavior. An agent has everything needed to decide when to call and how to interpret results, though one could imagine explicit exclusion of sibling tools, which is not necessary here.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already documents all four parameters with clear descriptions (100% coverage), so the description does not need to explain them. It does add a practical tip for the 'verdict' parameter (using it to review failures), but adds nothing for 'limit' or 'response_format' beyond what the schema states. This matches the baseline for full schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb ('List') and resource ('a handle's most recent submissions') and specifies order ('newest first'). It is clearly distinct from sibling tools that deal with problems, ratings, contests, and user profiles, so an agent can differentiate without ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a concrete, actionable use case ('Filter by verdict to pull up recent failures for review') that tells an agent when this tool is appropriate. It does not explicitly mention alternatives or when not to use it, but the purpose is specific enough that no exclusions are necessary.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• codeforces_search_problemsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already carry readOnlyHint, idempotentHint, and destructiveHint, so the safety profile is covered. The description adds behavioral value: the 'easiest first' ordering and the semantic of exclude_solved_by. It does not contradict annotations and adds useful context beyond the structured fields.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise paragraphs, the first stating the core purpose and the second adding the key differentiator. Every sentence earns its place, and the critical filter is highlighted up front. No filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with an output schema, the description covers all essential behavior: rating band, tags, ordering, and the unique exclude filter. Nothing an agent needs to invoke it correctly is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema documents every parameter. The description adds meaning beyond the schema for exclude_solved_by, explaining its practical significance and why it's the main reason to use this tool. That lifts it above the baseline 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a clear verb+resource statement: 'Find Codeforces problems by rating band and tags, easiest first.' It specifies the criteria and ordering, and its function is immediately distinguishable from the sibling tools (which cover performance, submissions, profiles, history, and contests).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to reach for this tool by highlighting the unique value of the exclude_solved_by filter that 'the Codeforces website cannot express.' This gives a concrete use case, but it does not explicitly contrast with sibling tools or state exclusions, so it stops short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• codeforces_tag_performanceA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnly/idempotent annotations, the description discloses important behavioral details: computation from full submission history, deduplication per problem (so repeated attempts count once), and the emphasis on 'meaningful number of attempts' via min_attempted. This adds context not present in annotations, making invocation expectations clear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each earning its place: the first states the core purpose, the second explains the computational nuance, and the third gives practical usage guidance. The purpose is front-loaded, and there is no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only analysis tool with an output schema (not shown but present), the description fully covers what the tool does, how results are derived, and when to use it. It omits nothing an agent needs to select and invoke the tool correctly; remaining details like response_format are handled by the schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All parameters are fully described in the schema (100% coverage), so the baseline of 3 applies. The description adds a minor tie-in by referring to 'meaningful number of attempts' which aligns with min_attempted, but it does not provide new semantic details beyond the schema definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Rank a handle's problem tags by solve rate, weakest first') with a clear resource (problem tags of a handle). It distinguishes from siblings by specifying the computed nature and the lack of a Codeforces endpoint, which other tools likely do not cover.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear use context ('Use it to decide what to practise') and explains the deduplication logic that affects ranking. It does not explicitly name alternatives or state when not to use it, but it does imply uniqueness by noting Codeforces exposes no endpoint for this, giving adequate guidance for choosing it over similar tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.