Server Quality Checklist

Tool Scores

• get_solveA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and openWorldHint=true, so the safety profile is covered. The description adds valuable context: that reading never re-bills, that the result may exist even after timeout, and the 404 meaning. This goes beyond annotations and helps the agent understand the tool's behavior in edge cases.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences long, each earning its place: the core function, the critical use case, and error interpretation. It is front-loaded with the primary action and avoids repetition of the title. No extraneous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple get-by-id tool with one properly documented parameter, the description covers the core usage: when to call it, what the id means, and how to interpret 404. It does not specify the return format, but the explanation of the timeout-retrieval scenario is the most crucial context, and it is well addressed. Premium output schema is absent, but the description needn't explain return values if not available.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for the single id parameter, so the schema documents the parameter format (slv_8Kd2aF9). The description mentions 'the id is in that error message' which adds a source for the id value, but this is more usage context than parameter semantics. Baseline 3 is appropriate since schema already covers the parameter well.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: fetch a previously created solve by its id. The verb 'Fetch' is specific and the resource 'solve' is unambiguous. It also distinguishes itself from solve_turnstile by explicitly mentioning the timeout-and-retry scenario, so an agent can differentiate it from siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use guidance: use this when solve_turnstile timed out while pending. It also explains the 404 error semantics, which helps an agent interpret the outcome. No alternatives are named, but the context is clear enough to avoid misuse.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• solve_turnstileA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Adds valuable behavioral context beyond annotations: tokens are single-use, expire in minutes, failed solves are never billed, and the tool needs SOLVEGATE_API_KEY. These lifecycle and billing details materially affect invocation and go beyond what readOnlyhew/openWorldHint/idempotentHint convey.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Tightly written and front-loaded with purpose, then delivers the parameter nuance, token lifecycle, and billing policy in sequence. No filler or redundant repetition of schema content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema and rich annotations, the description supplies the remaining operational context: API key requirement, token lifecycle, failure billing, and host-binding constraint. An agent has enough to select and call this tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although all parameters are already described in the schema, the description adds meaningful semantics: it explains when each `gate` value applies, why the URL must be the real rendering page, and why sitekey/url are both required. It does not add details for proxy or action, but those are adequately covered in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb ('Clear') and resource ('Cloudflare challenge') plus the token output. It clearly identifies the two gate flavors but does not explicitly distinguish this tool from sibling tools like inspect_page or get_solve, which is the only thing preventing a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context for choosing gate=turnstile vs gate=waf, and explains the URL must be the actual page where the challenge renders because sitekeys are host-bound. It does not explicitly state when not to use this tool or compare it to siblings, so it falls just short of full guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• inspect_pageA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnlyHint, it adds meaningful behavioral specifics: it performs an HTML inspection rather than solving a challenge, requires no API key, is free, and has a specific hard limitation around widget mode. These details substantially describe side effects and boundaries beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Although the description is long, each clause delivers a distinct, useful fact: what is fetched, what is reported, cost/auth requirements, usage timing, and a critical limitation. The most important action and resource are front-loaded, and no sentence is slack.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given a single required parameter, an output schema, and annotations, the description covers everything an agent needs: when to call it, what it returns, its constraints, and what it cannot infer. The mitigation of return format details is appropriate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema fully describes the single 'url' parameter with an example, so parameter documentation coverage is 100%. The description does not add much beyond the schema, but the schema already carries the necessary semantic weight. This aligns with the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Fetch') and a concrete result ('report what its HTML says about Cloudflare Turnstile') and enumerates exact outputs: widget presence, sitekey, render method, data-* configuration, test keys, and full-page WAF challenge. This clearly separates it from sibling solving/result-retrieval tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says 'Use this before solving when the sitekey is unknown,' giving a direct invocation condition. It also provides an exclusion: it cannot determine the widget's mode because that data is not in the page, so agents know when this tool is not sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.