Server Quality Checklist

Tool Scores

• submit_codeA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate this is a mutating action (readOnlyHint=false) and may have external effects (openWorldHint=true). The description adds the copy-paste flow context but does not disclose what happens after submission (e.g., session establishment, error behavior, idempotency). No contradiction with annotations, so a baseline score is appropriate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that conveys the action, the prerequisite, and the parameter source without wasted words. Every clause earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter tool with no output schema, the description is sufficient: it ties the tool to the login flow and specifies exactly what input to provide. The only omission is post-submission behavior, but given the low complexity and high schema coverage, this is not a significant gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and both the schema property description and the tool description explain the 'code' parameter consistently. The description restates the code's origin ('shown on the Yandex page after you approved access') but adds no new semantic detail beyond the schema, so the baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action: completing a copy-paste sign-in by passing the code Yandex showed. It explicitly references the login sibling tool, making the purpose unambiguous and distinguishing it from other Yandex API tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context by noting this completes a sign-in 'started by login,' implying it should be used after initiating login. It does not explicitly list when not to use it or name alternatives, but the prerequisite and flow are clear enough for an agent to sequence correctly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_diagnosticsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses behavioral traits beyond annotations: returns active problems by default, severity levels, worst-first ordering, and optional includeAbsent flag. Aligns with readOnlyHint and openWorldHint, adding value.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, efficient, front-loaded with the core purpose and immediately useful details (severity, defaults). No redundant phrasing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description sufficiently explains what the tool returns, the states, and the parameter effect. Without an output schema, it provides enough context about severity and ACTIVE filtering. Some format details are absent but acceptable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema covers both parameters fully. The description reiterates includeAbsent behavior and mentions hostId default via get_hosts, but adds no new parameter-specific meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as retrieving Yandex Webmaster site diagnostics, with specific examples of problem types and severity levels. It distinguishes itself from sibling get_* tools by focusing on host-level problems.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides context on default behavior (active problems only) and how to change it (includeAbsent). It implicitly signals use cases (checking site health) but doesn't explicitly mention alternatives or conditions for using this vs other tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_external_linksA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description explicitly states 'Read-only,' which is consistent with the readOnlyHint annotation. It goes beyond annotations by detailing what each report returns (sample links with source page, target URL, discovery date, plus total count; time series of total count over time). This adds useful behavioral context without contradicting structured data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured, with a clear opening statement followed by two bullet points for the report types. Every sentence provides useful information, and the format is easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the relatively simple tool (read-only, no output schema), the description covers the essential context: what the tool returns for each report type. It lacks detailed response structure or error handling, but that's not critical for a read-only tool with well-defined report outputs.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds some context by clarifying that 'limit' and 'offset' apply only to report='samples' and that report='history' returns a time series, but this information is already present in the parameter descriptions. The description reinforces defaults but doesn't significantly augment the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Inbound (external) links pointing to the host — the backlinks Yandex knows.' It uses a specific verb ('get') and resource ('external links') with clear scope, and distinguishes itself from siblings by focusing on backlinks, which no sibling tool covers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use the tool and how to use it, explaining the two report modes ('samples' and 'history') with their respective outputs. It doesn't explicitly mention alternatives, but since no sibling tool provides external link data, this is not a significant gap.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_queriesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint and openWorldHint, and the description repeats 'Read-only' plus adds specific behavioral details: the ranked-list vs. time-series modes, how queryId affects the trend, and default ordering. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded: it states the purpose, then breaks down report modes with bullets, and ends with a useful tip. Every sentence earns its place with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 10 parameters and two modes, the description provides enough context about result types (ranked list, time series) and usage flow. An output schema is absent, but the description covers the main user decisions and even suggests external correlation, making it fairly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema descriptions cover 100% of parameters with defaults, enum constraints, and usage notes. The description adds minor usage context (e.g., 'top' for seeing phrases, 'trend' aggregate) but does not materially extend what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it provides search-query analytics for a host, listing specific metrics (impressions, clicks, average positions). This distinguishes it from sibling tools focused on indexing, sitemaps, and diagnostics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly explains the two report modes and when to use each: 'top' to see which phrases people search, 'trend' for time series. It also adds a practical tip about cross-referencing with Yandex Metrica. It doesn't explicitly name alternatives among siblings, but the context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_hostsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With readOnlyHint annotation already present, the description adds valuable behavioral context: it lists response fields, explains the optional hostId behavior that fetches a host summary, and notes the host_id dependency. It does not discuss auth or rate limits, but that is not expected here given the annotations and simple scope.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with 'Discovery tool,' and no filler. Every sentence adds value—purpose, output contents, optional parameter behavior, and usage order.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given one optional parameter, no output schema, and simple scope, the description is complete: it names the returned fields, explains the optional summary, and positions the tool as the first call. No important aspect is missing for the agent to select and invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already fully documents hostId, including optionality and behavior. The description reiterates that behavior without adding new syntax or format details. However, it reinforces the hostId's importance by noting that other tools need it, which is useful context beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'list' and the resource 'sites (hosts) available to your Yandex Webmaster token,' with specific output details (host_id, URL, verification state). It establishes this as the discovery tool and distinguishes it from sibling tools by saying 'Call this first' and explaining host_id is needed by every other tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Call this first' and explains that host_id is required by other tools, giving clear context for when to use it. It does not name alternative tools or explicitly state when not to use it, but the guidance is strong enough to earn a 4.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_sitemapsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint=true, so the safety profile is known. The description adds value by detailing the output fields (URL, type, last access date, URL count, error count, discovery source), which is particularly useful since there is no output schema. It gives a concrete sense of what the tool returns beyond the read-only flag.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loads the core action and result set, and includes a useful usage directive. Every clause earns its place without redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of an output schema, the description compensates by enumerating the returned fields and expected granularity. The simple tool shape (2 optional params, no nested objects) means this description fully covers what an agent needs to select and invoke the tool correctly. It also fits logically among sibling tools without ambiguity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% — both 'hostId' and 'limit' have descriptions, so baseline is 3. The tool description does not add additional parameter-level semantics beyond what the schema already provides, so no elevation is warranted.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description begins with a specific action and resource: 'List the Sitemap files Yandex knows for the host.' It clearly enumerates the fields returned, distinguishing this from sibling tools like recrawl_status or get_indexing. This is a precise, non-tautological statement of purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states the intended use: 'Use it to check a sitemap is being read and is error-free.' It provides clear context for when to call the tool, though it does not explicitly mention alternatives or exclusion cases. The added 'Read-only' note reinforces safe usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_indexingA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, and the description reinforces this with 'Read-only' while adding valuable behavioral details: default report, grouping by HTTP status classes, and return content for each report. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact, front-loaded with a summary, and uses a bulleted list for the three report types. Every sentence adds value, with no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description compensates by describing exactly what each report returns (time series, example URLs, in-search URLs with titles) and the use case for each. It also notes the default report, making it complete for a 6-parameter read-only tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds cross-parameter meaning: it explains the report enum values, the default, that dateFrom/dateTo apply only to history, and that limit/offset apply to sample reports. This goes beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns reports about how Yandex crawls and indexes the host, and enumerates three distinct report types with specific outputs. This distinguishes it from sibling tools like get_diagnostics and list_sitemaps.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives explicit guidance on when to use each report, e.g., history is for spotting spikes of crawler errors. However, it does not mention alternative tools or explicitly state when not to use this tool, so it lacks exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• recrawl_statusA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and openWorldHint=true, and the description adds meaningful context such as the states (IN_PROGRESS, DONE, FAILED), the notion of a daily quota, and the temporal relationship with recrawl_submit. It does not repeat the annotations but complements them with operational details, such as the default behavior when taskId is omitted.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each earning its place. It front-loads the read-only nature, then describes the two modes concisely, and ends with a practical usage tip. No redundant words or unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema, the description adequately explains what the agent can expect: remaining quota, recent tasks, and their states. It also covers the two invocation modes and provides a clear use case ('before recrawl_submit'). The moderate parameter count and read-only annotation reduce the need for additional behavioral disclaimers, making this complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for all three parameters, and the schema descriptions already explain the limit, hostId, and taskId semantics. The description reinforces the conditional logic of taskId and the limit's role, but it does not add substantial new meaning beyond the schema, so a baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Read-only view of the host recrawl activity,' clearly stating the verb and resource. It further differentiates the tool by describing the quota and task-list output, and explicitly distinguishes it from recrawl_submit by noting the taskId origin and the pre-submit usage.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit usage guidance: 'Use before recrawl_submit to confirm quota is available' and differentiates between the no-taskId mode (quota + recent tasks) and the taskId mode (check one task). It also references recrawl_submit as the source of taskId, clarifying when to use this tool versus the sibling.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• recrawl_submitA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description goes beyond annotations by stating the specific side effect (consumes one unit of limited daily quota), and discloses return values (task_id and remaining quota). Annotations already indicated non-read-only, but the description adds context about quota limits and the returned tracking identifier.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences cover purpose, quota warning, URL requirements, and return value/tracking. Every sentence earns its place with no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Since there is no output schema, the description correctly explains the return values (task_id and remaining quota). It also covers the quota limitation, URL constraints, and how to track progress, making it complete for a tool with only two parameters and straightforward behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and both parameters (url and hostId) are already described in detail. The description only reiterates the URL constraint ('must belong to the host, full absolute URL') already present in the schema, adding no new semantic meaning. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Ask Yandex to recrawl a specific page'), identifies the resource (a URL belonging to the host), and distinguishes itself from recrawl_status by framing this tool as a submit action and recrawl_status as the status tracker.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly tells users to check quota with recrawl_status before calling, warns that it is NOT read-only and consumes quota, and directs tracking progress via recrawl_status. This provides clear when-to-use and alternative guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• loginA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations (readOnlyHint=false, openWorldHint=true), the description adds critical behavioral details: it opens a browser, code returns automatically via local redirect, fallback returns a URL, token lasts ~1 year, and it should be run once. These details are not in the annotations and give the agent a clear model of the tool's side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each carrying essential information: what it does, how the flow works, when to run it. It is front-loaded with the purpose and avoids any redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description fully covers the tool's operation given its simplicity: it explains the typical flow, the fallback path, the token validity, and the usage context relative to sibling data tools. No output schema exists, so describing the return value is not strictly necessary, and the description is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description for the single parameter 'oob' is already clear ('Force the copy-paste flow...'), and the tool description elaborates on the default automatic flow and the fallback to submit_code. This adds context on when one might set oob, but it doesn't explicitly tie the parameter to the fallback scenario, so a small gap remains.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Sign in to Yandex from here,' clearly stating the verb (sign in) and resource (Yandex). It distinguishes itself from sibling data tools by focusing on authentication, and the description of the browser-approval flow reinforces its unique purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool: 'needed before the data tools if you are not signed in yet.' It also provides an alternative flow by naming 'submit_code' when the local port is unavailable, giving clear context on when to use this tool versus fallback.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.