Server Quality Checklist

Tool Scores

• finish_loginA
  4/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations declare idempotentHint=true, but the description states the code is one-time and that a rejected code requires calling start_login again for a fresh one. Repeated calls with the same arguments are therefore not idempotent, so the description contradicts the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three dense sentences cover the workflow, file permissions, live verification, postcondition, and recovery path. The main action is front-loaded and every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the prerequisite, success condition, file permission, and failure recovery, which is strong for a one-parameter tool. However, the misleading idempotentHint leaves retry semantics ambiguous despite the one-time-code warning.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents the code parameter. The description adds important meaning beyond the schema: the code comes from start_login, is valid for 10 minutes, and is single-use.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as the second step of connecting Metrika, specifying that it exchanges the confirmation code from start_login for an access token, saves it, and verifies it with a live request. This distinguishes it from siblings by naming its role and the exact operation it performs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says this is the second step after start_login, explains that other tools become usable immediately after success, and tells the agent to call start_login again for a fresh code if the current code is rejected. This is clear when-to-use and recovery guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_countersA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses that the tool returns counters with id, name, and site2, and that it supports a search filter. It does not contradict the annotations (read-only, idempotent, non-destructive). It does not mention rate limits or errors, but for a simple read operation this is acceptable. The behavior is transparent enough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded, stating the main purpose in the first clause. It includes relevant supplementary information (id usage in other tools) without unnecessary verbosity. The structure is clear and free of redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read operation, the description is complete. It explains what the tool returns (counters with specific fields) and mentions the search filter, which is the main functionality. Pagination is implicit via the schema's offset and perPage parameters. The lack of an output schema is compensated by the description of the returned fields. It is adequate for an agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds little beyond the schema: the search parameter is described in the schema as a substring filter, and the description repeats that. The mention of counter fields (id, name, site2) is about the response, not parameters. Since schema coverage is 100% and descriptions are provided, the baseline is 3, and the description does not elevate it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns Yandex Metrika counters available to the token, and it specifically mentions that the id field is used in related tools (get_statistics and list_goals), which helps differentiate it from those tools. The verb 'Returns' is precise, and the resource is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by stating that the id from counters is used in get_statistics and list_goals, suggesting this tool is a prerequisite for those operations. It also explains the search parameter for filtering. However, it does not explicitly state 'when to use this tool vs alternatives' in direct terms, but the context is sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_statisticsA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark it as read-only, idempotent, and non-destructive. The description adds valuable behavioral context: default returns a single aggregated row, totals field should not be summed, sampling with sample/accuracy behavior, autoPaginate semantics, and default counter/env variable fallback. This goes well beyond the structured annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single dense paragraph, but it is logically organized: purpose, default behavior, dimensions/metrics, response semantics, then default counter. Though long, it avoids waste and includes practical notes (autoPaginate, sampling). Loses a point for lack of bullet points or clear sections.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers default values (date range, limit, counter), response structure (totals, pagination), and sampling behavior. Missing explicit error cases or output schema, but annotations and parameter descriptions cover most operational needs. It's solid for an API wrapper.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers all parameters with descriptions, but the tool description adds crucial usage context: date formats and relatives, goal ID format (from get_goals), explanation of autoPaginate, and the meaning of `–` prefix for sort. This materially helps an agent set parameters correctly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Сообщается конкретный глагол + ресурс:

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions that goal IDs come from get_goals, implying a relationship, and states defaults for date range and counter ID allocations. However, it does not explicitly state when to prefer this tool over get_analytics, get_visits, or raw_request, nor does it state when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_goalsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnly, idempotent, and non-destructive behavior. The description adds transparency by mentioning the default counterId (YANDEX_METRIKA_COUNTER_ID) and that it uses the Management API, providing useful context without conflicting with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise: it states the main purpose, the use case, and the default behavior in three short sentences. The parameter description is also brief and to the point. No unnecessary details or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description provides enough context for an agent to understand when and why to call this tool: to fetch goal IDs for conversion metrics in get_statistics, with a default counterId. It does not detail the return structure, but given the simple listing nature and lack of output schema, this is acceptable. The mention of Management API adds relevant context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The parameter counterId is described in the schema, and the description reinforces its role and default. The description adds context about why the parameter matters (goal IDs for conversion metrics), but the core parameter semantics are already clear from the schema, so the added value is moderate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Возвращает цели (конверсии), настроенные на счётчике Метрики' (Returns goals configured on the Metrika counter). It also distinguishes itself from siblings by specifying it's for listing goals, not counters or statistics, and explains the relevance to get_statistics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool: to obtain goal IDs needed for conversion metrics in get_statistics. It also notes the default counterId behavior. However, it does not explicitly contrast with sibling tools like list_counters, though the purpose is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• raw_requestA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Given destructiveHint=true and readOnlyHint=false in annotations, the description adds crucial context about write operations requiring confirmWrite=true, which is not in the annotations. It also mentions that POST/DELETE are write operations. However, it doesn't detail what is destroyed or the exact consequences, but the annotations already set the destructive nature, and the description adds a practical guard.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, front-loaded with the purpose, and each sentence earns its place. It includes examples, usage conditions, and parameter behavior without redundancy. The structure is efficient and easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 5 parameters and nested objects, the description together with the schema provides adequate guidance. It lacks mention of response format, but there is no output schema, and for a generic request tool, response behavior is inherently variable. It covers the essential aspects: when to use, how authentication works implied, and write confirmation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters. The description adds value by explaining the role of 'query' as query string parameters and 'body' as JSON for POST, which aligns with the schema. However, it doesn't add much beyond the schema for paths, method defaults, or confirmWrite requirements beyond what the schema describes.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it is a universal request tool that directly accesses any path of the Yandex Metrica API, with concrete examples. It also explicitly notes it is for endpoints without a dedicated tool, distinguishing it from siblings like list_counters and get_statistics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says when to use it (for endpoints without a dedicated tool) and implies when not to (when a dedicated tool exists). It also provides conditions for POST/DELETE requiring confirmWrite=true, and notes GET is free. This effectively routes agents to the right tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• logoutA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations already indicate destructiveHint=true, but the description adds important context: what exactly is deleted, what remains (env token), and that the app's access stays active on Yandex's side. This exceeds what annotations provide alone.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each providing distinct value: the primary action, the exception, and the side effect. No fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a no-parameter destructive action with robust annotations and no output schema, the description fully covers what an agent needs to know: what is affected, what isn't, and the external implication.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so the description doesn't need to explain any. It correctly focuses on the effect of the operation itself.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool removes the saved Metrika token from disk, and distinguishes it from the environment variable token. It uses precise language and is not a tautology.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains what the tool does and explicitly mentions what it does NOT do (env var token). It doesn't explicitly compare to siblings like start_login or finish_login, but the context makes it clear this is a logout operation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• start_loginA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations already declare readOnlyHint, openWorldHint, idempotentHint, and destructiveHint false. The description adds meaningful behavioral context beyond those hints: no config editing or client restart is required, the code expires in 10 minutes, and the code can only be exchanged by this server. It does not discuss exact return URL format or error conditions, but the safety profile is already covered by annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Every sentence adds a distinct piece of information: the step's role, the no-config/no-restart property, the returned link, the exact user actions, the handoff to finish_login, the 10-minute validity, and the security property. There is no filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter first step of a two-step flow, the description is nearly complete: it explains what is returned, what the user must do, where the code goes next, and the code lifetime. It does not mention what to do if the user is already authenticated, but that is a minor gap given the sibling auth_status tool and the focused scope.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema carries no burden; the baseline is 4. The description clarifies that no configuration changes are needed and focuses on the returned OAuth link rather than any inputs, which is sufficient for a parameterless tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Первый шаг подключения Яндекс Метрики', naming a specific action (starting Yandex Metrika OAuth connection) and its resource. It also differentiates itself from the sibling finish_login by explicitly stating that the received code must be passed to finish_login, so an agent can distinguish the two halves of the flow.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit user-facing instructions: show the full link, ask the user to open it in a browser under an account with access to the needed counters, confirm access, and send back the confirmation code. It also names the follow-up tool finish_login, making the intended sequence unambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• auth_statusA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description transparently discloses that the tool does not send anything over the network and does not expose the token itself, effectively communicating its read-only and privacy-preserving nature. This aligns with the annotations and gives additional context beyond basic metadata.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise—two sentences—and efficiently covers the tool's purpose, the information it provides, its side effects, and when to use it. There is no unnecessary verbosity; every sentence contributes to understanding the tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of an output schema, the description sufficiently outlines what the tool reports (connection status, token source, expiry, file location) and when to invoke it. It also mentions the tool's non-destructive and non-network nature, providing enough context for a user to decide whether to call it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so no parameter descriptions are needed. The description does not need to explain parameters, and the schema reflects this. The description is complete in this regard.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: to show whether Yandex Metrika is connected, including token presence, source, expiry, and file location. It also provides a condition for when to call it (if Metrika tools report connection not configured). This is specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says when to use the tool: 'Вызовите это, если инструменты Метрики отвечают, что подключение не настроено.' It also mentions that it does not send anything to the network and does not reveal the token, providing clear behavior expectations for users.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.