Server Quality Checklist

Tool Scores

• elba_describe_operationB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the core behavior — returning parameters, request body, and responses from the OpenAPI schema — which makes the read-only introspection nature evident. However, it does not cover error behavior (e.g., unknown path/method combination) or explicitly confirm the absence of side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single front-loaded sentence that states the verb and the returned content with zero filler. It is efficiently structured, though it leaves room for a usage hint without becoming verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a low-complexity tool (two required scalar parameters, one enum, no output schema), the description plus schema cover the essentials for calling it correctly. The main gaps are error behavior and routing guidance among siblings; without annotations, slightly more disclosure would make it complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 50%: path has a description and example, while method has only an enum with no description. The description adds only the framing that both parameters together identify 'конкретную операцию' (a specific operation), which largely restates what the schema already implies. It does not meaningfully compensate for the undocumented method parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Возвращает' — returns) and a specific resource (parameters, request body, and responses of a specific operation from the OpenAPI schema). The word 'конкретной' (specific) signals a single-operation lookup, which implicitly separates it from the listing sibling elba_list_operations and the execution sibling elba_request, though no sibling is named explicitly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use or when-not-to-use guidance is given. The intended workflow — describing a specific operation discovered via elba_list_operations and before calling elba_request — is only implied by the tool name and sibling set, not stated in the description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• elba_list_organizationsB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the disclosure burden. It does communicate that the tool is a read-only fetch ('Получает') and that results depend on the configured API key. However, it does not mention pagination behavior, failure modes, response shape, or rate-limit/auth consequences, so transparency is only partially addressed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One sentence with no filler. The main action and key scoping condition are front-loaded, and every word contributes to the meaning. It is appropriately concise for such a simple list operation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is adequate for a basic call: it names a clear outcome and the authentication context, and no parameters are required. But it is incomplete in meaningful ways: no parameter guidance, no explicit pagination note, no alternative routing, and no output-schema hints. This is a minimally viable, but not rich, definition.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% and the description does not explain the two parameters (limit and offset) at all. It does not add any meaning beyond the bare parameter names and schema constraints, and because coverage is low, the description was expected to compensate but does not.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource: 'Получает список организаций' (gets a list of organizations) and adds the scope 'доступных по настроенному API-ключу' (available via the configured API key). This makes the purpose clear and distinguishes it from sibling tools like elba_list_operations, which targets operations instead of organizations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context — call this when you need the list of organizations accessible with the current API key — but it gives no explicit guidance about when to choose this tool over elba_list_operations or elba_request, and no exclusions or alternatives are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• elba_requestA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full behavioral burden. It does disclose real behavior: the tool refuses to execute method/path combinations not allowed by the OpenAPI schema, and mutation calls require prior schema review — genuinely useful context beyond the title. However, it's silent on auth requirements, error behavior when validation fails, response format, and it never warns that DELETE/PUT/POST have destructive side effects. For an unannotated tool that can mutate, this is a partial disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, each carrying distinct information: the first states the core behavioral constraint (schema-gated execution), the second gives actionable pre-requisite guidance for mutating calls. No repetition of the title, no filler, and the most important fact is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no annotations and no output schema, the description bears full responsibility for completeness. It covers the two most critical aspects — the validation gate that prevents invalid calls, and the describe-first workflow for mutations. But it leaves out what an agent needs at runtime: response/return format, error and status semantics, authentication prerequisites, and an explicit warning about irreversibile destructive operations. For a generic executor with destructive potential, the agent is left to guess on operational behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 75%: method has a self-describing enum, path has pattern plus a concrete example ('/v1/organizations/{id}/bills'), while body and query have only tautological descriptions ('JSON-тело запроса', 'Query-параметры'). The description adds genuine inter-parameter meaning — method and path must jointly form an allowed operation in the OpenAPI schema, so arbitrary combinations will be rejected. It doesn't elaborate body/query semantics, which is acceptable since those are operation-specific and discoverable via elba_describe_operation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action: 'Выполняет операцию, только если сочетание метода и пути разрешено встроенной OpenAPI-схемой' — a generic request executor gated by the built-in OpenAPI schema. This is a clear verb+resource pairing that distinguishes it from the introspective sibling tools (elba_list_operations, elba_describe_operation), which discover rather than execute. It doesn't explicitly contrast itself against siblings, but the execution role plus schema gate makes the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The second sentence gives explicit when-to-use guidance: 'Для изменяющих операций сначало изучите схему через elba_describe_operation' — naming the exact sibling tool for the mutating case. The rules imply GET-level calls can proceed directly and the tool enforces schema permissibility. It doesn't cover when to use elba_list_organizations or elba_list_operations, but the relevance routing for the riskiest case is explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• elba_list_operationsA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It reveals that this is a read-only listing of methods/routes derived from a built-in OpenAPI schema, which implies no destructive side effects. It could be more explicit about return format or search behavior, but the core behavioral profile is clear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence conveys the action, object, and source with no wasted words. It is appropriately sized for the tool's simplicity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter introspection tool with no output schema, the description and parameter schema are sufficient to call it. The main gap is the lack of explicit routing to sibling tools, but that is more about usage guidance than completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool description does not describe the 'search' parameter beyond naming the action, but the input schema already provides a complete description of it (optional search by route or description). With 100% schema coverage, the baseline 3 applies and no compensation is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('shows') and a concrete resource ('available methods and routes of the Elba Public API'), and it names the source ('built-in OpenAPI schema'). This clearly distinguishes the tool from sibling tools like describe_operation or request.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no guidance on when to use this tool instead of elba_describe_operation, elba_list_organizations, or elba_request. There is no mention of discovering operations first, nor any exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.