Server Quality Checklist

Tool Scores

• eu_evidenceA
  3.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate read-only, idempotent, non-destructive behavior. The description adds transparency by listing possible errors (`out_of_scope`, `corpus_error`), which helps agents anticipate failure modes. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (two sentences) and front-loaded with the core purpose. It includes optional narrowing and error information without unnecessary detail. Could benefit from separating errors into a dedicated section.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of an output schema, the description explains what the tool retrieves (evidence artifacts with auditor questions) but does not describe the structure of the response. It mentions errors, which adds completeness. Annotations fill some gaps, but more detail on return format would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 50% (only 'article' has a description). The tool description adds meaning: it explains that the tool returns evidence 'for which article' and that narrowing to one article is optional. For 'regulation', it implies selection of regulation but lacks explicit parameter semantics beyond the enum.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that this tool returns evidence artifacts required by regulations, optionally filtered by article. It distinguishes from siblings by specifying audit-related content and mentioning possible errors. However, it could be more explicit about listing evidence items.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies that the tool is used to retrieve evidence artifacts for a regulation, but it does not explicitly state when to use it versus siblings like eu_article or eu_search. No exclusion criteria or alternatives are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• eu_articleA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false, indicating a safe read operation. The description adds value by listing specific error scenarios (out_of_scope, missing_arg, not_found) and clarifying that the tool returns title and chapter in addition to the article text.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that clearly states the tool's purpose and return values, followed by a list of possible errors. It is concise and front-loaded with essential information, with no unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple retrieval tool with two parameters and no output schema, the description covers purpose, return content (full text, title, chapter), and error scenarios. It could be improved by describing the output format or structure, but the given information is sufficient for an agent to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 50% (article_number has a description, regulation only has an enum). The description reiterates that parameters are regulation identifier and article number but does not add new semantic details beyond what the schema provides. No explanation of the regulation enum values.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves the full verbatim text of an article along with its title and chapter, given a regulation and article number. It distinguishes itself from sibling tools like eu_search by specifying it returns the exact text, not search results.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context through error messages (e.g., 'out_of_scope' for regulations not in the 6 ICP list), but it does not explicitly state when to use this tool versus alternatives like eu_search or eu_compare. No direct guidance on when-not-to-use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• eu_searchA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds value by stating it returns snippets verbatim with highlighting and lists specific errors (empty_query, corpus_error), which are not in annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no wasted words. The first sentence front-loads the core purpose, the second covers errors. Perfectly concise for a search tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given full schema coverage and adequate annotations, the description covers purpose, error handling, and snippet behavior. It could mention ordering or default pagination, but is sufficiently complete for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds context beyond schema by clarifying the query type ('fraza w języku naturalnym'), optional regulations subset, and error cases (empty_query, corpus_error), enhancing semantic understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states full-text search (FTS5) in EU regulation articles, specifies the exact regulations covered, and mentions snippet highlighting. It distinguishes from sibling tools like eu_article by focusing on search rather than retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implicitly indicates use for finding articles by content, but lacks explicit guidance on when to use vs. alternatives (e.g., eu_article for known article IDs). No 'when not to use' or alternative references.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• eu_check_applicabilityA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare `readOnlyHint=true`, `destructiveHint=false`, `idempotentHint=true`, indicating safe, non-destructive, repeatable behavior. The description adds value by clarifying that results include confidence levels and article bases, emphasizes the non-binding expert nature, and documents specific error types. This contextualizes the tool's output and limitations beyond what annotations provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (three sentences) and well-structured: first sentence states purpose and scope, second describes output, third adds caveat and errors. Each sentence is necessary and provides distinct information with no redundancy. The Polish language is appropriate for the tool's likely audience.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 parameters, no output schema, rich annotations), the description covers all essential aspects: what it does, what it returns, its non-binding nature, and error cases. The output description ('rules, confidence level, article basis') is sufficient without an explicit output schema. The tool's role among siblings is clear from the purpose.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 50% (sector is well-defined with enum, subsector has a brief description). The description adds meaning by mentioning '6 EU regulations' as the context for sector selection, and by stating the output includes applicability rules and confidence. It also notes errors. This enriches understanding beyond the schema alone, though it does not detail parameter formats or constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly specifies the tool's purpose: checking which of 6 EU regulations apply to a given sector and optional subsector. It details output (applicability rules with confidence level and article basis) and distinguishes it from sibling tools (eu_article, eu_compare, eu_evidence, eu_search) by focusing on regulation applicability rather than article details, comparisons, evidence, or general search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides partial guidance: it states the tool returns an expert hint, not a binding legal assessment, and lists possible errors (`missing_arg`, `corpus_error`). However, it does not explicitly instruct when to use this tool versus its siblings (e.g., when to prefer `eu_article` or `eu_search`). The usage context is implied but not sharply defined.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• eu_compareA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read-only, idempotent behavior. Description adds that it returns verbatim snippets and lists possible errors (empty_query, corpus_error), adding value beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences plus an example and error list. Every sentence adds value, front-loaded with purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema, but description adequately explains return format (best matching article snippet) and errors. Given parameter count and sibling tools, it is complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%. Description explains that 'regulations' parameter defaults to all six if omitted and requires min 2, and 'query' is the issue to compare. This adds critical usage context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it compares an issue across multiple regulations and returns the best matching article snippet. It distinguishes from siblings like eu_article and eu_search by focusing on comparative analysis.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description specifies when to use it (comparing same issue across regulations) and provides an example. It does not explicitly exclude alternatives, but the sibling context makes it clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.