Server Quality Checklist

Tool Scores

• lookup_articleA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read-only and idempotent behavior. The description adds that the text is verbatim and from the Official Journal, enhancing transparency beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three clear, front-loaded sentences with no redundant information. Every sentence adds value: core function, usage guidance, alternatives.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool and absence of output schema, the description fully explains what the tool returns and the constraints on input parameters. Sufficient for an AI agent to understand usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers 100% of parameters with descriptions and examples. The tool description does not add new parameter information beyond what the schema provides, so baseline score applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool returns verbatim text of a specific EU AI Act article, distinguishing it from sibling tools for keyword search or classification.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly specifies when to use this tool (user names an article number) and provides alternatives for other use cases, with clear references to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_eu_ai_actA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and openWorldHint=true, so the safety and idempotency are clear. The description adds context about returning verbatim provisions and covering all Articles, Recitals, and Annexes, which is useful but does not detail pagination or result limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single paragraph that efficiently covers purpose, when to use, parameter hints, and sibling differentiation. Every sentence serves a clear role, with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given it is a search tool with no output schema, the description sufficiently explains what is returned (provisions matching terms, grounded verbatim). It could mention that results include identifiers (Article/Recital/Annex numbers), but the level of detail is adequate for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%: both 'query' and 'language' have solid descriptions and examples. The description adds value with guidance to 'Use legal/topic terms rather than full questions for best matches', which goes beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it performs full-text keyword search across the entire EU AI Act corpus, returning provisions grounded verbatim. It distinguishes itself from siblings by specifying when to use lookup_article and classify_ai_system, naming them explicitly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises to USE THIS when searching by topic without knowing the Article number, and provides contrasting use cases for siblings (e.g., 'to fetch one known Article use lookup_article; to assess risk tier use classify_ai_system').

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• classify_ai_systemA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint, idempotentHint, openWorldHint. Description adds behavioral context: it returns risk tier, category, Articles grounded verbatim in law, and uses EU AI Act regulation number. No contradictions, but no additional detail on auth or side effects beyond what annotations imply.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences plus usage guidance. Front-loaded with core purpose, efficiently covers usage, returns, and alternatives. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, description adequately explains return values (risk tier, category, Articles). It doesn't specify exact format but is sufficient for agent to understand output. All parameters are documented.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%. Description adds value: for 'description' it explains specificity improves precision and gives examples; for 'language' it lists language codes and explains they are EU official languages; for 'full' it clarifies inclusion of verbatim text.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it classifies AI systems under the EU AI Act, returning risk tier, Annex III category, and binding Articles. It distinguishes from sibling tools (lookup_article, search_eu_ai_act) by specifying its unique purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'USE THIS when the user asks whether an AI system is high-risk or prohibited, what obligations apply, or which Articles bind a specific AI use-case.' It also names alternatives: 'For looking up one known Article use lookup_article; for keyword search use search_eu_ai_act.'

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_compliance_deadlinesA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and idempotentHint=true, indicating safe, idempotent reads. The description adds context about the output including the Digital Omnibus adjustment and fine tiers, enhancing transparency beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with zero waste: first sentence explains what it returns, second sentence provides usage guidance. Information is front-loaded and efficiently presented.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description adequately explains the tool's output (timeline and fine tiers). For a parameterless, read-only tool, this is sufficient for the agent to understand what to expect.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has no parameters, and the description states 'Takes no arguments.' With 100% schema coverage and zero parameters, the baseline is 4; the description adds no further meaning needed beyond confirming no arguments.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool returns the EU AI Act application timeline and penalty/fine tiers under Article 99, specifying it takes no arguments. This clearly distinguishes it from siblings like search_eu_ai_act and classify_ai_system.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance: 'USE THIS when the user asks when the EU AI Act applies, what the key compliance deadlines are, or how large the fines can be.' This clearly tells the agent when to invoke this tool versus alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.