Server Quality Checklist

Tool Scores

• explain_obligationA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false, so the bar is lower. The description adds useful context beyond those annotations: the tool works offline and returns obligations with Article citations. There is no contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exactly two short sentences, front-loaded with the action and scope. The one-word sentence 'Offline.' is concise and communicates an important operational trait without waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given a simple 2-enum-param tool with complete schema and read-only annotations, the description adequately describes the output shape (list with Article citations) and the offline nature. Return ordering/format are unspecified, but no output schema exists, so this is a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with rich descriptions for 'role' and 'risk' including the classify-tool pipeline. The tool description itself adds no parameter-level meaning, so the baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'List the EU AI Act obligations for a role + risk tier', using the specific verb 'List' and the resource 'EU AI Act obligations'. It further adds 'each with its Article citation' and 'Offline', making the tool's function and distinctive scope clear relative to sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no explicit when-to-use guidance or alternatives. 'Offline' is an operational note, not a usage rule. The important prerequisite 'call classify first and pass its risk value' lives only in the schema's parameter description, so the description itself does not carry usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_recordA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false, so the safety profile is covered. The description adds behavioral context beyond annotations by stating the tool is 'Online' and performs a network read, and it restricts to 'published' records, which gives helpful operational expectations. There is no contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences, front-loaded with the core purpose and followed by a useful behavioral note. No unnecessary words or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter read-only tool with strong annotations and full schema coverage, the description is nearly complete. It communicates the resource, the input, and the network behavior. It does not detail the return format, but that is not strictly necessary given the simplicity and the absence of an output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema documentation covers 100% of the parameter meaning, with detailed description for 'slug' including format and example. The main description merely repeats 'by org slug', adding no additional semantic value beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Fetch') and a specific resource ('published Legalithm Trust Center compliance record') with the required input ('by org slug'). This clearly distinguishes it from sibling tools like classify or generate_disclosure, which imply different operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides context ('Online (reads the public API)') and implies this is for retrieving published records, but does not explicitly state when to use this tool versus alternatives or when not to use it. No exclusions or alternative tool names are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• classifyA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false. The description adds valuable behavioral context: 'Offline' (no network calls), 'checked against Regulation (EU) 2024/1689' (authoritative source), 'not legal advice' (limitation disclaimer), and it describes the output as 'risk tier + cited rationale'. This goes beyond the annotations without contradicting them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence starting with the verb and resource, followed by two terse context clauses. Every element earns its place with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 4-parameter tool with no output schema, the description covers the return contract (risk tier + rationale), the legal basis, offline behavior, and disclaimer. Combined with the comprehensive schema descriptions, it is largely complete; the only gap is explicit sibling differentiation, which is covered under usage guidelines.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with each parameter having thorough meaning including enum semantics and special guidance (e.g., domain breadth, audience selection, use_case phrasing). The tool description itself adds no further parameter information, but the schema carries the full burden, so the baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'classify' with a clear resource ('an AI use case under the EU AI Act') and outcome ('risk tier + cited rationale'). It is immediately clear what the tool does and is distinct from sibling tools like explain_obligation or generate_disclosure.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage whenever classification under the EU AI Act is needed, but does not explicitly state when to use this tool instead of siblings like explain_obligation or check_record. The 'Offline' and 'checked against Regulation' context hints at behavior, but there are no exclusions or alternative references, so guidance is implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_disclosureA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false, so the read-only nature is known. The description adds the 'Offline' trait, which is useful context, but it does not describe the output format, any prerequisites, or other behavioral details. With annotations covering safety, this is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that conveys the core action, the categories, the languages, and a key behavioral trait ('Offline'). Every element earns its place with no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a simple tool with only two parameters, both fully described in the schema. The description succinctly states the tool's purpose and adds the 'Offline' detail. It does not explicitly mention the return format, but for a 'snippet' generator this is largely implied and not a significant gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and both parameters (locale and scenario) have enums with detailed descriptions explaining the relevant Article 50 duties. The tool description itself adds no parameter-specific meaning beyond the schema, so a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Generate') and specifies the exact resource ('Article 50 transparency disclosure snippet'), plus the relevant categories (chatbot, genai-content, deepfake, emotion) and languages (EN/DE). This clearly distinguishes it from sibling tools like classify or explain_obligation, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies its use case by naming the disclosure types and languages, and the parameter schema further clarifies when each scenario applies. However, it does not explicitly contrast with sibling tools (e.g., explain_obligation) or state when not to use it, so it lacks explicit exclusion guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.