Server Quality Checklist

Tool Scores

• sign_classification_certC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It adds minimal behavioral insight beyond the tool name, only noting the signing algorithm and edition. It fails to disclose side effects, authorization requirements, or idempotency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no wasted words. It is concise and direct, though it sacrifices completeness for brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite the existence of an output schema, the description leaves many gaps: parameter meanings, prerequisites, usage context, and behavioral details. It is insufficient for an agent to fully understand and correctly invoke the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% (no descriptions), and the description does not explain any of the 7 parameters (e.g., entity_name, annex_i_score). The agent has no semantic understanding of what each parameter represents from the description alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Generate' and the specific resource 'CRA classification certificate', including the signing algorithm (HMAC-SHA256) and edition (Pro/Enterprise). This differentiates it from sibling tools like classify_product and generate_doc_template.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool over siblings or in what context. There is no mention of prerequisites, conditions, or exclusions, leaving the agent without direction on appropriate usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• classify_productC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must fully disclose behavioral traits. It mentions return values (suggested class, conformity path, fines exposure, next steps) but does not state whether the tool is read-only, whether it requires authentication, or any side effects. The lack of detail on behavior limits transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very short (two sentences) and front-loads the purpose. It is efficient, but the returns section could be more structured. Overall, it is concise without unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 6 parameters, 0% schema descriptions, no annotations, and an output schema exists, the description is incomplete. It does not explain what 'CRA class hierarchy' is, how the parameters influence classification, or any usage context. The output schema existence slightly reduces the burden, but the description still lacks crucial information for proper invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, meaning no parameter descriptions are provided. The tool description also does not explain the meaning of any parameters beyond their names. For example, 'api_key' and 'product_description' are self-explanatory, but 'intended_for_consumer_use' and similar booleans lack context. The description fails to add value over the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Classify' and the resource 'product with digital elements (PDE)' into 'CRA class hierarchy'. It is distinct from sibling tools like audit_essential_requirements, generate_doc_template, and sign_classification_cert, which focus on auditing, document generation, and certification.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives, no prerequisites, and no exclusions. It only states what it does, leaving the agent to infer usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_doc_templateC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so description carries full burden. It does not disclose authentication needs (api_key), output format, or whether the operation is safe or destructive. Minimal behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence with 10 words, no redundancy. However, it is too brief for a 2-parameter tool with no schema descriptions, sacrificing completeness for brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Output schema exists but is not referenced. Parameters lack explanation, and usage context (e.g., what 'skeleton' implies) is missing. Incomplete for an agent to use effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%; description only explains 'class' indirectly via 'for a given class'. The api_key parameter is entirely unexplained. Defaults exist but no semantic cues.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description uses specific verb 'generate' and resource 'Annex VIII technical documentation skeleton', clearly distinguishing from sibling tools like audit, classify, or sign.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives. While siblings differ in actions, the description does not provide context or exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• audit_essential_requirementsA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It states the action (audit) and input format but omits side effects, idempotency, or whether it modifies state. 'Audit' suggests read-only, but not explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, zero waste. Front-loaded with purpose, followed by parameter format hint. Each sentence serves a clear function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Output schema exists but unseen. Description covers purpose and key parameter format. Missing usage context, prerequisites, or error scenarios. Adequate for a straightforward audit but not rich.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% coverage (only titles). Description adds meaning to 'requirements_satisfied_csv' by specifying format (comma-separated IDs) and example. However, 'api_key' and 'product_name' remain minimally described. Compensates significantly but not fully.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Verb 'audit' combined with specific resource '15 Annex I + II essential requirements' clearly states what the tool does. Differentiates from siblings (classify, generate doc, sign cert) which have distinct purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies usage for auditing compliance against essential requirements, but no explicit when-to-use, when-not-to-use, or alternatives. The examples suggest typical input format but no contextual guidance relative to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.