Server Quality Checklist

Tool Scores

• install_snippetA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description significantly extends beyond annotations. It discloses idempotency, safe handling of non-HTML files (no corruption), behavior when </head> is missing (returns instructions instead of modifying), and the need for deployment verification. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (5-6 sentences) with a clear front-loaded main action. Every sentence adds essential information, with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description fully explains tool behavior: what it does, idempotency, handling of edge cases (non-HTML files), and follow-up action (verify_snippet). Complete for its complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline 3. The description adds minimal extra meaning beyond schema defaults (e.g., lang defaults to page language, appId from env). No substantial additional semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool inserts a LexVibe cookie-banner and hosted-policies snippet into an HTML file before </head>. It distinguishes itself from sibling tools like verify_snippet, scan_project, etc., which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context (installing snippet) and mentions post-installation verification with verify_snippet, but does not explicitly list when to use or not use this tool versus alternatives. Still clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_ai_actA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds behavioral context beyond annotations: read-only, no signup required, requires network access to LexVibe API, error returns {error} with isError. It also explains parameter defaults default to the safest 'not applicable' value, which aligns with destructiveHint=false. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is lengthy but well-structured: starts with core purpose, then parameter grouping, return format, and usage notes. It front-loads the main action and avoids redundancy. Every sentence earns its place, though brevity could be improved.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of 14 boolean parameters and no output schema, the description adequately explains the return structure ({level, headline, explanation, obligations}) and each parameter's impact. It also covers error handling and network requirements, making the tool usable without additional context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so parameter descriptions exist. The tool description adds value by grouping parameters into three categories (transparency triggers, prohibited practices, high-risk triggers) and mapping to scan_project's outputs (e.g., usesGenerativeAI maps to usesAI and generatesContent). This enhances understanding beyond the schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool classifies a product's risk level under the EU AI Act, returning risk levels (minimal, limited, high, prohibited) and obligations with deadlines. It also differentiates from siblings like scan_project and make_compliant, noting mapping of scan_project's outputs to parameters and that make_compliant includes a basic version of this check.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage guidance: use when the user asks whether the EU AI Act applies to them or after adding an AI feature. It also explains when not to use (leave unknowns at defaults) and mentions an alternative (make_compliant already includes a basic version of this check).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_websiteA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond annotations indicating read-only and non-destructive, the description explains server-side fetching, detection of tracking/processing, and the output structure. It also notes rate-limit behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Front-loaded with key benefits, then details on functionality and usage. Every sentence adds value, no redundancy. Structure is logical and concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given simple input (one parameter) and no output schema, the description fully explains the return structure and behavior. It also references sibling tools and usage context, making it self-contained.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema already fully describes the 'url' parameter with examples and requirement. The tool description adds no new parameter information beyond restating that it's a public URL, so baseline 3 is appropriate given 100% schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs a compliance check of a deployed website by URL, distinguishing itself from sibling scan_project which reads local source. It specifies the output format and the context of use.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says when to use: after deploying or for sites without source code. Mentions rate-limiting and that it complements scan_project, guiding the agent on alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• claim_appA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Describes the claim-link expiration (30 minutes), the need for user confirmation, and the follow-up call to get_claim_status. This adds significant value beyond annotations by explaining the mutation's deferred nature.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Around 150 words, covering all key points. Could be slightly more concise, but information density is good and front-loaded with core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 7-parameter tool with nested objects and no output schema, the description covers the workflow, return values, and motivation. Lacks error cases or rate limits, but openWorldHint allows flexibility.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with good parameter descriptions, but the description adds a crucial constraint: 'Requires url or appName (errors otherwise)', which is not in the schema's required list. This provides decision guidance beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it creates a real LexVibe app, replaces a placeholder, and returns a claim URL with code and expiry. It distinguishes from sibling tools like get_claim_status by describing the post-confirmation step.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use this whenever no real LEXVIBE_APP_ID is configured' and requires either `url` or `appName`. Provides clear context for when to invoke this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_complianceA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond annotations (readOnlyHint=true, destructiveHint=false), description adds key behaviors: never modifies files, errors on invalid dir/path, and explains return value structure including 'agentPrompt' for dev agent. No contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Front-loaded with purpose and usage; every sentence adds value. Slightly long but dense with information, no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema, but description fully explains return object fields and 'ready' condition. Covers error conditions, iterative workflow, and required parameters. Complete for a tool of this complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions. Description adds context like 'markets are a required human fact' and 'overrides the derived one' for entity/appName, enriching meaning beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states verb ('Read-only compliance readiness check') and resource ('project'), with specific use case after adding integrations. Distinguishes itself from sibling tools like make_compliant by indicating it's a precursor check.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly tells when to use ('after adding any SDK, analytics, payment, auth or AI integration') and when not to (implies not for full generation, as call before make_compliant). Provides iterative re-run strategy until 'ready' is true.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• make_compliantA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses detailed behavior including: writes to <dir>/legal, installs cookie banner only for web apps, skips for native apps, gives manual instructions for JSX layouts, aborts if dir doesn't exist. It also explains the return value structure and the conditional re-run logic. This goes well beyond the annotations (readOnlyHint: false, openWorldHint: true) and provides complete transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single paragraph that effectively front-loads the main purpose and then provides detailed caveats and return info. For the complexity involved, it is reasonably concise, though it could potentially be broken into bullet points for easier scanning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema is provided, but the description fully documents the return value structure and explains how to interpret the fields (e.g., agentPrompt, missingFacts). It covers all major behaviors (document generation, banner installation, AI Act classification, conditional execution) and edge cases (JSX, native apps, missing dir). Given the tool's complexity, this is complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds value by explaining defaults (appId from env var, markets to eu), constraints, and the need to confirm the default markets with the user. It also clarifies when parameters are optional (e.g., appName derived from repo). This extra context justifies a score of 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description begins with 'One-step legal compliance' and enumerates specific actions (scan, generate documents, install cookie banner, classify AI Act risk). It distinguishes from sibling check_compliance by explicitly stating when to use each, making the purpose crystal clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use ('when user asks to make app legally compliant, GDPR-ready, or to add privacy policy or cookie banner') and when to use check_compliance instead ('for a read-only report'). Also explains abort conditions and re-run triggers, providing comprehensive usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scan_projectA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false. The description adds further context: 'Never modifies files', error conditions (dir does not exist or not a project root), and details about return structure. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with key action and purpose. It is slightly verbose but each sentence adds meaningful information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description fully explains the return format and fields. It also covers prerequisites, post-processing steps, and siblings. Complete for a single-parameter tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a single param 'dir' described. Description adds value by explaining error behavior if dir is invalid and suggesting using absolute paths.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs a read-only scan of a local project for legal compliance detection. It specifies the resource (local project) and action (scan), and distinguishes from sibling tool check_website for deployed sites.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs to 'Run it first' and describes how to pass results to generate_policies and claim_app. Also provides alternative: use check_website for deployed sites without code access.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_claim_statusA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint=true, and the description reinforces this by explaining the tool only reads status. It adds critical behavioral context like expiration timing, condition for not installing snippet, and error handling for unknown codes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Despite being lengthy, every sentence provides essential information. The structure flows logically from status outcomes to action items, and there is no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description fully compensates by describing the return structure and behavior for each case. It covers the lifecycle from pending to claimed, including edge cases like expired and errors.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already describes the 'code' parameter, but the description adds the crucial clarification that it must be the `code` field from claim_app, not the URL. This adds value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool checks claim status and specifies the different return states and their implications. It distinguishes itself from siblings like claim_app (which creates the claim) by focusing on status checking.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit instructions on how to handle each status: wait and retry for pending, recreate for expired, and detailed steps for claimed including not installing snippet if published is false. This gives clear when-to-use and follow-up actions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_snippetA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, consistent with the read-only nature. The description adds detail on behavior: fetches the public URL, looks for widget marker, validates URL (rejects private/localhost). It also explains the exact return format and meanings. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise: 4 sentences covering purpose, usage context, constraints, and return values. It is front-loaded with the main action. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple verification tool with one parameter and no output schema, the description is complete. It covers what, why, when, constraints, and result interpretation. Annotations handle safety, so the description meets all needs.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already has a good description for the url parameter (100% coverage). The tool description adds further constraint: 'Only public http(s) hosts are allowed — localhost, private-network and reserved addresses are rejected,' which is not in the schema. This adds meaningful context beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool verifies that the LexVibe cookie-banner snippet is live on a deployed site. It distinguishes from the sibling tool install_snippet by noting that install_snippet edits local files while this confirms production. The verb 'verify' and specific resource 'snippet live on deployed site' are precise.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says when to use ('Run it after deploying') and gives constraints ('Only public http(s) hosts allowed'). It provides alternative actions if the snippet is missing ('check that the deploy included the change, or re-run install_snippet and deploy again'). This is comprehensive guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_policiesA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark readOnlyHint=true and destructiveHint=false. The description adds key details: documents are returned only, never persisted server-side, and template source means not yet personalized. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single focused paragraph front-loaded with purpose. Every sentence adds value: purpose, prerequisite, behavior, return format, and distinction from sibling. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 6 parameters, nested objects, and no output schema, the description covers all critical aspects: return structure (documents array with fields), special values (template source), prerequisite (scan_project), and network requirement. It compensates fully for the missing output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, each parameter has a description. The description adds contextual meaning: explains relationship between 'answers' and 'scan_project', clarifies 'global' market is not a union, and describes the return format. This adds significant value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool generates legal documents (privacy policy, terms of service, AI disclosure) with localization and market-specific frameworks. It distinguishes from siblings like 'make_compliant' and 'scan_project', providing a specific verb and resource.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says to 'Run scan_project first and pass its suggestedAnswers as `answers`', indicates the tool requires network access to an API, and clarifies that persistence is handled by another tool ('make_compliant'). This provides clear when-to-use and when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.