Server Quality Checklist

Tool Scores

• backendA
  3.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full responsibility. It mentions actions and port freeing, but does not disclose side effects such as making served apps unavailable when stopping or restarting. This is a significant gap for a management tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that immediately states the resource and actions, with no filler or redundant content. It is optimally concise and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (one param, no output schema), but the description lacks details about the status command's output and the impact of stop/restart on running apps. It is adequate for basic understanding but not fully complete given the absence of annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides 100% coverage with an enum describing the action parameter. The description merely lists the same enum values without adding extra meaning or details about each action's behavior, so it meets the baseline but adds no new value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Manage the local express backend serving the transpiled apps' and enumerates specific actions (status | start | stop | restart). This distinguishes it from siblings like run_app, which is mentioned as a separate tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides explicit guidance: 'run_app starts it automatically; use this for diagnostics or to free the port.' This tells when to use this tool and names the alternative (run_app) for normal start.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generation_rulesA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the burden of behavioral disclosure. The 'rulebook' and 'Read it' phrasing imply a read-only, informational tool, but it doesn't explicitly state that it has no side effects or describe the output format. This is adequate but not fully transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core purpose and followed by a direct usage instruction. Every word earns its place; no fluff or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple documentation tool with no parameters or output schema, the description is nearly complete. It states what the tool provides, what it covers, and when to use it. The only minor gap is the absence of explicit statement that it's a read-only reference, which would fully satisfy contextual completeness in the absence of annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, which warrants a baseline of 4. The description adds context about the content of the rulebook, which is meaningful given no input schema exists.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as 'the canonical rulebook' for writing abap2UI5 apps, specifying the exact areas covered (dispatcher skeleton, view/attribute idioms, binding and event rules). This is specific and distinguishes it from operational siblings like deploy_app and run_app.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The instruction 'Read it once before generating ABAP' provides a clear when-to-use directive. While it doesn't explicitly name alternatives, the context implies this is a prerequisite learning tool, distinct from the other sibling tools that perform actions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• remove_appA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that removal takes effect only after the next build_backend, which is important behavioral context, and that omitting class_name lists deployed apps, indicating a non-destructive mode. This adds meaningful detail beyond the obvious 'remove' action, though it could go further (e.g., error states or permanence).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary action, followed by the critical timing detail and the alternate usage. Every word earns its place, with no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-optional-parameter tool with no output schema, the description covers the main function, the deferred effect, and the list mode. It does not describe return values or error behavior, but given the low complexity and schema richness, this is not a critical gap. The tool's dual nature is fully explained, making it largely complete for an agent to invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already fully describes the class_name parameter, including the 'omit to list' behavior, achieving 100% coverage. The description's mention of 'Without class_name lists the deployed dev apps' largely duplicates the schema text, adding no substantial new meaning. Thus a baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Remove a previously deployed dev app'), specifies the resource location ('from src/zz_dev/'), and distinguishes itself from siblings like deploy_app and build_backend by noting the deferred effect. The alternate listing behavior ('Without class_name lists the deployed dev apps') adds further specificity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool (to remove a previously deployed dev app) and notes the dependency on build_backend for the effect to take place. It does not explicitly mention when not to use it or name alternatives, but the dual listing behavior implies the useful single-argument use case.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scope_ofA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full transparency burden. It discloses the data source (OpenUI5 source JSDoc), the version constraint, and the required checkout, which are meaningful behavioral details beyond the tool's basic purpose.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loads the key purpose, and includes essential caveats without any fluff. Every word contributes value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with a single simple parameter and no output schema, the description explains the core purpose, source, version constraints, and prerequisite. It does not detail return format or error behavior, but these are reasonably inferable and the description is largely complete for its simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The sole parameter 'entities' is fully described in the schema with examples. The description adds no additional semantic detail beyond what the schema already provides, so the baseline score of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool provides an 'authoritative in/out-of-scope verdict for UI5 control entities', which is a specific verb+resource combination. It also adds clarifying context (exists since UI5 <= 1.71, not deprecated) and distinguishes from siblings by its focus on scope determination.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies when to use the tool (when needing a scope verdict for UI5 control entities) and states a prerequisite (OpenUI5 checkout). It does not explicitly compare with sibling tools or list exclusions, but the purpose is distinct enough that an agent can infer appropriate usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• deploy_appA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full disclosure burden. It discloses that it writes to a gitignored dev sandbox, creates a sidecar file, lints using repo config, and requires the class to implement z2ui5_if_app. This gives the agent essential behavioral context for a file-writing operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The three-sentence description is efficient, covering purpose, side effects, prerequisites, and follow-up actions without redundancy. A minor typo 'ababGit' does not detract from overall clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is complete for a deploy tool: it specifies the target directory, the linting step, the required interface, and the post-deploy build/run sequence. It lacks error-handling details but given the tool's straightforward nature, this is sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds value by explaining the lint parameter's default and trade-off, and by stating the interface prerequisite for class_name. This goes beyond the schema's basic parameter types and provides decision-relevant guidance.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Deploy an abap2UI5 app' and details the concrete actions: writing a .clas.abap file with a sidecar into a specific directory and linting. It distinguishes from sibling tools by referencing build_backend and run_app as subsequent steps, making the deploy scope explicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides workflow guidance, explicitly stating to run build_backend and run_app after deployment, and explains the lint:false flag to skip linting. While it doesn't explicitly list exclusions, it clearly defines where this tool fits in the deployment pipeline compared to siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• build_backendA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that the tool stops a running backend first, explains the two modes (incremental vs full) with timing, and highlights the failure condition for incremental builds. It does not cover error handling or side effects beyond stopping the backend, but it provides substantial behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single dense paragraph of three sentences. It front-loads the primary purpose and then provides mode details. While not broken into bullets or sections, it is efficient and every sentence conveys necessary information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of rebuilding a backend, the description covers all essential aspects: the modes, timing, prerequisites (prior full build), trigger conditions for full mode, and the fact that it stops a running backend. There is no output schema, but for a build tool this is acceptable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema only lists 'mode' with enum values and default 'auto.' The description adds rich meaning: it explains what auto does (incremental when prior full build exists), what full does (complete e2e-build), and when to choose each. This goes far beyond the schema's raw enum values.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb-resource pair: 'Rebuild the transpiled Node backend' and connects it to the downstream effect 'so run_app picks up deployed/edited ABAP.' This clearly distinguishes the tool from sibling tools like run_app or deploy_app, which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly explains when to use each mode: auto is incremental when a prior build exists, full is needed for initial builds or framework/port changes. It also mentions when incremental fails (rejects a construct) and suggests fallback actions. It lacks an explicit 'use this instead of X' comparison, but the context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• capabilitiesA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses return behavior: 'Without arguments returns a summary; with `query` returns matching entries' and enumerates status values. It also adds context that the map is verified and every entry names a proving port. No contradictions with annotations (since none exist).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with the core purpose. Each sentence serves a distinct function: what it does, when to use it, and how the arguments affect the response. No filler or redundant phrasing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema exists, but the description explains the two primary return modes (summary vs matching entries) and the status filter. It could provide more detail on entry structure, but for a capability discovery tool this is adequate and the sibling context reinforces its role.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds meaningful behavior: it explains the effect of omitting vs providing `query` and enumerates the `status` options. This goes beyond the schema's bare descriptions and helps the agent understand parameter influence.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb and resource: 'Query what abap2UI5 can express.' The reference to a 'verified capability map (CAPABILITIES.md)' distinguishes this from sibling tools like 'validate_view' or 'build_backend', making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use guidance: 'Call this BEFORE deciding a UI5 feature cannot be built.' This is a clear directive. It doesn't name explicit alternatives or when-not-to-use, but the context is strong enough for an agent to know this is the first stop for capability checks.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_appA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses key behaviors: runs headless, filters benign UI5 noise, returns page errors and failed backend calls, and produces a full-page screenshot. It doesn't mention side effects or required permissions, but for a dev-loop tool this is solid transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with a front-loaded action and clear deliverables. Every clause adds value, and there is no redundant repetition of schema fields.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema exists, but the description enumerates the key return values (boot status, real page errors, failed backend calls, screenshot) and the environment. It doesn't explain the 'loop' in detail, but for a 2-parameter tool this is adequately complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions for both parameters. The description adds extra meaning by showing how class_name is used ('?app_start=<class>'), going beyond the schema. timeout_ms is not elaborated, but the schema covers it sufficiently.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Boot') and identifies the resource ('app class'), environment ('headless in Chromium against the local backend'), and clear outputs ('booted/ok', errors, screenshot). It distinguishes itself from sibling tools like validate_view by positioning as the visual verification step.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for when to use the tool ('The visual verification step of the loop') and notes compatibility with existing ports and specific app classes. It doesn't explicitly name alternatives or exclusion criteria, but the context is strong enough for an agent to choose it over other tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_viewA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the tool reconstructs the view from builder calls or raw XML, runs two gates (property gate and headless render), and uses typed mock models. It also mentions performance ('Seconds'). It does not elaborate on edge cases like how deviations are handled, but the schema covers 'allow'. Overall, it provides substantive behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no filler. The first sentence telegraphs the core purpose and mechanism, the second adds the speed benefit and workflow recommendation. Every clause earns its place, and the structure is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is complete enough for a validation tool with 5 params and no output schema. It explains the two validation gates, the input forms, the performance win, and the next step. It doesn't detail return values or failure messages, but since there's no output schema, that burden falls somewhat on the agent. The lack of explicit mention of the 'allow' parameter is minor, as the schema covers it. Overall, solid.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds meaning by tying parameters to behavior: it references 'z2ui5_cl_ai_xml builder calls' (abap_source), 'raw view XML' (xml), 'UI5 property gate' (min_ui5), and 'renders it headless' (render). It doesn't explicitly explain the 'allow' parameter, but the schema does. The description adds context beyond the schema, hence a 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Fast static validation via ai-view-check', a specific verb+resource combination. It details that it reconstructs the view, runs a UI5 property gate, and renders headless, distinguishing it from build/run tools. The explicit mention of 'BEFORE the build/run loop' and 'then deploy_app once it is clean' clearly separates it from sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit usage context: 'use it after writing ABAP, then deploy_app once it is clean'. It contrasts with the build+boot cycle and names deploy_app as the next step after validation passes. This makes the tool's position in the workflow clear and provides a concrete alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.