Server Quality Checklist

Tool Scores

• explain_abap_ruleA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare read-only and idempotent; description adds domain boundary (abaplint only) and states it does not run analysis. Useful context beyond annotations, though not exhaustive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with purpose, followed by usage and scope limitation, ending with a concrete example. Every sentence is informative.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter read-only tool with output schema, the description covers purpose, usage, and limitations sufficiently. No need to detail return values due to output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers the single parameter with examples; description's example is redundant. No additional semantic meaning beyond schema, so baseline 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States it explains one abaplint rule in depth, listing included elements (title, description, rationale, tags, docs URL, examples). Distinguishes from siblings by referencing lint_abap/check_cloud_readiness findings. Clear verb+resource+scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use this when a lint_abap or check_cloud_readiness finding needs justification' and notes it does not run analysis and excludes SAP ATC documentation. Provides clear when/when-not guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• format_abapA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and idempotentHint=true, so the safety profile is covered. The description adds useful context beyond that: it does not change any logic, fails cleanly on unparseable source, and is an offline equivalent of Pretty Printer in ADT/SE80. This extra behavioral detail goes beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each earning its place: purpose, usage guidance with exclusions, and a concrete example. It is front-loaded with the core purpose and contains no filler or redundant phrases.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a moderately simple tool with an output schema and good annotations, the description is complete. It covers what the tool does, when to use it, what it deliberately does not do, failure behavior, and includes an example. The sibling tools are clearly distinguishable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides rich descriptions for both parameters (source and filename), including filename inference behavior. The description's example demonstrates the 'source' parameter but does not add new semantics for 'filename' beyond what the schema already states. With 100% schema coverage, the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb+resource: 'Pretty-print one ABAP source' with specific details about normalizing keyword casing and indentation. It also differentiates from siblings by noting it is the offline equivalent of ADT/SE80 Pretty Printer and explicitly excludes CDS views and behavior definitions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit usage guidance is provided: 'Use this when generated or hand-written ABAP has inconsistent casing/indentation and you want it normalized before review or commit.' It also lists exclusions (CDS views, behavior definitions, logic changes) and gives a concrete example, making the intended use very clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scaffold_rap_boA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint=true and idempotentHint=true, and the description does not contradict them. The description adds useful behavioral context: round-trip validation via abaplint, that behavior/service definitions are canonical templates not deeply parsed, and ADT activation as the final check. This goes beyond the annotations to set expectations on quality and limitations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but densely informative—every sentence contributes: artifact list, usage, exclusions, validation, example. It is well-structured with a clear opening scope, conditional notes, and a concrete example. Not perfectly concise but appropriately detailed for a complex code-generation tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool generates many artifacts and has an output schema; the description covers generated artifacts, potential caveats (validation depth, binding/table not created), next steps, and an example. With 7 parameters and an output schema, this description is enough for an agent to select and invoke correctly, and to interpret the result.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 100% parameter description coverage, so baseline is 3. The description adds value through a concrete example that illustrates parameter usage and relationships (entityName, sqlTable, keyField, fields, draft), and clarifies defaults (e.g., prefix Z, admin fields auto-added). This supplements the schema without redundancy.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description uses a specific verb ('Generate') and details the exact resource ('the complete, canonical RAP managed business-object stack for one root entity') with an explicit list of artifacts. It clearly distinguishes from sibling tools (lint/format/check) and states its scope in the first sentence.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit usage guidance is provided: 'Use this when starting a new RAP business object in ABAP Cloud or S/4HANA' and contrasts with hand-writing. It also states exclusions ('does not create the table or the service binding', 'single-entity BOs: model compositions (parent-child) yourself'), giving clear when-to-use and when-not-to-use signals.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compare_abapA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnlyHint and idempotentHint, and the description adds meaningful behavioral context beyond that: it explains matching by content (moved-but-unchanged code is not noise), explicitly notes that it does not connect to any SAP system, and highlights a limitation ('behavior can change while every number improves'). This is valuable disclosure beyond the structured annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded: it starts with the main functionality, then usage guidance, then exclusions, ending with an illustrative example. Every sentence adds necessary information without repetition or fluff, and the length is justified by the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers purpose, usage context, limitations, and provides an example. Since the tool has an output schema, the description does not need to detail return values. The mention of reuse of check_cloud_readiness logic and the explicit exclusions round out the context, making the description complete for an agent to select and use the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the input schema already documents all parameters with descriptions. The description adds an example call that illustrates the structure of 'before' and 'after' arrays, but it does not provide any additional parameter semantics that are not already in the schema. Therefore, a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Compare a BEFORE and an AFTER version of ABAP source' and enumerates the exact kinds of results (lint findings, cloud-blocker/score/grade movement, structural changes). It explicitly differentiates itself from a textual diff tool ('use git diff to see the edits'), making it clear what this tool does and does not do.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage contexts: 'Use this when reviewing a refactor, a modernization step or an AI-generated rewrite' and gives exclusions: 'It is not a textual diff tool' and 'cannot judge functional equivalence.' It also mentions that it uses the same diff as check_cloud_readiness, linking to a sibling tool without confusion.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_abap_outlineA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and idempotentHint, and the description adds critical behavioral context: it does not return method bodies, does not analyze code quality, and produces empty outlines for CDS/behavior files. It also explains the optional mermaid output. This goes well beyond the annotation bare minimum.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is longer than a one-liner but each sentence earns its place: purpose, usage scenario, mermaid option, limitations, and an example. It's structured with a clear progression and no fluff, though it could arguably be trimmed slightly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description fully covers the tool's scope, constraints (no method bodies, empty CDS), usage context, and provides an example. With an output schema present, no return-value detail is required. This is a complete package for an AI agent to decide when and how to invoke it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (both params have descriptions), so baseline is 3. The description adds value by providing a concrete usage example and explaining the mermaid flag's purpose ('for documentation visuals'). The example clarifies the expected input shape, which is helpful beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Return the structural outline') and enumerates exactly what it covers (classes with methods/visibility/attributes/interfaces/inheritance, interfaces, FORM routines). It distinguishes itself from siblings by contrasting with lint_abap, making it clear this is a structural navigation tool, not an analysis tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It directly states when to use it ('when navigating a large class or legacy program to decide which part to read or edit next') and positions it as a 'cheap first call.' It explicitly names the alternative for code quality (lint_abap) and calls out CDS/behavior-definition files as yielding empty outlines, so the agent knows when not to rely on it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_cloud_readinessA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With annotations already declaring readOnlyHint=idempotentHint=true, the description adds significant behavioral context: it parses twice at different baseline versions, diffs the results, and separates existing broken code from migration blockers. It explicitly discloses limitations (static, parser-level, no API release checks, no SAP connection, 'ready' is not a certification), going well beyond annotation-provided safety hints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but every sentence carries information: method, output categories, grade, use cases, limitations, and an example. It is front-loaded with the core purpose and methodology, then expands with usage guidance and exclusions. No filler words or redundant restatements of the tool name.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (dual-pass parsing, diffing, categories, score, grade, verdict) and the presence of an output schema, the description is remarkably complete. It covers what, when, how, and limitations, plus includes an example. All essential decision-making information is present, and the output schema covers exact return structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 100% description coverage for both parameters, so the baseline is 3. The description adds meaning by explaining how baselineVersion functions ('used to separate broken-anyway code from cloud blockers') and provides a concrete example call showing parameter usage. It also enriches the files parameter by implying the source text and optional filename semantics, though the schema already covers this well.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb 'Assess' and clearly defines the resource and goal: 'how far ABAP source is from ABAP Cloud (Clean Core tier 1)'. It also distinguishes itself from sibling tools by detailing its parser-level, static approach and explicitly separating cloud blockers from broken code.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit use cases: 'when someone asks is this code cloud-ready... before porting classic ABAP into an ABAP Cloud environment, or for a graded tech-debt assessment of an abapGit export.' It also names an alternative (check_released_api) and explicitly states what the tool does NOT do (released-API checks, SAP connectivity, certification), clarifying when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_released_apiA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and idempotentHint=true; the description adds critical context: offline snapshot, snapshot date, non-authoritative behavior, and conservative interpretation of absent results. This goes beyond annotations without contradicting them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single dense paragraph that front-loads the purpose, then details use cases, limitations, and an example. Every sentence contributes substantive information with no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity, the description covers all essential aspects: what it does, status categories, successor hints, offline/snapshot limitations, authoritative alternative, and example invocation. An output schema exists to handle return-value details, which are sufficiently referenced.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides 100% coverage of parameter semantics, so baseline is 3. The description adds a concrete example (check_released_api with mixed string/object array) and clarifies per-object output, which slightly improves usability beyond schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly specifies the verb ('look up'), the resource ('ABAP repository objects'), and the result (released/deprecated/not-released status per object). It explicitly distinguishes this tool from check_cloud_readiness, making sibling differentiation strong.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly states when to use the tool ('when you need to know if your code may reference a given object in ABAP Cloud'), names check_cloud_readiness as the alternative that delegates system ATC, and provides exclusions (does not connect to SAP, not authoritative). This is model usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lint_abapA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare read-only and idempotent, but the description adds significant context: fully offline, no ATC, cannot judge undefined objects unless provided, and preset behavior differences. It also explains focus tags and rule overrides. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but well-structured: purpose first, then when to use, limitations, preset semantics, focus/overrides, alternative, and an example. Every sentence conveys distinct information—no filler or repetition. The length is justified by the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's 5 parameters, nested objects, enums, and output schema, the description covers all critical aspects: input types, limits (via schema), return value contents, preset meanings, and the boundary case (check_cloud_readiness). It even includes an example, making it self-contained for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (all five parameters have descriptions), but the description adds meaningful semantics beyond the schema: explains why the default preset skips whole-program checks, how focus packs work with rule overrides, how filename omission triggers type inference, and includes a concrete invocation example that maps to the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with a specific verb and resource: 'Run abaplint static analysis over ABAP, CDS or behavior-definition sources' and clearly states the output (structured findings). It distinguishes from siblings by naming check_cloud_readiness as the alternative for migration verdicts, and the detail about offline operation further differentiates it from system-connected tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit use case: 'Use this when you have written or modified ABAP code and want style and correctness feedback before it goes anywhere near a system.' It provides exclusions (does not connect to SAP system, does not run ATC) and presets with scenario guidance (style for isolated snippets, full when all dependencies provided). It names an alternative tool for a different intent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_abap_rulesA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and idempotentHint=true, but the description adds further transparency: it is a 'read-only catalog' that does not run analysis or alter configuration. It also describes the optional filtering behavior (free-text query or tag) and the example call. This goes beyond the annotations without contradicting them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is tightly packed: first the action, then when-to-use, then safety/exclusion, then an example. Every sentence earns its place, with no fluff or repetition of schema field names. It is front-loaded and concise for the level of detail provided.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has an output schema (though not shown), the description already mentions return fields. Both parameters are covered, usage guidance is explicit, and safety behavior is disclosed. For a read-only listing tool, this is fully complete—an agent can confidently select and invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description enriches param understanding by explaining 'tag' as an abaplint tag with examples and 'query' as a case-insensitive substring matched against specific fields. It also provides a concrete invocation example, which is more than the schema alone. This meaningfully aids correct parameter usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with the specific verb+resource: 'List the abaplint rules this server can check'. It clearly distinguishes from siblings like lint_abap (which runs checks) and explain_abap_rule (which explains a single rule). It also enumerates the return fields (key, title, one-line description, tags, documentation URL), leaving no ambiguity about what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use: 'Use this when deciding which rules to enable or override in lint_abap, or to discover what a Clean-ABAP-style check exists for.' It also says what it does not do: 'It does not run any analysis and does not change configuration', which clarifies when not to use it (e.g., for actually running linting). Sibling tool lint_abap is named as the alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.