Server Quality Checklist

Tool Scores

• sap_read_dumpsA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries the burden. It discloses date defaults behavior ('defaults to today'), which is useful. However, it omits other behavioral aspects like authorization requirements, whether results are limited by max_rows (though schema covers that), or pagination. Since no annotations exist, a score of 3 reflects that the description adds some value but not comprehensive behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, very concise. Front-loaded purpose, then a single key detail about date defaults. No wasted words; each sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read tool with no output schema and no annotations, the description covers the essentials: what it reads and date defaults. However, it does not describe return format or typical usage context, which is a minor gap. Sibling tools exist but no guidance, so completeness is adequate but not rich.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so parameters are already described in the schema. The description does not add much beyond the schema, but it clarifies date format and default behavior, which supplements the schema. Baseline is 3 given high schema coverage and no extra parameter details needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description specifies clear verb+resource: 'Read ST22 ABAP runtime errors (short dumps)'. It distinguishes from siblings by naming ST22 runtime errors specifically, which differs from reading tables, code, jobs, etc. However, it does not explicitly contrast with 'sap_read_table_data' or others, but the ST22 reference is sufficiently unique.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies usage: it's for reading runtime errors, but does not explicitly state when to use versus when not to use alternatives. It mentions date defaults, but no explicit alternatives or exclusions. With several sibling tools for similar operations, the lack of explicit guidance on when to choose this over others is a gap.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sap_read_codeA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. The verb 'Read' implies a non-destructive operation, and 'from the connected system' provides environmental context. However, it does not disclose output format, behavior on invalid objects, or any permissions/limitations beyond what the name itself suggests.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the core action and resource. Every word adds meaning, with no redundancy or filler. It earns a perfect score for conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read tool with only two parameters and no output schema, the description sufficiently covers the main purpose and object types. It lacks explicit mention of return value structure, but the operation 'read source code' inherently implies returning the source. Given the low complexity and full schema coverage, the description is largely complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds modest value by mapping object_type codes (PROG, CLAS, FUGR, INCL, INTF) to the human-readable list (programs, classes, etc.), but it does not provide additional detail on parameter formats or constraints beyond the schema examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb ('Read') and resource ('ABAP source code'), and enumerates the object types (programs, classes, function groups, includes, interfaces). This differentiates it well from sibling tools that read table data, dumps, jobs, or syntax checks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear context ('from the connected system') but does not explicitly state when to use this tool versus alternatives, nor does it mention when not to use it. Usage is implied for reading ABAP source code, but no sibling tool comparisons or exclusions are given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sap_syntax_checkA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full disclosure burden. It explicitly says 'without saving anything' and adds state context with 'release-accurate' and 'connected system's syntax rules.' This conveys non-persistence and system-version sensitivity, though it does not mention permissions or whether any server-side state might be affected.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no filler. Every clause adds meaningful detail: validation, syntax rules, release accuracy, and non-persistence.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter tool with full schema coverage and no output schema, the description is largely complete. It covers the primary function and side effects, though it could explicitly state what the check returns (e.g., errors/warnings) to fully support invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already fully describes both parameters (source and program_name), so the description adds little parameter-level detail. It does provide useful contextual framing about ABAP source and syntax rules, but does not go beyond the schema in terms of parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's verb ('Validate'), resource ('ABAP source code'), and validation target ('the connected system's syntax rules'). This is sufficiently specific and distinct from the sibling read/search tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The use case is implied: validate ABAP source code before or without persisting it. However, the description does not explicitly state when to use this tool versus alternatives, nor does it mention any prerequisites, exclusions, or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sap_read_jobsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds useful context such as status code meanings and date format, but it does not describe the return format, pagination behavior, or any access-related requirements. The term 'runtime' is ambiguous without further explanation of what runtime data is returned or how it is calculated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, consisting of just two sentences. It front-loads the primary purpose ('Read SM37 background jobs') and immediately follows with essential supporting details (status codes and date format). Every word earns its place; there is no superfluous content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is sufficient for a simple read-only tool with no output schema. It provides the necessary value codes and date format to invoke parameters correctly. However, it could be slightly more complete by explaining what 'runtime' and 'scheduling' refer to in the response, but this is a minor gap given the simplicity of the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although the schema already provides 100% parameter coverage, the description adds significant meaning by explaining the status code values (F, A, R, S, P) and the date format (YYYYMMDD). These enrich the schema definitions for 'status' and 'date_from/date_to', guiding the agent to use correct input formats. This goes beyond merely repeating the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Read SM37 background jobs — status, runtime, scheduling.' It uses a specific verb (Read), a specific resource (SM37 background jobs), and outlines the key data aspects (status, runtime, scheduling). This distinguishes it from sibling tools, which all focus on other SAP artifacts like tables, code, and dumps.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool: to read background jobs with status, runtime, and scheduling info. It also includes critical usage details like status code meanings (F=Finished, A=Aborted, etc.) and date format (YYYYMMDD). However, it does not explicitly mention alternatives or when-not-to-use, though sibling tools make the differentiation obvious.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sap_search_tablesA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations present, the description bears the full burden of behavioral disclosure. It transparently explains that the search matches both table names and descriptions, and it implies a read-only operation. However, it does not mention any potential side effects, error behavior, or limitations (e.g., case sensitivity, wildcards), leaving some ambiguity about the exact behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exceptionally concise, containing just two sentences, yet it packs in the action, the resource, a clarifying example, and a usage directive. Every word adds value, and the structure front-loads the core purpose before amplifying with the example.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple search tool with only two parameters, no output schema, and no nesting, the description adequately covers the essential context: what it searches, how it matches, and the primary use case. There are no missing critical details that would impede correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although schema coverage is 100%, the description enriches the 'keyword' parameter with a concrete example ('vendor' finds LFA1/LFB1) that goes beyond the schema's terse 'Search term' description. This helps the agent understand the intended semantic context, though it doesn't discuss max_results beyond what schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: searching the SAP Data Dictionary for tables by keyword, with a specific, illustrative example ('vendor' finds LFA1/LFB1). It distinctly separates this search/discovery function from sibling tools like sap_read_table_structure or sap_read_table_data, making its purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'Use this to discover the right table before querying' provides clear context on when to use this tool (as a preliminary discovery step). While it doesn't explicitly mention alternatives or exclusions, the context is strong enough to infer appropriate usage relative to the sibling tools that handle reading structure/data.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sap_read_where_usedA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explains the non-obvious direction parameter semantics (forward/inverse) which is a critical behavioral trait. While it doesn't mention side effects or errors, the tool appears to be a read-only lookup and the description covers the primary gotchas.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the core concept ('Cross-reference lookup') and immediately clarifies the ambiguous 'direction' parameter with parallel phrasing. Every word earns its place, and the SE84 reference adds immediate recognition for SAP-savvy users.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (four parameters, no output schema), the description effectively explains the core behavior and the key parameter that affects results. It doesn't discuss max_results defaults, but the schema already documents it. The description feels complete for the main use case without being verbose.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although the input schema already describes all parameters (100% coverage), the description adds valuable meaning to the 'direction' parameter by clarifying the forward/inverse distinction with concrete examples. It also provides realistic example values for object_name and object_type in the schema, reducing ambiguity beyond a simple list of allowed values.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as a cross-reference lookup and distinguishes it from typical read operations by explaining the two direction semantics: 'forward' finds users of an object and 'inverse' finds dependencies of a program. It effectively disambiguates the tool from siblings like sap_read_table_data or sap_read_code.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use each direction with explicit examples ('what programs/classes use this object?' vs. 'what does this program use?'). It does not explicitly mention when not to use the tool or name alternatives, though the sibling list contains no directly competing tool, so this is acceptable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sap_read_table_dataA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explains the ABAP operator set (EQ NE GE LE GT LT LIKE IN), single-quote value rule, date format YYYYMMDD, leading zero treatment for vendor/material keys, and gives a full example. This goes well beyond the bare schema. It would be improved by explicitly stating this is a read-only, no side-effect operation, but 'Query rows' implies a safe read.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a tight, focused paragraph. It front-loads the purpose in one short line, then layers in contextual guidance (when to use sibling tools), syntax rules, format tips, and an example. Every sentence contributes to practical use and there is no fluff or repetition of schema information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description offers enough context for a 3-parameter tool with no output schema: the where-clause syntax, data formats, key leading zeros, and the example make it usable. It does not explicitly describe the return shape or output, but the intent of 'query rows' and the max_rows parameter cover the main expectations. Since output schema is absent, describing observed row keys or response structure would push it to 5.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Though schema coverage is 100%, the description adds crucial meaning for the where_clause parameter: allowed operators, quoted values, date formats, leading-zero semantics, and an example. The schema's where_clause description only says 'ABAP-style WHERE condition (see tool description)' and defers to the tool description, so the description absolutely compensates and is essential. It also adds context for table_name with numeric key conventions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a clear, specific verb and resource: 'Query rows from an SAP table with optional WHERE filtering.' It clearly identifies what the tool does and is easily distinguished from siblings like sap_search_tables (finding tables) and sap_read_table_structure (reading table metadata). The mention of using those tools when unsure of table/field names reinforces this differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives an explicit prerequisite: 'Use sap_search_tables / sap_read_table_structure first if unsure of the table or field names.' It names the alternative tools and tells the agent when to consult them, which is strong usage guidance. However, it doesn't enumerate when NOT to use this tool, such as when reading code or dumps, so it is not a full exclusion list.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sap_read_table_structureA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It accurately states the tool reads field definitions without mentioning side effects. It doesn't contradict any annotations (none exist) and gives a clear behavioral summary, though it doesn't mention potential permissions or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, directly stating the purpose and suggesting a usage context. It is structured with a clear action (Get), resource (field definitions of an SAP table), and outcome. No unnecessary details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  It includes a usage hint and the required parameter, making it sufficient for basic selection. It doesn't provide output schema details, but that's not always necessary. It leverages sibling tool names to imply alternatives. Slightly missing explicit outputs, but it's adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Only one parameter, table_name, is required and well-described with an example (EKKO). The schema coverage is 100% and the parameter is clearly named and explained. No ambiguity exists.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool retrieves field definitions of an SAP table or structure, specifying the types of information provided (names, types, lengths, key fields, descriptions). It clearly distinguishes itself from sibling tools like sap_read_table_data (data) and sap_read_code (code) by focusing on structure.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides explicit guidance: 'Call this before querying a table you are not sure about.' This indicates a clear use case. It doesn't explicitly state when not to use it, but the context implies it's for structure discovery, not data retrieval, which is covered by sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.