Server Quality Checklist

Tool Scores

• sap_get_pageC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must fully disclose behavioral traits DNA. It only states the action and output, but omits critical details such as authentication requirements, error handling, potential side effects (e.g., session state changes), or rate limits. It also fails to note whether it modifies data or is purely read-only, which is significant given the WebGUI context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one sentence, concise, and front-loads the primary action. However, it omits essential details that could be included without bloat, so it's efficient but potentially under-specified.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (interfacing with SAP WebGUI), the description is insufficient. There is no output schema and no annotations; the description does not explain return format, potential errors, session dependencies, or navigation side effects. Sibling tools suggest a need for orchestration (e.g., login before fetch), but that is not documented.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter, url_path, has a schema description that covers 100% of its content (explains it's a relative path with example). The tool description adds no additional semantics beyond what the schema provides, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Fetch any SAP WebGUI URL and return the rendered page text' clearly states the verb (fetch) and resource (SAP WebGUI URL), and specifies the output (rendered page text). It distinguishes from siblings like sap_run_report or sap_execute_command, which perform other actions, though it does not explicitly contrast with them. The scope 'any' implies flexibility but not exact differentiation from other fetch-like tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance is provided on when to use this tool versus alternatives. The description does not mention exclusions, prerequisites (e.g., requiring sap_login first), or when to choose sap_execute_command or sap_run_report instead. The context of 'any URL' implies broad use, but there is no roadmap for decision-making.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sap_run_reportC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It only states the action and does not disclose potential side effects of running an ABAP report, whether output is returned, or if authentication is required. Running a report could have side effects, which is not addressed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence that immediately conveys the core purpose. There is no redundancy or wasted text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema and no annotations, the description should provide more context about expected outcomes, prerequisites, or limitations. The tool's nested parameter object adds complexity, but the description does not explain what happens after execution or how to handle output.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% because both parameters have detailed descriptions. The tool description itself adds no parameter details, but the schema fully documents them, meeting the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (Run) and resource (ABAP report/program) along with the transaction codes (SA38/SE38). It is specific but does not explicitly differentiate from sibling tools like sap_execute_command or sap_se16_browse.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives. The description implies usage through purpose but provides no exclusions, prerequisites, or context about choosing this tool over others.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sap_navigateB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description must disclose behavior fully. It mentions returning HTML page text but does not describe side effects like session state changes, need for authentication, or error handling. For a navigation action, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, concise and front-loaded with 'Navigate'. No wasted words. Could add a bit more context without hurting conciseness, but it is efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no output schema, the description is adequate but not complete. It lacks mention of prerequisites (like being logged in) or session implications, which are relevant given sibling tools like sap_login. Moderate completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with the 'transaction' parameter described clearly with examples. Description reiterates examples but does not add new semantics beyond schema. Baseline 3 applies because schema covers it fully.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states tool navigates to an SAP transaction code and returns HTML page text. It distinguishes from siblings like sap_login (authentication) and sap_get_page (getting a page) but could be more explicit about how it differs from sap_execute_command.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies usage: navigate to a transaction code to access a screen. No explicit guidance on when to use it vs. sap_execute_command or sap_get_page, nor prerequisites like requiring sap_login first. Adequate but lacks explicit alternatives or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sap_se16_browseA
  3.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden, and it does disclose two key behaviors: only the first page is returned, and the output is plain text. It does not mention authorization requirements or pagination follow-up, but for a simple read-only browsing tool this is meaningful transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one short sentence with no filler, front-loading the core purpose and immediately stating the return behavior. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter tool with a fully documented schema, the description is sufficient for basic selection and invocation: it names the action, resource, output type, and first-page limitation. Minor gaps exist around pagination and comparison with sap_rfc_read_table, but they do not undermine basic usability.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds little parameter-specific meaning beyond the phrase 'first page,' which loosely relates to max_rows; the schema already documents both table_name and max_rows with constraints and examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('browse'), resource ('database table'), and method ('via SE16 / Data Browser'), and it specifies the output ('first page... as text'). However, it does not explicitly differentiate this from the sibling sap_rfc_read_table, which likely also reads table data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided about when to use SE16 browse versus the sibling sap_rfc_read_table, nor are preconditions or exclusions mentioned. The description implies usage only through the tool name and purpose, leaving the agent to infer selection criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sap_execute_commandA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It clearly states that the tool submits arbitrary commands as if typed in the command field, but it does not disclose potential side effects, session requirements, or the fact that commands may navigate, mutate state, or log out. For an 'arbitrary command' tool, more cautionary transparency would be valuable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single focused sentence that front-loads the action and object. Every part contributes meaning, and there is no redundant filler or repetition of schema details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with one well-described parameter and an output schema is absent. However, the description omits guidance about side effects, when to use sap_execute_command versus sap_navigate, and what happens if no session exists. Given the tool's arbitrary-command nature, this leaves meaningful gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides 100% coverage for the single 'command' parameter with a clear description and examples ('/nSE11', '=back', '/nEXIT'). The tool description does not add extra parameter semantics beyond the schema, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the verb ('Submit'), the resource ('arbitrary command string'), and the target context ('current SAP WebGUI session'). The phrase 'as if typed in the command field' precisely conveys the tool's scope and distinguishes it from sibling tools like sap_navigate or sap_run_report.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool—whenever an arbitrary OK code or transaction command needs to be submitted to the current session. However, it does not explicitly state when to prefer this over alternatives like sap_navigate, nor does it mention prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sap_rfc_read_tableA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the underlying mechanism (RFC_READ_TABLE over SOAP), authentication (Basic Auth), and statelessness (no session). However, it does not describe return format, error handling, or potential side effects, which are notable gaps for a read operation without an output schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, opening with the core purpose and followed by a concise feature list. Every word adds value; no redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has moderate complexity (5 params, no output schema). The description covers key aspects like filtering, pagination, and auth, but omits the shape of the returned data, which a user might need to know to integrate with the tool. Given the lack of an output schema, this is a notable gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides descriptions for all five parameters (100% coverage), so the baseline is 3. The description reinforces the purpose of fields, where, max_rows, and row_skips at a high level but adds no new per-parameter detail beyond what the schema offers.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: reading data from any SAP transparent database table via RFC_READ_TABLE over SOAP. It enumerates specific capabilities (field selection, WHERE filters, row limits, offsets) and distinguishes itself from siblings by noting it requires no session and uses Basic Auth directly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool ('No session required — uses Basic Auth directly'), implying it is suitable when no SAP session exists. However, it does not explicitly mention when to avoid it or name alternatives among siblings (e.g., sap_se16_browse), leaving usage guidance partially implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sap_loginA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the full burden and does well: it discloses side effects (logs in), prerequisite behavior (must be called first), and return value ('Returns the welcome screen text on success'). It falls slightly short by not describing failure modes or authentication error handling, otherwise this would be a 5.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each with a distinct purpose: core action + prerequisite, ordering constraint, and return value. Zero wasted words; everything earns its place. This is a model of concise, front-loaded technical writing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with moderate complexity, this covers all critical aspects: purpose, prerequisite, ordering, and return value. It could add failure/error details, but given the tool's simplicity and the absence of an output schema, this is more than sufficient for an agent to use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, placing us at the baseline of 3. However, the description goes beyond by explaining that the username and password default to environment variables (SAP_USERNAME/SAP_PASSWORD), which isn't just a repeat—it's the key to understanding why the parameters are optional. This adds genuine semantic value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource ('Log in to the SAP system via WebGUI'), instantly distinguishing it from the sibling tools, which all start with different actions (navigate, browse, run, etc.). It clearly states the tool handles authentication and provides a success condition, leaving no ambiguity about its purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'Must be called before using any other SAP tool' provides an explicit when-to-use directive and establishes a hard ordering requirement. While it doesn't name alternatives, the sibling list makes it obvious no other tool handles login, making this guidance highly effective.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.