Server Quality Checklist

Tool Scores

• execute_odata_queryA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses that GET returns CSV and POST/PATCH/DELETE return JSON, which is valuable beyond schema. However, with no annotations, description carries full burden; it omits side effects, permissions, error handling, or idempotency. Adds some context but incomplete for a CRUD tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with core purpose, no wasted words. Efficiently conveys return format and usage recommendations.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given tool complexity (10 params, CRUD, no output schema), description is adequate but has gaps: does not mention pagination behavior (top/skip are in schema but not described), single-record vs list retrieval via entityKey, or error handling. Points to siblings for discovery but could be more complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. Description does not add significant meaning beyond schema; it only hints at using sibling tools for valid input values. No parameter-specific enrichment in description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it executes OData CRUD operations against SAP S/4HANA, with specific verb and resource. Distinguishes from sibling tools discover_sap_services, get_entity_metadata, and get_field_values, which are used for preparation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly recommends using sibling tools before write operations for discovery of service names, field names, and valid input values. Provides clear context but does not cover all usage scenarios (e.g., when to use GET vs POST is inferred from method parameter).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• discover_sap_servicesA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must cover behavioral traits. It describes the discovery operation as safe and returns names, titles, and descriptions, but lacks details on limitations (e.g., default top of 50), error handling, or what happens if no results are found.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences: first states purpose, second gives usage guidance. No wasted words, highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema exists, so the description should compensate by explaining the return structure. It mentions returning names, titles, and descriptions but doesn't specify if it's a list or the format. Also, it could clarify how the output feeds into sibling tools, though it's implied.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with both parameters described. The description adds no additional meaning beyond the schema, so baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool discovers OData services from SAP Gateway, returning names, titles, and descriptions. It also distinguishes itself from sibling tools by specifying to use it first when the service name is unknown.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states to use this tool first when you don't know the service name to pass to execute_odata_query or get_entity_metadata, providing clear guidance on when to use it vs alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_field_valuesA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must fully disclose behavior. It states it fetches options but omits details like read-only nature, distinctness, or error handling. While the input schema covers parameters, the description itself adds minimal behavioral context beyond the core function.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences efficiently convey the tool's purpose and usage. Every word earns its place; no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (5 params, no output schema), the description ties the tool to sibling 'execute_odata_query' and explains the value-help concept. However, it omits the return format (array of value-label pairs) and pagination behavior of the 'top' parameter, leaving some ambiguity for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All 5 parameters have descriptions in the input schema (100% coverage), so the description's mention of 'entity set that acts as the value-help source and the target field name' adds no new semantic meaning. It restates schema info without deepening parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Fetches') and resource ('dropdown and value-list options from a SAP S/4HANA OData entity'). It effectively distinguishes itself from siblings like 'execute_odata_query' and 'get_entity_metadata' by focusing on value discovery before writing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly directs when to use the tool ('before writing data with execute_odata_query') and provides actionable guidance on how to provide the entity set and target field. This fulfills a high standard of usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_entity_metadataA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, description explains the tool returns metadata (entity sets, fields, navigation properties) and implies no side effects. Could be more explicit about being read-only or requiring authentication, but sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences: first states purpose, second gives usage guidance. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Completeness is high given no output schema: describes return content and provides usage context. Could mention authentication or scope, but not critical for a metadata retrieval tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 100% coverage for the single 'service' parameter, and description provides an example (API_BUSINESS_PARTNER) adding some value beyond the schema description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it fetches and summarizes OData service metadata from SAP S/4HANA, returning entity sets with key fields, properties, and navigation properties. This distinguishes it from sibling tools like execute_odata_query.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use this before calling execute_odata_query to understand the available fields and correct entity names,' providing clear context and alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.