Server Quality Checklist

Tool Scores

• sap_help_get_pageA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden for behavioral disclosure. It mentions 'cleaned text content,' which indicates the output is processed rather than raw HTML, but it does not disclose potential limitations such as authentication requirements, error handling, or rate limits. This is a moderate level of transparency, neither minimal nor comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, each earning its place: the first states the core function, and the second gives a usage hint and output type. No unnecessary words, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite the lack of an output schema and annotations, the description is sufficient for a simple tool with one parameter. It clearly states the input type and the cleaned-text output, and the context signals show there are no nested objects or enums. It lacks examples or error scenarios, but for this complexity, the description is nearly complete, earning a 4 rather than a 3.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description for the 'url' parameter covers its meaning and format ('URL or path of the SAP Help page'), so schema coverage is 100%. The tool description does not add additional parameter details beyond saying to use URLs from search results, which is a minor contextual hint. Thus, the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Fetch' and identifies the resource 'full content of a specific SAP Help page,' which clearly distinguishes it from sibling tools that search or list products. It also notes that it returns 'cleaned text content,' adding clarity about the output.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The instruction 'Use URLs from search results' provides a clear context for when to invoke this tool, implying it is complementary to search tools. It does not explicitly name alternative tools, but the sibling names make this differentiation obvious, and the guidance is sufficient for selecting this tool after a search.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sap_help_searchA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the burden of behavioral disclosure. It does reveal the shape of the return data (titles, URLs, snippets, product info), which is useful. However, it does not mention whether the operation is read-only, any rate limits, or authentication requirements. For a search tool, the behavior is largely obvious, but there is room for more transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the core action ('Search SAP Help Portal documentation'), and every word contributes meaning. It efficiently covers what the tool does, what it returns, and when to use it without any fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the essential return values (titles, URLs, snippets, product info), which is important given the lack of an output schema. It also captures the broad scope ('across all SAP documentation'). It does not explicitly mention optional filters, but those are fully described in the schema. Overall, the description is complete enough for a search tool with well-documented parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, meaning all five parameters have individual descriptions in the schema itself. The tool description does not add additional parameter semantics beyond what the schema already provides, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches SAP Help Portal documentation using keywords and that it returns topic titles, URLs, snippets, and product info. It distinguishes itself from the sibling tools (semantic search, list products, get page) by explicitly mentioning 'specific keyword searches'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: 'Use this for specific keyword searches across all SAP documentation.' It does not explicitly name alternatives or exclusions, but the context implies when this tool is appropriate versus semantic search or listing products. This is clear context without explicit when-not guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sap_help_semantic_searchA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description bears the full burden of behavioral disclosure. It adds that the tool 'returns ranked results with relevance scores,' which is useful context beyond basic search, but lacks details on limitations, errors, or pagination behavior. This is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core purpose and key differentiator ('semantic search') followed by an example and output behavior. Every sentence earns its place with no waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (5 simple parameters, no output schema), the description explains the return format and the core value proposition. It does not mention filtering options (product, version, language) but these are fully documented in the schema, so the description is sufficiently complete for invoking the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers 100% of parameters with descriptions, so the baseline is 3. The tool description adds the semantic-search context ('natural language') that aligns with the query parameter, but it does not introduce additional meaning beyond what the schema already provides for other parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('search') and resource ('SAP Help Portal') with a clear differentiator: 'AI-powered semantic search.' It contrasts with sibling tools like sap_help_search by emphasizing natural language queries, making its purpose distinct and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'Better for natural language questions' provides clear context for when to use this tool versus alternatives. However, it does not explicitly exclude cases where keyword search might be preferable or name alternatives, so it lacks full usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sap_help_list_productsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It states the scope ('all available') and that it returns names and IDs, but it does not mention pagination, rate limits, authentication, or other behavioral details. For a simple list read operation, this is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two clear, front-loaded sentences with no fluff. Every word adds value: the action, the resource, the return content, and the use case.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the core purpose and return information. Given the tool's simplicity (no params, no output schema), it is reasonably complete, though it could mention output format or any list limits.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema is trivially complete. The description does not add parameter info, but none is needed. Baseline for 0 params is 4, and no contrast exists.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and a clear resource ('available SAP products on the Help Portal'). It also explicitly states the return value (product names and IDs) and their purpose (filter searches), which distinguishes this tool from the sibling search tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a clear use case: 'can be used to filter searches.' This implies when an agent needs product identifiers to constrain search results, but it does not explicitly mention alternatives or when not to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.