Server Quality Checklist

Tool Scores

• list_btp_instancesA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses that the tool lists all instances shows states (SUCCEEDED, FAILED, IN_PROGRESS). It does not explicitly state it is read-only or mention limitations like pagination or rate limits. Given no annotations, this is adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded: first sentence states core purpose, then defines instance, shows states, and provides examples. No unnecessary words or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers purpose, usage examples, and state information. The output schema handles return values. However, the lack of any parameter explanation leaves a gap in completeness, as the agent may not know how to filter by service name.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has one parameter (service_name) with no description in the schema (0% coverage) and no explanation in the tool description. The agent must infer its filtering purpose from the name alone, risking misuse or underutilization.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'lists' and the resource 'service instances', with specific scope 'in your BTP subaccount'. It distinguishes from sibling tools like list_btp_services (services) and get_btp_destinations (destinations) by focusing on instances.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit example queries to guide usage, such as checking for existing instances or failed ones. However, it does not mention when not to use this tool or suggest alternative tools for different needs (e.g., listing services), leaving some ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_btp_service_plansA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions that some plans have a free tier and implies a read-only operation (before creation), but does not explicitly state no side effects or authorization needs. It adds some behavioral context but is not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences with a line break, front-loading the primary purpose. Every sentence adds value: the first states the core function, and the second adds practical usage guidance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given a single parameter with 100% schema coverage and an output schema present, the description covers the essential purpose and usage. It could mention what the output contains, but the output schema likely handles that. The description is complete for this simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and the input schema already describes the parameter well, including examples. The description does not add significant new meaning beyond what the schema provides; it repeats the concept of plan options but adds no syntax or additional constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb and resource: 'Returns all available plans for a specific BTP service.' It distinguishes from sibling tools like list_btp_services and get_btp_destinations, which focus on services, instances, or destinations rather than plans.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage context: 'Use this to understand what options exist before creating an instance.' It notes that plans differ in capacity, features, and cost, helping the agent decide when to call this tool. No explicit when-not or alternatives, but the context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_btp_destinationsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It implies a read operation ('Lists all destinations') but does not explicitly state readonly behavior, potential performance, or error conditions. Adequate but not detailed beyond that.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a clear intro, then filter options, then usage examples. Every sentence serves a purpose, no waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 optional params, output schema exists), the description covers purpose, parameters, and usage adequately. It could mention the return format briefly, but output schema presence mitigates that.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description adds meaningful value by explaining the filter options (proxy_type: 'OnPremise' or 'Internet'; auth_type: e.g., 'OAuth2ClientCredentials') with examples, compensating for the schema's lack of descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it lists all destinations in a BTP subaccount, using specific verbs and resources. It distinguishes from sibling tools like get_btp_service_plans by focusing on connections to external systems.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides example queries for when to use the tool, such as 'What destinations are configured in BTP?' but does not explicitly state when not to use it or mention alternative tools for similar tasks.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• recommend_btp_serviceA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses that the tool checks for running instances and that entitlement checking is temporarily disabled (is_entitled always null), providing useful behavioral context beyond no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise, well-structured description with examples and a note about disabled feature; no unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With one parameter, an output schema, and clear purpose, the description is fully adequate for an agent to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter 'use_case' has no schema description, but the description adds meaning by listing example queries, effectively showing valid input forms.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool recommends BTP services for a described use case and checks existing instances, distinguishing it from sibling tools like list_btp_services.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit examples of when to use the tool (e.g., 'What BTP service should I use for async messaging?'), giving clear context; does not explicitly exclude any cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_btp_servicesA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It explains the tool lists existing services not necessarily running, and mentions optional keyword filtering. Additional details on pagination or order would improve transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, with a clear header, differentiation, filter details, and usage examples. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has an output schema, the description adequately covers the tool's purpose and filter capability. It could mention if there are limits or default ordering, but overall it is complete enough for selection.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With schema coverage at 0%, the description adds critical meaning: it explains the keyword parameter searches name, description, and tags, which the schema alone (empty description) does not provide.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists SAP BTP services from the global catalog and distinguishes itself from the sibling tool list_btp_instances by stating it shows services that exist, not necessarily provisioned. Examples further clarify purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit usage guidance is provided: use this tool for available services, and use list_btp_instances for provisioned ones. Example queries give concrete context for when to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.