Server Quality Checklist

Tool Scores

• run_reportA
  3.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states that the tool runs a report and returns output, but does not disclose whether this operation is read-only, whether specific permissions are required beyond the fusion_user parameter, whether the report may have side effects, or any potential failure modes. The parameter explanations add some context (e.g., fusion_user runs as), but key behavioral traits are left unstated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (two sentences plus a short arg list) and front-loaded with the main action. Every sentence earns its place; the Args block provides necessary parameter details without unnecessary prose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with only two parameters and an output schema, the description is reasonably complete. It covers the purpose and both parameters adequately. However, it does not mention when to use this vs. alternatives or discuss error scenarios, making it slightly incomplete in an agent context. Given the output schema exists, the lack of return-value detail is acceptable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, but the description compensates fully by explaining both parameters. 'Full catalog path, e.g. /Custom/MyReports/MY_RPT.xdo' gives a concrete example, and 'Fusion user to run as (default bip_user)' clarifies the default and purpose. This adds significant value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action: 'Run a deployed BIP report and return its output.' The verb 'Run' and resource 'deployed BIP report' are specific and distinguish it from sibling tools like deploy_report (which deploys, not runs) and query_fusion (which queries data).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies that the report must already be deployed (it says 'deployed BIP report'), so an agent can infer that deploy_report should be used first if the report is not yet deployed. However, it does not explicitly state when to use this tool over alternatives like query_fusion for data retrieval, nor does it provide exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• deploy_data_modelA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full disclosure burden. It explicitly states 'WRITE operation', describes destructive behavior via 'replace: Delete any existing model of the same name first', and mentions the required BIP Author role for fusion_user. This is substantive behavioral disclosure beyond simple mutation, though it omits details about failure modes when replace=false.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a two-sentence overview followed by a bulleted list of parameters. Each line is purposeful and directly useful; no fluff or repetition. The length is appropriate for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the core purpose, parameters, and critical behavioral constraints (write, replace, auth). An output schema exists, so return value details are not needed. It lacks some edge-case context like folder existence requirements or behavior when replace is false, but overall it is sufficient for an agent to deploy a data model safely.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must fully compensate. It provides meaningful semantics for all parameters: name formats, xdm_xml content guidance (ApplicationDB_FSCM), folder examples, replace behavior, and fusion_user role requirement. This significantly exceeds basic schema field names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action: 'Deploy a BIP data model (.xdm) to a catalog folder.' It identifies the resource (data model), format (.xdm), and destination (catalog folder), which distinguishes it from sibling tools like deploy_report. The verb 'deploy' is precise and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by stating the tool's purpose, but it does not explicitly contrast with alternatives like deploy_report or provide when-not-to-use guidance. It gives clear context for deployment but no exclusions or alternative tool references.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• deploy_reportA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It discloses the write nature, the destructive replace default, and the BIP Author role requirement, which are critical behavioral traits beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a compact docstring with a clear summary line followed by a structured Args listing. No redundant text; every line adds needed detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a write tool with an output schema, the description covers purpose, parameters, destructive behavior, and role prerequisites. It misses explicit guidance on when to prefer other tools, but is otherwise complete given the complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Even though schema coverage is 0%, the Args section provides detailed meaning for every parameter: name restrictions, an example dm_path, the replace behavior, and the default fusion_user role. This far exceeds the schema's type-only info.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool deploys a BIP report (.xdo) linked to a data model, uses a specific verb 'Deploy', and calls out 'WRITE operation' to distinguish it from read-oriented sibling tools like query_fusion and run_report.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description doesn't explicitly state when to use this tool over alternatives or provide exclusion criteria. It implies usage through the 'WRITE operation' note and the replace parameter behavior, but no direct comparison to sibling tools is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_fusionA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and does well: it discloses read-only safety, warns that BIP serializes the entire result set (implying a need for row limits), and specifies single-statement SQL with no trailing semicolon. It doesn't mention authentication or error cases, but the fusion_user parameter and default help cover access.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a one-sentence summary leads into a clear Args list with specific details for each parameter. No unnecessary words; every line adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers purpose, read-only behavior, output format, serialization risk, and parameter constraints. An output schema is present, reducing the need to detail return values. Missing items like error handling and authentication are minor given the fusion_user parameter and default, and the tool appears adequately specified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must fully explain parameters, and it does. For sql, it explains syntax criteria and result-size implications; for columns, it dictates aliasing and ordering; for fusion_user, it gives the default. This far exceeds the bare schema and provides all necessary operational meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource+operation: 'Run a read-only SELECT against Oracle Fusion and return the rows as XML.' This clearly distinguishes it from sibling tools like run_report or deploy_report, which handle predefined reports and data models, while query_fusion is for direct SQL queries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides practical parameter usage guidance (e.g., 'Add WHERE/ROWNUM to keep the result small') but does not explicitly state when to use query_fusion versus alternatives like run_report or get_catalog_object. The read-only SELECT nature implies ad-hoc querying, but no when/when-not comparisons are given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_catalog_objectA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. The description indicates a read-only download operation and discloses the fusion_user impersonation behavior in the Args section. It does not discuss error handling or authentication, but the download semantics are transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, with a single sentence plus a short parameter list. All information is relevant and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple two-parameter interface, the presence of an output schema, and the clear download behavior, the description provides adequate context. It lacks explicit mention of error conditions or related tool selection, but is sufficient for a basic read tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description's Args section provides a concrete example for object_path and explains the default for fusion_user, adding meaning beyond the schema's bare property names. This compensates for the 0% schema description coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Download') and resource ('catalog object definition (.xdm or .xdo)') and clarifies the purpose ('for inspection'). This clearly distinguishes it from siblings like deploy_report or query_fusion.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'for inspection' implies the tool is intended for viewing or examining object definitions. However, it does not explicitly mention alternative tools or when not to use this tool, so the guidance is clear but not exclusionary.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.