Server Quality Checklist

Tool Scores

• resolve_fabric_idsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden. It discloses the core conversion behavior but does not explicitly state whether the operation is read-only, requires authentication, or has any side effects. The non-destructive nature is implied but not clearly disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the verb, and every word serves a purpose. No waste or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple and the description covers purpose and usage, but there are gaps: parameter semantics are not explained and the return behavior is not described beyond 'display names.' The output schema exists but is not referenced or enriched. Adequate but with clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage. The description mentions GUIDs and URLs, but it does not map workspace_id or model_id to specific URL components or explain what model_id (the optional parameter) is for. Users are left to guess parameter meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Turn' and clearly states the resource: GUIDs in a Fabric URL converted to display names. It distinguishes this as a resolver tool from sibling tools like inspect_model or show_measure.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use: 'when the user pastes a Fabric Studio link instead of typing names.' It does not mention when not to use or list alternatives besides the implied direct-name input, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• show_measureA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It discloses the dependency on inspect_model and the flexible input formats, which is useful. However, it does not explicitly state that the operation is read-only or describe any side effects, although 'Show' strongly implies non-mutating. It also omits error behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short sentences, with the purpose in the first sentence and supporting details following. No redundant phrases, and every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description adequately covers the main required parameter and the prerequisite call, and the output schema presumably handles return values. The only notable gap is the lack of explanation for 'model_label', but for a read-only display tool this is a minor omission.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It thoroughly explains the 'measure' parameter with accepted formats. However, the optional 'model_label' parameter is not mentioned at all, leaving a gap for a two-parameter tool. The partial explanation earns a score below the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Show the current DAX expression of one measure') and the resource ('the table it belongs to'). This distinguishes it from sibling tools like 'update_measure_dax' (which modifies) and 'inspect_model' (which inspects the whole model).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides an explicit prerequisite ('Call inspect_model first') which is a clear usage guideline. It also specifies the accepted input formats, helping the agent invoke it correctly. It does not explicitly mention alternatives, but the dependency and format hints are sufficient for typical use cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ask_agent_toolA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It adds useful context by explaining that the agent's instructions, knowledge, and tools live server-side, so the caller does not need to understand its internals. It does not mention potential latency, side effects, or authentication requirements, but for a straightforward ask-and-answer interaction this is acceptable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded: the core purpose is in the first sentence, followed by brief usage context and a clear exclusion. Every sentence adds value; there is no redundant or repeated information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter tool with an output schema and few siblings, the description provides sufficient context: what it does, when to use it, how to phrase input, and when to avoid it. The explicit alternative and clear scope make it safe for an AI agent to select and invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has only a `prompt` string with no description (0% coverage). The description compensates by instructing the agent to pass the user's request in plain language and clarifying that the agent handles the domain semantics. This adds meaningful guidance for the single parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the action ('Ask the Microsoft Foundry agent a question') and the result ('return its answer'). It also distinguishes itself from sibling tools by explicitly warning against using it for Power BI model reads, for which `inspect_model` is the appropriate alternative.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It tells the agent when to use the tool ('anything the agent is specialised in') and how ('Pass the user's request in plain language'). It also provides an explicit exclusion: 'Do NOT use this to read a Power BI model' and names the better alternative, `inspect_model`.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_measure_daxA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully owns the behavioral disclosure. It highlights the model-modifying nature, the double opt-in (confirm flag and write access), and the validation behavior ('an invalid expression is rejected rather than saved'), providing essential context beyond any schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact, front-loaded with the core action and a prominent warning, and every sentence contributes value. It covers purpose, permissions, workflow, and validation in five sentences without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple mutation tool and the presence of an output schema, the description provides all essential context: what it does, when to use it, prerequisites, and safety behavior. The inclusion of permission requirements and validation makes it complete enough for an agent to select and invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has zero property descriptions (0% coverage), so the description must compensate. It explicitly explains the critical confirm parameter ('must pass confirm=True') and implies measure/new_expression through the main sentence, but it does not mention model_label or provide format/syntax details. Partial coverage, not comprehensive.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Write a new DAX expression for a measure.' It also clearly flags 'MODIFIES THE MODEL,' which distinguishes this from the read-only siblings like show_measure. This is unambiguous and differentiates the tool's intent.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs to use show_measure first: 'Always show the user the current expression with show_measure and get their agreement before calling this.' It also details the exact conditions required (confirm=True and write access), making it clear when and how to use this tool versus alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• inspect_modelA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description fully carries behavioral disclosure. It explicitly states the tool is read-only ('never modifies the model'), explains authentication and row-level security implications, and mentions the summarisation behavior. No contradictions with annotations exist.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with the core purpose, followed by parameter modes and safety/usage notes. Every sentence adds value, with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers purpose, usage modes, safety (read-only), authentication effects, and sequencing. The output schema exists, so return values do not need elaboration. For a tool with 3 optional parameters and two usage modes, this is complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description compensates fully by explaining the two modes: workspace+model (by display name or GUID) or folder path. This directly maps to the three parameters (workspace, model, folder) and gives meaningful context for each.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool connects to a Power BI semantic model and summarises its contents. It distinguishes itself from siblings by explicitly noting it runs before `show_measure`, which helps differentiate it from related operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit usage context: 'Run this before `show_measure`' indicates the appropriate timing. It also explains the two possible input modes (Fabric workspace+model or local folder), giving clear guidance on when and how to use the tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.