Server Quality Checklist

Tool Scores

• get_dataset_detailsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It mentions raising ToolError for invalid input, but does not disclose read-only nature, permissions, or other behavioral traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Reasonably concise with a docstring-style format. Each sentence adds value, though the Args/Raises section slightly expands length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers parameters and error conditions, but does not explain return values (output schema exists) or provide details on what 'metadata and structure' includes. Adequate for a simple retrieval tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has no descriptions (0% coverage), but the description adds meaningful context by specifying UUID format and purpose for both parameters, compensating for the schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves dataset metadata and structure, with a specific verb and resource. It is distinct from siblings like list_datasets_in_workspace, but does not explicitly differentiate itself.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives like list_datasets_in_workspace or execute_dax_query. Does not mention prerequisites or conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_datasets_in_workspaceA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description must cover behavioral traits. It only states 'List datasets' and describes the input, but omits any output behavior (e.g., pagination, whether the list includes all datasets, or any side effects). Even though an output schema exists, the description does not reference it or any operational characteristics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded with the core purpose. It includes a structured Args and Raises section, using minimal sentences without any filler. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the existence of an output schema (covering return structure), the description adequately explains the input and error handling. However, it could mention that it lists all datasets in the workspace or any limitations, though the output schema compensates somewhat. Still, it is mostly complete for a simple list operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, but the description adds meaningful information for workspace_id: 'The unique identifier of the Power BI workspace (UUID format)'. This clarifies the required format beyond the schema's plain type: string, making it easier for the agent to correctly invoke the tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action 'List datasets' and the context 'in the specified workspace', using a specific verb and resource. It distinguishes from siblings like get_dataset_details (single dataset) and execute_dax_query (querying), making the tool's purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives (e.g., get_dataset_details). It only mentions the required parameter and error, but no context for selection, such as 'use this to get a high-level overview' or contrasting with sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• execute_dax_queryA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses common errors, limitations (max rows/values), and that query must start with EVALUATE. Does not explicitly state idempotency or read-only nature, but provides substantial behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with clear sections (Args, Returns, Errors, Example). Slightly verbose but front-loaded with purpose. Each section adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of DAX query execution and lack of annotations, the description is complete: covers parameters, errors, limitations, and example. Output schema exists but description adequately describes return types.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description thoroughly explains each parameter: workspace_id/dataset_id as UUIDs, dax_query must start with EVALUATE. Includes example, adding significant meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool executes DAX queries against Power BI datasets, using specific verbs and resource. It distinguishes itself from siblings which are about listing workspaces/datasets.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for executing DAX queries but does not explicitly state when not to use it or provide alternatives. However, siblings are clearly distinct, so confusion is minimal.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• powerbi_list_workspacesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Describes the return format (IDs and names) and common errors, implying a read-only listing. Without annotations, it carries the burden well, though no side-effect disclosure is needed for a list operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short paragraphs with clear front-loading: purpose and return first, errors second. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Adequate for a simple list tool with an output schema. Covers workspace identification and errors, but could mention pagination or rate limits for completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, so schema coverage is 100%. The description adds value by specifying the scope ('the user has access to') and return structure (workspaces with IDs and names).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states 'List all Power BI workspaces the user has access to' with a specific verb and resource, and differentiates from sibling tools like get_workspace_id and list_datasets_in_workspace.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides context that it is useful for identifying accessible workspaces, but lacks explicit guidance on when not to use or alternatives. Includes common error codes for troubleshooting.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_workspace_idA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, but description discloses that the tool raises a ToolError if not found. It's a simple get operation with no destructive side effects; description sufficiently covers behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is concise with a brief summary and structured Args/Returns/Raises sections. No wasted sentences, front-loaded with purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 1-param tool with output schema, description covers purpose, parameter semantics, return value, and error case. No missing information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has no description for parameter workspace_name (0% coverage). Description provides meaning: 'The display name of the Power BI workspace.' Fully compensates for schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it gets the workspace ID for a given workspace name. Distinguishes from sibling tools like powerbi_list_workspaces which likely list all workspaces, whereas this is a lookup by name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says when to use: when only the workspace name is known, and explains that the ID is needed for other operations. Does not explicitly mention when not to use, but context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.