Server Quality Checklist

Tool Scores

• entra_list_usersA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and non-destructive behavior. The description adds valuable behavioral context beyond this: pagination via `has_more` and `offset`, and the required Graph application permission User.Read.All. This is useful, actionable transparency, though it does not discuss rate limits or failure modes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is tight and front-loaded: one sentence states the core purpose, a second gives practical use cases, and the final two sentences add essential operational details. Every sentence earns its place, and there is no redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With a rich output schema, the description does not need to explain return values. It covers purpose, typical use cases, pagination behavior, and required permissions, and is paired with a fully documented parameter schema. An agent has enough information to select and invoke this tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and the schema already documents limit, offset, department, and enabled_only thoroughly, including exact-match semantics and the audit-waste nuance for enabled_only. The description adds some context around paging, but does not materially expand parameter meaning beyond the schema, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: "List accounts in the Microsoft 365 tenant with the licences each one holds." It is clearly distinct from the sibling tools, especially by emphasizing the per-user license information and by framing it as a first step before deeper user-specific investigation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives concrete use cases: answering "who works here", "who is in Finance", and as a first step before looking at a specific person. It provides clear context but does not explicitly name alternatives or when-not-to-use conditions relative to the sibling tools, so it stops just short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• entra_find_stale_accountsA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnly/idempotent annotations, the description discloses meaningful behavior: it joins user list, sign-in activity, and licence SKUs; lists disabled accounts first as pure waste; requires specific Graph permissions; and fails gracefully with an explanatory message when Entra ID P1 is missing. This is rich, non-obvious context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Every sentence earns its place: purpose, business question, behavior, ordering, and prerequisites. The description is front-loaded with the core function and remains compact despite covering several important behaviors.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is complete for a read-only reporting tool: it states what data is joined, what output is produced, what permissions are needed, and the tenant licensing prerequisite. An output schema exists, so return-value details are not required.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and each parameter already has a clear meaning. The description does not add parameter-level detail beyond what the schema provides, so the baseline of 3 applies without additional credit.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Find accounts that have not signed in recently and the licences they hold.' It then clarifies the real-world question it answers and explains the data join, making it clearly distinct from siblings like entra_list_users or entra_get_license_usage.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly frames when to use the tool: it answers 'which licences are we paying for that nobody uses?' and describes the reporting output. It does not explicitly name sibling alternatives or state when not to use it, but the use case is clear enough to route an agent correctly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• entra_get_userA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare read-only, idempotent, and non-destructive behavior. The description adds value by specifying required permissions (User.Read.All and Directory.Read.All) and by enumerating exactly what data is returned (profile, manager, group memberships, licences), which is not visible in the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each earning its place: the core action, the usage context, and the required permissions. It is front-loaded with the primary purpose and contains zero filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter, read-only tool with an output schema present, the description provides everything an agent needs: what it does, when to use it, and what permissions are required. Nothing essential is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides 100% coverage for the single 'user' parameter with a clear description (UPN or object id). The description adds no additional parameter-level detail beyond reinforcing that this is for a specific person, which is already implied by the parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb ('Get'), resource ('one account's profile, manager, group memberships and licences'), and scope that distinguishes it from sibling tools like entra_list_users, which lists multiple users. The phrase 'one account' immediately signals this is the single-user counterpart.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly names the preceding tool (entra_list_users) and gives the condition for when to use this tool: 'when a question is about a specific person.' This provides clear routing guidance and is the strongest form of usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• entra_get_license_usageA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, covering the safety profile. The description adds meaningful context beyond annotations by explaining the concept of unassigned seats and their savings value, as well as the required permission. This enriches the agent's understanding without contradicting the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-structured: it leads with the core purpose, adds a practical insight about unassigned seats, routes the agent to a sibling tool for a related case, and closes with the permission requirement. Every sentence earns its place; there is no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter read-only tool with an output schema present, the description covers the purpose, the usage context, the alternative tool, and the required permission. There is nothing meaningful missing for an agent to decide whether to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is no parameter documentation burden. The schema is vacuously complete at 100% coverage, and the baseline for no-parameter tools is 4. No additional parameter explanation is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Show, per licence SKU, how many seats were purchased and how many are assigned.' It goes beyond a generic label and distinguishes itself from siblings by naming entra_find_stale_accounts as the tool for a different scenario, while the other siblings focus on users rather than license SKU seat counts.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It clearly states when to use this tool: to see purchased vs assigned seats per SKU, highlighting unassigned seats as the cheapest saving. It also provides an explicit exclusion: for seats assigned to people who never use them, call entra_find_stale_accounts instead. Additionally, it specifies the required permission, Organization.Read.All.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.